Bug #20806 Random disconnect
Submitted: 1 Jul 2006 22:42 Modified: 4 Aug 2006 8:39
Reporter: Dmitri Toubelis Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:4.1.20 OS:Linux (Centos 4.3)
Assigned to: CPU Architecture:Any

[1 Jul 2006 22:42] Dmitri Toubelis
Description:
I'm using Mysql with Asterisk FreePBX and I'm getting DB errors way too frequently. It also happens when using phpMyAdmin. I looked at Mysql log file and it is full of errors like this:

060701 17:50:29  mysqld started
060701 17:50:29  InnoDB: Started; log sequence number 0 43634
/usr/libexec/mysqld: ready for connections.
Version: '4.1.12'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=5
max_connections=100
threads_connected=4
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x93188f8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xb12753dc, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x813198c
0xcd7888
0x9318a88
0x8179cf3
0x817a366
0x81473dc
0x8148dfa
0x8149638
0x814a263
0x814ab5a
0xcd1371
0xbd29be
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x9319c60 = SELECT * FROM applications
thd->thread_id=81
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0
060701 18:09:01  mysqld restarted
060701 18:09:02  InnoDB: Started; log sequence number 0 43634
/usr/libexec/mysqld: ready for connections.
Version: '4.1.12'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution

The table name may vary and I do not see any pattern. The tables are MyISAM and I run recovery and optimization on each of them. And it gives different error messages time after time. The ones I remeber are #2002 and something about disconnected in the middle of the query.

How to repeat:
I'm getting this consistently in my environment which has been installed from asterisk-2.8.iso and updated with yum to the latest patch level. I guess it can be repeated this way. The machine is P-III, 1GHz with Intel mitherboard.

Suggested fix:
I tried enabling "query cache" as someone suggested for similar problems and tied setting "thread cache size" to 20-40. Nothing really helps.
[3 Jul 2006 11:19] Valeriy Kravchuk
Thank you for a problem report. Please, use only latest version, 4.1.20, and, for it, send resolved stack trace for each of the crashes like that you described (as explained in http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html). SHOW CREATE TABLE and SHOW TABLE STATUS for each table mentioned in the SQL statement that crashed server will be also useful.
[4 Jul 2006 7:56] Dmitri Toubelis
This is same log report but from 4.1.20:

060704 03:39:43  mysqld started
060704  3:39:44  InnoDB: Started; log sequence number 0 43634
/usr/libexec/mysqld: ready for connections.
Version: '4.1.20'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=1
max_connections=100
threads_connected=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x9e230d0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xb133139c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8136da4
0xcd7888
0x817ac8b
0x81801c9
0x8180894
0x814cb82
0x814e66c
0x814f08a
0x814f8e5
0x8150330
0xcd1371
0xbd29be
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x9e729b0 = SELECT * FROM modules WHERE modulename = 'recordings'
thd->thread_id=10
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0
060704 03:41:14  mysqld restarted
060704  3:41:14  InnoDB: Started; log sequence number 0 43634
/usr/libexec/mysqld: ready for connections.
Version: '4.1.20'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution

=== SHOW CREATE TABLE modules ===

CREATE TABLE `modules` (
  `id` int(11) NOT NULL auto_increment,
  `modulename` varchar(50) NOT NULL default '',
  `version` varchar(20) NOT NULL default '',
  `enabled` tinyint(4) NOT NULL default '0',
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

=== SHOW TABLE STATUS modules ===

+---------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+
| Name    | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time         | Check_time          | Collation         | Checksum | Create_options | Comment |
+---------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+
| modules | MyISAM |       9 | Dynamic    |   22 |             23 |         524 |      4294967295 |         2048 |         0 |             27 | 2006-05-27 19:08:47 | 2006-07-01 02:30:59 | 2006-07-01 02:30:59 | latin1_swedish_ci |     NULL |                |         |
+---------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+
1 row in set (0.01 sec)
[4 Jul 2006 8:39] Valeriy Kravchuk
Thank you for additional information. Please, send the resolved stack trace also.
[4 Aug 2006 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".