| Bug #19154 | MySQLD Core, glibc detected double free or corruption (fasttop) | ||
|---|---|---|---|
| Submitted: | 17 Apr 2006 23:47 | Modified: | 10 May 2006 16:07 |
| Reporter: | Jonathan Miller | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Cluster: Cluster (NDB) storage engine | Severity: | S1 (Critical) |
| Version: | 5.1.10 | OS: | Linux (Linux 32 Bit OS) |
| Assigned to: | Jonas Oreland | CPU Architecture: | Any |
[17 Apr 2006 23:47]
Jonathan Miller
[17 Apr 2006 23:50]
Jonathan Miller
060417 19:32:32 [ERROR] NDB tablespace: distributing ./ts1 timed out. Ignoring... 060417 19:32:43 [ERROR] NDB create table: distributing ./TESTER_2/t1 timed out. Ignoring... 060417 19:33:01 [ERROR] NDB drop table: distributing ./TESTER_2/t1 timed out. Ignoring... 060417 19:33:12 [ERROR] NDB drop db: distributing ./TESTER_2/ timed out. Ignoring... 060417 19:33:23 [ERROR] NDB tablespace: distributing ./ts1 timed out. Ignoring... 060417 19:33:34 [ERROR] NDB tablespace: distributing ./ts1 timed out. Ignoring... 060417 19:33:45 [ERROR] NDB logfile group: distributing ./lg1 timed out. Ignoring... 060417 21:34:30 [ERROR] NDB create db: distributing ./TPCB/ timed out. Ignoring... 060417 21:36:40 [ERROR] NDB logfile group: distributing ./lg1 timed out. Ignoring... 060417 21:40:46 [ERROR] NDB tablespace: distributing ./ts1 timed out. Ignoring... 060417 21:44:54 [ERROR] NDB tablespace: distributing ./ts1 timed out. Ignoring... 060417 21:45:06 [ERROR] NDB create table: distributing ./TPCB/account timed out. Ignoring... 060417 21:45:18 [ERROR] NDB create table: distributing ./TPCB/branch timed out. Ignoring... 060417 21:45:30 [ERROR] NDB create table: distributing ./TPCB/teller timed out. Ignoring... 060417 21:45:43 [ERROR] NDB create table: distributing ./TPCB/history timed out. Ignoring... 060417 21:45:55 [ERROR] NDB create table: distributing ./TPCB/trans timed out. Ignoring...
[17 Apr 2006 23:51]
Jonathan Miller
mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=8388600 read_buffer_size=131072 max_used_connections=3 max_connections=100 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=0x89d5650 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xb7faef44, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x81d0974 0xe29420 (nil) 0x9fdb80 0x9559ce New value of fp=(nil) failed sanity check, terminating stack trace! Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at (nil) is invalid pointer thd->thread_id=1 The manual page at http://www.mysql.com/doc/en/Crashing.html contains information that should help you find out what is causing the crash. Number of processes running now: 0 060418 01:07:46 mysqld restarted /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'general_log' is marked as crashed and should be repaired /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'slow_log' is marked as crashed and should be repaired 060418 1:07:51 [Note] Starting MySQL Cluster Binlog Thread 060418 1:07:51 [Note] /home/ndbdev/jmiller/builds/libexec/mysqld: ready for connections. Version: '5.1.10-beta-log' socket: '/tmp/mysql.sock' port: 3306 Source distribution mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=8388600 read_buffer_size=131072 max_used_connections=0 max_connections=100 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=0x97f59d0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xb1f6bf44, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x81d0974 0xaff420 (nil) 0x9fdb80 0x9559ce New value of fp=(nil) failed sanity check, terminating stack trace! Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at (nil) is invalid pointer thd->thread_id=1 The manual page at http://www.mysql.com/doc/en/Crashing.html contains information that should help you find out what is causing the crash.
[17 Apr 2006 23:51]
Jonathan Miller
Number of processes running now: 0 060418 01:08:40 mysqld restarted /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'general_log' is marked as crashed and should be repaired /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'slow_log' is marked as crashed and should be repaired 060418 1:08:45 [Note] Starting MySQL Cluster Binlog Thread 060418 1:08:45 [Note] /home/ndbdev/jmiller/builds/libexec/mysqld: ready for connections. Version: '5.1.10-beta-log' socket: '/tmp/mysql.sock' port: 3306 Source distribution mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=8388600 read_buffer_size=131072 max_used_connections=0 max_connections=100 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=0xa22d9d0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xb1f33f44, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x81d0974 0x8b3420 (nil) 0x9fdb80 0x1db9ce New value of fp=(nil) failed sanity check, terminating stack trace! Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at (nil) is invalid pointer thd->thread_id=1 The manual page at http://www.mysql.com/doc/en/Crashing.html contains information that should help you find out what is causing the crash. Number of processes running now: 0 060418 01:10:00 mysqld restarted /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'general_log' is marked as crashed and should be repaired /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'slow_log' is marked as crashed and should be repaired 060418 1:10:16 [Note] Starting MySQL Cluster Binlog Thread 060418 1:10:16 [Note] /home/ndbdev/jmiller/builds/libexec/mysqld: ready for connections. Version: '5.1.10-beta-log' socket: '/tmp/mysql.sock' port: 3306 Source distribution mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=8388600 read_buffer_size=131072 max_used_connections=1 max_connections=100 threads_connected=1 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=0x8f0b9d0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xb1f55f44, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x81d0974 0xb85420 (nil) 0x9fdb80 0x9559ce New value of fp=(nil) failed sanity check, terminating stack trace! Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at (nil) is invalid pointer thd->thread_id=1 The manual page at http://www.mysql.com/doc/en/Crashing.html contains information that should help you find out what is causing the crash.
[17 Apr 2006 23:52]
Jonathan Miller
Number of processes running now: 0 060418 01:11:55 mysqld restarted /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'general_log' is marked as crashed and should be repaired /home/ndbdev/jmiller/builds/libexec/mysqld: Table 'slow_log' is marked as crashed and should be repaired 060418 1:12:06 [Note] Starting MySQL Cluster Binlog Thread 060418 1:12:06 [Note] /home/ndbdev/jmiller/builds/libexec/mysqld: ready for connections. Version: '5.1.10-beta-log' socket: '/tmp/mysql.sock' port: 3306 Source distribution mysqld got signal 6; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=8388600 read_buffer_size=131072 max_used_connections=0 max_connections=100 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=(nil) Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xb1f4f36c, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x81d0974 0x6b9420 (nil) 0x8e822a 0x8ee124 0x8ee65f 0x8420fd3 0x83625f4 0x83684a7 0x8366d25 0x83455bd Stack trace seems successful - bottom reached Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it The manual page at http://www.mysql.com/doc/en/Crashing.html contains information that should help you find out what is causing the crash.
[17 Apr 2006 23:52]
Jonathan Miller
resolve_stack_dump -s /tmp/mysqld.sym -n stack.trace 0x81d0974 handle_segfault + 438 0x6b9420 (?) (nil) 0x8e822a (?) 0x8ee124 (?) 0x8ee65f (?) 0x8420fd3 _ZdaPv + 23 0x83625f4 _ZN17EventBufData_list10add_gci_opENS_6Gci_opEb + 312 0x83684a7 _ZN17EventBufData_list11append_dataEP12EventBufData + 81 0x8366d25 _ZN14NdbEventBuffer11insertDataLEP21NdbEventOperationImplPK12SubTableDataP16LinearSectionPtr + 421 0x83455bd _ZN3Ndb20handleReceivedSignalEP12NdbApiSignalP16LinearSectionPtr + 4047
[21 Apr 2006 13:12]
Tomas Ulin
several memory corrupting bugs have been fixed that are the likely cause of this, retesting is needed when those are merged into the main tree
[10 May 2006 7:19]
Tomas Ulin
for documentation, fixed in 5.1.10
[10 May 2006 16:07]
Paul DuBois
Noted in 5.1.10 changelog. Eliminated some memory corruption problems that resultsd in <literal>double free or corruption</literal> errors and a server crash.
