Description:
I am experiencing occasional crashes, approximately once a week mysqld will crash and restart itself and after several of these the mysqld process will hang leaving no option than to kill -9 the process. There doesn't appear to be any consistent reason for the crash, perhaps associated with heavy server loads at the time but I've never been present during a crash to know.
Please let me know what other information I can provide to be helpful.
The following is output from the syslog::
mysqld[4241]: 081216 3:54:29 - mysqld got signal 11;
mysqld[4241]: This could be because you hit a bug. It is also possible that this binary
mysqld[4241]: or one of the libraries it was linked against is corrupt, improperly built,
mysqld[4241]: or misconfigured. This error can also be caused by malfunctioning hardware.
mysqld[4241]: We will try our best to scrape up some info that will hopefully help diagnose
mysqld[4241]: the problem, but since we have already crashed, something is definitely wrong
mysqld[4241]: and this may fail.
mysqld[4241]:
mysqld[4241]: key_buffer_size=16777216
mysqld[4241]: read_buffer_size=131072
mysqld[4241]: max_used_connections=101
mysqld[4241]: max_connections=100
mysqld[4241]: threads_connected=8
mysqld[4241]: It is possible that mysqld could use up to
mysqld[4241]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 233983 K
mysqld[4241]: bytes of memory
mysqld[4241]: Hope that's ok; if not, decrease some variables in the equation.
mysqld[4241]:
mysqld[4241]: thd=0xb0bf1558
mysqld[4241]: Attempting backtrace. You can use the following information to find out
mysqld[4241]: where mysqld died. If you see no messages after this, something went
mysqld[4241]: terribly wrong...
mysqld[4241]: Cannot determine thread, fp=0xaf2dcbf8, backtrace may not be correct.
mysqld[4241]: Stack range sanity check OK, backtrace follows:
mysqld[4241]: 0x81f3811
mysqld[4241]: 0x82efef3
mysqld[4241]: 0x82f2bbe
mysqld[4241]: 0x82f319b
mysqld[4241]: 0x82f3951
mysqld[4241]: 0x82074ae
mysqld[4241]: 0x820ddfe
mysqld[4241]: 0x8210783
mysqld[4241]: 0x821120a
mysqld[4241]: 0x8211b84
mysqld[4241]: 0xb7f494c0
mysqld[4241]: 0xb7d5b61e
mysqld[4241]: New value of fp=(nil) failed sanity check, terminating stack trace!
mysqld[4241]: Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved
mysqld[4241]: stack trace is much more helpful in diagnosing the problem, so please do
mysqld[4241]: resolve it
mysqld[4241]: Trying to get some variables.
mysqld[4241]: Some pointers may be invalid and cause the dump to abort...
mysqld[4241]: thd->query at 0xafbdcf70 is invalid pointer
mysqld[4241]: thd->thread_id=120804
mysqld[4241]: The manual page at http://www.mysql.com/doc/en/Crashing.html contains
mysqld[4241]: information that should help you find out what is causing the crash.
mysqld_safe[12144]: Number of processes running now: 0
mysqld_safe[12148]: restarted
Dmysqld[12151]: InnoDB: Log scan progressed past the checkpoint lsn 14 3920410572
Dmysqld[12151]: 081216 3:54:33 InnoDB: Database was not shut down normally!
Dmysqld[12151]: InnoDB: Starting crash recovery.
Dmysqld[12151]: InnoDB: Reading tablespace information from the .ibd files...
mysqld[12151]: InnoDB: Restoring possible half-written data pages from the doublewrite
mysqld[12151]: InnoDB: buffer...
mysqld[12151]: InnoDB: Doing recovery: scanned up to log sequence number 14 3920419010
mysqld[12151]: 081216 3:54:41 InnoDB: Starting an apply batch of log records to the database...
mysqld[12151]: InnoDB: Progress in percents: 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
mysqld[12151]: InnoDB: Apply batch completed
mysqld[12151]: InnoDB: Last MySQL binlog file position 0 13002192, file name /var/log/mysql/mysql-bin.000963
mysqld[12151]: 081216 3:54:42 InnoDB: Started; log sequence number 14 3920419010
mysqld[12151]: 081216 3:54:42 [Note] Recovering after a crash using /var/log/mysql/mysql-bin
mysqld[12151]: 081216 3:54:43 [Note] Starting crash recovery...
mysqld[12151]: 081216 3:54:43 [Note] Crash recovery finished.
mysqld[12151]: 081216 3:54:44 [Note] /usr/sbin/mysqld: ready for connections.
mysqld[12151]: Version: '5.0.51a-18-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)
Last entry in /var/log/mysql/mysql-bin.000963 ::
)~RFI^B^A^@^@^@�^A^@^@t|^@^@^@^A^@^@^@^@^@^K^@^@^Z^@^@^@@^@^@^A^@^@^@^@^@^@^@^@^F^Cstd^D^H^@^H^@^H^@kangaroo_is^@INSERT INTO eZSession_SessionSess
( ID, Created, LastAccessed, Hash1 )
VALUES ( '1756225',
'1229361869',
'1229361869',
'4887283f05443c0dea5f7c9df3ef6980'
)
How to repeat:
Unfortunately I do not know what is causing this crash and therefore can't repeat it at will.
