Description:
I see MySQL use up 198% on top before the crash. The server has two cores running Centos 5.2 64bit.
Soon after I notice MySQL restart itself. Before MySQl restarted I get a lot of this in the logs:
InnoDB: error in sec index entry update in
InnoDB: index `user_id` of table `admin_db/phpbb_sessions_keys`
InnoDB: tuple DATA TUPLE: 2 fields;
0: len 3; hex 8849cc; asc I ;; 1: len 32; hex 3164313735633831623937316633396433303538366364646265313537336334; asc 1d175c81b971f39d30586cddbe1573c4;;
InnoDB: record PHYSICAL RECORD: n_fields 2; compact format; info bits 0
0: len 3; hex 8849cc; asc I ;; 1: len 30; hex 306464366539313836383931386238636230633733316434303361396434; asc 0dd6e91868918b8cb0c731d403a9d4;...(truncated);
TRANSACTION 0 29015037, ACTIVE 0 sec, process no 2813, OS thread id 1100745024 updating or deleting, thread declared inside InnoDB 497
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1216, undo log entries 2
MySQL thread id 1056, query id 3077640 192.168.1.30 admin_acct Updating
UPDATE phpbb_sessions_keys
SET last_ip = '794f2251', key_id = '8ced557fa6ad705f05f190fde4a1a55d', last_login = 1233306002
WHERE key_id = '1d175c81b971f39d30586cddbe1573c4'
InnoDB: Submit a detailed bug report to http://bugs.mysql.com
then MySQL restarts:
090130 18:11:36 InnoDB: Starting shutdown...
090130 18:12:21 InnoDB: Shutdown completed; log sequence number 3 367417338
090130 18:12:21 [Note] /usr/sbin/mysqld: Shutdown complete
090130 18:12:21 mysqld ended
090130 18:12:22 mysqld started
And then I get a lot of these errors:
090130 18:28:01 mysqld started
InnoDB: Log scan progressed past the checkpoint lsn 3 367448057
090130 18:28:03 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 3 367699960
090130 18:28:04 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 12 13 14 15 16 17 18 19 090130 18:28:04 InnoDB: Error: page 639232 log sequence number 3 370239540
InnoDB: is in the future! Current system log sequence number 3 367699960.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
090130 18:28:04 InnoDB: Error: page 639257 log sequence number 3 367790585
InnoDB: is in the future! Current system log sequence number 3 367699960.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: for more information.
Please help.
How to repeat:
I don't know how to repeat.
Suggested fix:
I wish I knew.