Description:
Mysql crasheed due to signl 11 and shows repeated Assertion failure errors.
We tried force recovery 1 and 2 but didn't helped in making database start.
110407 20:55:45 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
110407 20:55:45 InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 114 1103455872.
InnoDB: Doing recovery: scanned up to log sequence number 114 1103665185
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 520 row operations to undo
InnoDB: Trx id counter is 0 1004754944
110407 20:55:45 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 12840, file name ./posmysql-bin.001184
InnoDB: Starting in background the rollback of uncommitted transactions
110407 20:55:46 InnoDB: Rolling back trx with id 0 1004754500, 520 rows to undo
110407 20:55:46 InnoDB: Started; log sequence number 114 1103665185
110407 20:55:46 [Note] Recovering after a crash using posmysql-bin
110407 20:55:46 [Note] Starting crash recovery...
110407 20:55:46 [Note] Crash recovery finished.
110407 20:55:46InnoDB: Assertion failure in thread 1399532464 in file fsp0fsp.c line 1569
InnoDB: Failing assertion: frag_n_used > 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
key_buffer_size=0
read_buffer_size=131072
max_used_connections=0
max_connections=300
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 652797 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
thd=(nil)
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x536b150c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818bbf0
0x83464fb
0x8346617
0x8325610
0x83261fd
0x831805d
0x831855f
0x82e3622
0x82c31e5
0x83194a5
0x40033cf7
0x401632ee
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
110407 20:58:56 InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
110407 20:58:56 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
110407 20:58:56 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
110407 20:58:56 InnoDB: Started; log sequence number 0 0
110407 20:58:56 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.host' doesn't exist
How to repeat:
We need to know the reason why database is getting crashed and hence we can't reproduce the data corruption.
Whenever we try to start we got this error log.
