Bug #56373 | InnoDB Plugin : DROP TABLE on a corrupt table crashes server | ||
---|---|---|---|
Submitted: | 30 Aug 2010 14:49 | Modified: | 10 Sep 2010 9:55 |
Reporter: | Leandro Morgado | Email Updates: | |
Status: | Duplicate | Impact on me: | |
Category: | MySQL Server: InnoDB Plugin storage engine | Severity: | S3 (Non-critical) |
Version: | 5.1.48 | OS: | Any |
Assigned to: | Assigned Account | CPU Architecture: | Any |
[30 Aug 2010 14:49]
Leandro Morgado
[8 Sep 2010 15:50]
Mark Callaghan
I have recently hit this issue. I think the sequence is: 1) take hot backup from master using xtrabackup 2) restore on slave -- slave crashes I think the call stack on the slave is: dict_load_table -> fil_open_single_table_tablespace Note that dict_load_table may set DICT_TF_COMPACT in flags prior to passing that to fil_open_single_table_tablespace Also note that the call to fil_open_single_table_tablespace from dict_load_table is probably infrequently done as it only occurs when the ibd file is not found in the InnoDB dictionary I don't have a core file. Core files are harder to get with large innodb buffer pool size and mysqld did not write a call stack in this case. Contents of the error log: InnoDB: Doing recovery: scanned up to log sequence number 1407792966957 InnoDB: 22 transaction(s) which must be rolled back or cleaned up InnoDB: in total 47 row operations to undo InnoDB: Trx id counter is 10CA2DC00 100907 22:32:53 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed 100907 22:33:46 InnoDB: reading slave state from the transaction system header InnoDB: relay-log - filename , position (4294967295) InnoDB: master-log - filename , position (4294967295) InnoDB: recv_recovery_from_checkpoint_finish() - recovery is needed. InnoDB: Last MySQL binlog file position 0 175128008, file name /binlogs/binary-logs.001160 100907 22:33:46 InnoDB: Rolling back trx with id 10CA2DAB8, 21 rows to undo InnoDB: Dropping table with id 0 4486 in recovery if it exists 100907 22:33:46 InnoDB: error: space object of table'hs23670/__osc_old_assoc_info', InnoDB: space id 4474 did not exist in memory. Retrying an open. 100907 22:33:46 InnoDB: Assertion failure in thread 46924329626544 in file fil/fil0fil.c line 2832 InnoDB: Failing assertion: flags != DICT_TF_COMPACT InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html InnoDB: about forcing recovery. 100907 22:33:46 - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=16777216 read_buffer_size=1048576 max_used_connections=0 max_threads=5000 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 15429352 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = (nil) thread_stack 0x30000 /usr/libexec/mysqld(my_print_stacktrace+0x2e) [0x8b352e] /usr/libexec/mysqld(handle_segfault+0x33d) [0x5e1a3d] /lib64/libpthread.so.0 [0x320b00de70] /lib64/libc.so.6(gsignal+0x35) [0x36fe430155] /lib64/libc.so.6(abort+0x110) [0x36fe431bf0] /usr/libexec/mysqld [0x846def] /usr/libexec/mysqld [0x8380ad] /usr/libexec/mysqld [0x838dbb] /usr/libexec/mysqld [0x7f0955] /usr/libexec/mysqld [0x7e1d85] /usr/libexec/mysqld [0x772bd8] /usr/libexec/mysqld(ha_initialize_handlerton(st_plugin_int*)+0x36) [0x6d2216] /usr/libexec/mysqld [0x75653d] /usr/libexec/mysqld(plugin_init(int*, char**, int)+0x77d) [0x758fed] /usr/libexec/mysqld [0x5e3ed5] /usr/libexec/mysqld(main+0x825) [0x5e5615] I am trying to figure out what was in progress on the master during the hot backup. It was likely one of: * long running create table * long running drop [temp] table * long running create index
[8 Sep 2010 15:55]
Mark Callaghan
Note that the fix for http://bugs.mysql.com/bug.php?id=54009 probably fixes this as it masks DICT_TF_COMPACT
[9 Sep 2010 11:12]
Marko Mäkelä
This is a special case of Bug #10132 (I would say a duplicate of it).
[10 Sep 2010 9:55]
Calvin Sun
dup of bug#10132.
[27 Sep 2010 9:57]
Marko Mäkelä
I meant my comment "special case of Bug #10132" on the conceptual level, meaning that we should avoid crashing on corrupted data. Upon closer investigation, this appears to be a duplicate of Bug #54009. All callers now pass the tablespace flags to fil_open_single_table_tablespace() as 0 for REDUNDANT and COMPACT tables, and as SYS_TABLES.TYPE for DYNAMIC and COMPRESSED tables.