| Bug #59385 | mysqld crashes in various ways when starting to run into OS open files limit | ||
|---|---|---|---|
| Submitted: | 10 Jan 2011 5:09 | Modified: | 7 Dec 2011 16:05 |
| Reporter: | Roel Van de Paar | Email Updates: | |
| Status: | Duplicate | Impact on me: | |
| Category: | MySQL Server: General | Severity: | S1 (Critical) |
| Version: | 5.5-sec-bk | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
[10 Jan 2011 5:09]
Roel Van de Paar
[10 Jan 2011 5:10]
Roel Van de Paar
Enabled core dumping (did not get a core for the crash above) and restarted mysqld. Got this: ======================= 110110 11:57:05 InnoDB: highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 28871809 110110 11:57:05 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 29232899 InnoDB: 1 transaction(s) which must be rolled back or cleaned up InnoDB: in total 1 row operations to undo InnoDB: Trx id counter is 4200 110110 11:57:06 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 7 6 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Starting in background the rollback of uncommitted transactions 110110 11:57:08 InnoDB: Rolling back trx with id 4017, 1 rows to undo 110110 11:57:08 InnoDB: Assertion failure in thread 140100795864832 in file /mysql/mysql-5.5-security/storage /innobase/dict/dict0dict.c line 619 InnoDB: Failing assertion: mutex_own(&dict_sys->mutex) InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html InnoDB: about forcing recovery. 110110 11:57:08 - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=8388608 read_buffer_size=131072 max_used_connections=0 max_threads=151 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338633 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = (nil) thread_stack 0x40000 110110 11:57:08 InnoDB: 1.1.4 started; log sequence number 29232899 /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld(my_print_stacktrace+0x35)[0x8be7ca] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld(handle_segfault+0x31f)[0x554c76] /lib64/libpthread.so.0[0x32c240f3c0] /lib64/libc.so.6(gsignal+0x35)[0x32c1c34085] /lib64/libc.so.6(abort+0x186)[0x32c1c35a36] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0x9cf794] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0xa56387] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0xa56550] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0xa4c583] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0xa4c692] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0xa43839] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0xa43a6a] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0xa43b8a] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0x98142d] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0x9817e3] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld[0x9818cb] /lib64/libpthread.so.0[0x32c2406d5b] /lib64/libc.so.6(clone+0x6d)[0x32c1ce4a7d] The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file ======================= gdb bt on that core: ======================= #0 0x00000032c240c94c in pthread_kill () from /lib64/libpthread.so.0 #1 0x00000000008be856 in my_write_core (sig=6) at /mysql/mysql-5.5-security/mysys/stacktrace.c:423 #2 0x0000000000554f26 in handle_segfault (sig=6) at /mysql/mysql-5.5-security/sql/mysqld.cc:2503 #3 <signal handler called> #4 0x00000032c1c34085 in raise () from /lib64/libc.so.6 #5 0x00000032c1c35a36 in abort () from /lib64/libc.so.6 #6 0x00000000009cf794 in dict_table_get_on_id (table_id=1, trx=0x20dfad8) at /mysql/mysql-5.5-security/storage/innobase/dict/dict0dict.c:619 #7 0x0000000000a56387 in row_undo_mod_parse_undo_rec (node=0x7f6bac0012b8, thr=0x7f6bac001100) at /mysql/mysql-5.5-security/storage/innobase/row/row0umod.c:813 #8 0x0000000000a56550 in row_undo_mod (node=0x7f6bac0012b8, thr=0x7f6bac001100) at /mysql/mysql-5.5-security/storage/innobase/row/row0umod.c:861 #9 0x0000000000a4c583 in row_undo (node=0x7f6bac0012b8, thr=0x7f6bac001100) at /mysql/mysql-5.5-security/storage/innobase/row/row0undo.c:325 #10 0x0000000000a4c692 in row_undo_step (thr=0x7f6bac001100) at /mysql/mysql-5.5-security/storage/innobase/row/row0undo.c:367 #11 0x0000000000a43839 in que_thr_step (thr=0x7f6bac001100) at /mysql/mysql-5.5-security/storage/innobase/que/que0que.c:1245 #12 0x0000000000a43a6a in que_run_threads_low (thr=0x7f6bac001100) at /mysql/mysql-5.5-security/storage/innobase/que/que0que.c:1305 #13 0x0000000000a43b8a in que_run_threads (thr=0x7f6bac000a00) at /mysql/mysql-5.5-security/storage/innobase/que/que0que.c:1342 #14 0x000000000098142d in trx_rollback_active (trx=0x20dfad8) at /mysql/mysql-5.5-security/storage/innobase/trx/trx0roll.c:472 #15 0x00000000009817e3 in trx_rollback_or_clean_recovered (all=1) at /mysql/mysql-5.5-security/storage/innobase/trx/trx0roll.c:585 #16 0x00000000009818cb in trx_rollback_or_clean_all_recovered (arg=0x7fff4f57181c) at /mysql/mysql-5.5-security/storage/innobase/trx/trx0roll.c:621 #17 0x00000032c2406d5b in start_thread () from /lib64/libpthread.so.0 #18 0x00000032c1ce4a7d in clone () from /lib64/libc.so.6 =======================
[10 Jan 2011 5:10]
Roel Van de Paar
Testing some more, ran into this: ======================= 110110 15:20:18 [Note] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: ready for connections. Version: '5.5.9-valgrind-max-debug' socket: '/mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/socket-5500.s ock' port: 5500 Source distribution /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: read looped with error 4, aborting thread /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: read looped with error 4, aborting thread /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: read looped with error 4, aborting thread /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: read looped with error 4, aborting thread /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: read looped with error 4, aborting thread /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: read looped with error 4, aborting thread /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: read looped with error 4, aborting thread 110110 15:41:25 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/@00111e@0p@0091qV@00bc@0q@00a0@0I@1i@0095@0040@007d@1g@003f.frm' (errno: 24) 110110 15:41:25 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t62.frm' (errno: 24) 110110 15:41:25 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t81.frm' (errno: 24) 110110 15:41:25 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/@0092@008a@00b4X2@0080f@00b1J@1j@00b8@0094Z.frm' (errno: 24) 110110 15:41:25 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t11.frm' (errno: 24) 110110 15:41:25 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t80.frm' (errno: 24) 110110 15:41:25 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t94.frm' (errno: 24) 110110 15:41:29 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t31.frm' (errno: 24) 110110 15:41:29 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t90.frm' (errno: 24) 110110 15:41:29 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/@0h.frm' (errno: 24) 110110 15:41:29 [ERROR] /mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/bin/mysqld: Can't open file: './tes t/t30.frm' (errno: 24) mysqld: /mysql/mysql-5.5-security/mysys/my_seek.c:57: my_seek: Assertion `fd != -1' failed. ======================= Ignore the "read looped with error 4, aborting thread" - see bug #34150. This was running inside gdb. gdb bt: ======================= #0 0x00000032c1c34085 in raise () from /lib64/libc.so.6 #1 0x00000032c1c35a36 in abort () from /lib64/libc.so.6 #2 0x00000032c1c2c8c5 in __assert_fail () from /lib64/libc.so.6 #3 0x00000000008bac20 in my_seek (fd=-1, pos=0, whence=0, MyFlags=0) at /mysql/mysql-5.5-security/mysys/my_seek.c:57 #4 0x0000000000a65935 in inline_mysql_file_seek (src_file=0xc31c70 "/mysql/mysql-5.5-security/storage/csv/ha_tina.cc", src_line=355, file=-1, pos=0, whence=0, flags=0) at /mysql/mysql-5.5-security/include/mysql/psi/mysql_file.h:1234 #5 0x0000000000a66451 in write_meta_file (meta_file=-1, rows=0, dirty=true) at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:355 #6 0x0000000000a66798 in free_share (share=0x7ffff000d340) at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:419 #7 0x0000000000a67a59 in ha_tina::open (this=0x7ffff036f8e0, name=0x7ffff0a249d8 "./test/EP@00a1p@0nYF@0uU@0019D@0096@1h@0095p@00ad@1i@009c@0094@0sG@1L@0R@0089@00a5@00a9@0082@0h@0X@00b9@0013@0k@0O@0x@0k@0o@0v@0092t@000e @0022@0016@008e@1kS@001b@0V@00d7@0J4@0Q", mode=2, open_options=2) at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:937 #8 0x0000000000752a66 in handler::ha_open (this=0x7ffff036f8e0, table_arg=0x7ffff0029280, name=0x7ffff0a249d8 "./test/EP@00a1p@0nYF@0uU@0019D@0096@1h@0095p@00ad@1i@009c@0094@0sG@1L@0R@0089@00a5@00a9@0082@0h@0X@00b9@0013@0k@0O@0x@0k@0o@0v@0092t@000e@0022@0016@008e@1kS@001b@0V@00d7@0J4@0Q", mode=2, test_if_locked=2) at /mysql/mysql-5.5-security/sql/handler.cc:2141 #9 0x000000000069a6e1 in open_table_from_share (thd=0x208fac0, share=0x7ffff0a24610, alias=0x7ffff0105810 "EP¡pçYFîU\031D\302\226õ\302\225pö\302\234\302\224ìGÙË\302\211¥©\302\202áѹ\023äÈñäèï\302\222t\016\"\026\302\216 øS\033Ï×Ã4Ê", db_stat=39, prgflag=44, ha_open_flags=0, outparam=0x7ffff0029280, is_create_table=false) at /mysql/mysql-5.5-security/sql/table.cc:2003 #10 0x00000000005a1cd6 in open_table (thd=0x208fac0, table_list=0x7ffff08d2bf0, mem_root=0x7fffe29abf60, ot_ctx=0x7fffe29abfa0) at /mysql/mysql-5.5-security/sql/sql_base.cc:3033 #11 0x00000000005a3f5d in open_and_process_table (thd=0x208fac0, lex=0x20916e0, tables=0x7ffff08d2bf0, counter=0x7fffe29ac0c4, flags=0, prelocking_strategy=0x7fffe29ad720, has_prelocking_list=false, ot_ctx=0x7fffe29abfa0 , new_frm_mem=0x7fffe29abf60) at /mysql/mysql-5.5-security/sql/sql_base.cc:4362 #12 0x00000000005a4cb0 in open_tables (thd=0x208fac0, start=0x7fffe29ac080, counter=0x7fffe29ac0c4, flags=0, prelocking_strategy=0x7fffe29ad720) at /mysql/mysql-5.5-security/sql/sql_base.cc:4802 #13 0x00000000005a5b44 in open_and_lock_tables (thd=0x208fac0, tables=0x7ffff08d2bf0, derived=false, flags=0, prelocking_strategy=0x7fffe29ad720) at /mysql/mysql-5.5-security/sql/sql_base.cc:5399 #14 0x0000000000677c51 in mysql_alter_table (thd=0x208fac0, new_db=0x7ffff0022be0 "test", new_name=0x0, create_info=0x7fffe29ade50, table_list=0x7ffff08d2bf0, alter_info=0x7fffe29adf30, order_num=0, order=0x0, ignore=tru e) at /mysql/mysql-5.5-security/sql/sql_table.cc:5756 #15 0x000000000089552f in Alter_table_statement::execute (this=0x7ffff0351550, thd=0x208fac0) at /mysql/mysql-5.5-security/sql/sql_alter.cc:106 #16 0x00000000005f7462 in mysql_execute_command (thd=0x208fac0) at /mysql/mysql-5.5-security/sql/sql_parse.cc:4321 #17 0x00000000005f9eac in mysql_parse (thd=0x208fac0, rawbuf=0x7ffff0036580 "alter ignore table `test`.`EP¡pçYFîU\031D\302\226õ\302\225pö\302\234\302\224ìGÙË\302\211¥©\302\202áѹ\023äÈñäèï\302\222t\016\"\026\302\216øS\0 33Ï×Ã4Ê` add column `col78` tinyint zerofill not null", length=157, parser_state=0x7fffe29af630) at /mysql/mysql-5.5-security/sql/sql_parse.cc:5509 #18 0x00000000005ee3ef in dispatch_command (command=COM_QUERY, thd=0x208fac0, packet=0x20b2291 "alter ignore table `test`.`EP¡pçYFîU\031D\302\226õ\302\225pö\302\234\302\224ìGÙË\302\211¥©\302\202áѹ\023äÈñäèï\302\222t\01 6\"\026\302\216øS\033Ï×Ã4Ê` add column `col78` tinyint zerofill not null ", packet_length=158) at /mysql/mysql-5.5-security/sql/sql_parse.cc:1035 #19 0x00000000005ed725 in do_command (thd=0x208fac0) at /mysql/mysql-5.5-security/sql/sql_parse.cc:772#20 0x00000000006c9ad7 in do_handle_one_connection (thd_arg=0x208fac0) at /mysql/mysql-5.5-security/sql/sql_connect.cc:748 #21 0x00000000006c9668 in handle_one_connection (arg=0x208fac0) at /mysql/mysql-5.5-security/sql/sql_connect.cc:684 #22 0x00000032c2406d5b in start_thread () from /lib64/libpthread.so.0 #23 0x00000032c1ce4a7d in clone () from /lib64/libc.so.6 ======================= A similar stacktrace is listed here: bug #54486 - but notice in the corresponding error log the same 24 (too many open files) error...
[10 Jan 2011 7:11]
MySQL Verification Team
The CSV engine crash looks like bug #57895
[10 Jan 2011 7:21]
Roel Van de Paar
[root@mac mysql-5.5.9-linux-x86_64]# ulimit -n 1024 [root@mac mysql-5.5.9-linux-x86_64]# cat my.cnf [mysqld] core-file basedir=/mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/ socket=/mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/socket-5500.sock pid=/mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/pid-5500.pid port=5500 user=root log-bin-trust-function-creators character-set-server=latin1 log-error=/mysql/mysql-5.5-security/mysql-5.5.9-linux-x86_64/data/error.log
[10 Jan 2011 7:45]
Roel Van de Paar
More issues/crash after restarting mysqld and continuing to run some threads - lots of different errors.
Attachment: error.log (application/octet-stream, text), 191.04 KiB.
[10 Jan 2011 7:55]
Roel Van de Paar
Corresponding (with latter part in error log) gdb output
(gdb) bt
#0 0x00000032c1c34085 in raise () from /lib64/libc.so.6
#1 0x00000032c1c35a36 in abort () from /lib64/libc.so.6
#2 0x00000032c1c2c8c5 in __assert_fail () from /lib64/libc.so.6
#3 0x00000000008bac20 in my_seek (fd=-1, pos=0, whence=0, MyFlags=0)
at /mysql/mysql-5.5-security/mysys/my_seek.c:57
#4 0x0000000000a65935 in inline_mysql_file_seek (
src_file=0xc31c70 "/mysql/mysql-5.5-security/storage/csv/ha_tina.cc", src_line=355, file=-1, pos=0,
whence=0, flags=0) at /mysql/mysql-5.5-security/include/mysql/psi/mysql_file.h:1234
#5 0x0000000000a66451 in write_meta_file (meta_file=-1, rows=0, dirty=true)
at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:355
#6 0x0000000000a66798 in free_share (share=0x7ffff012ac50)
at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:419
#7 0x0000000000a67a59 in ha_tina::open (this=0x7ffff017d3a0, name=0x7ffff01671f0 "./test/#sql-2faa_216",
mode=2, open_options=2) at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:937
#8 0x0000000000752a66 in handler::ha_open (this=0x7ffff017d3a0, table_arg=0x7ffff01665d0,
name=0x7ffff01671f0 "./test/#sql-2faa_216", mode=2, test_if_locked=2)
at /mysql/mysql-5.5-security/sql/handler.cc:2141
#9 0x000000000069a6e1 in open_table_from_share (thd=0x20387a0, share=0x7ffff0166e88,
alias=0x7fffd5074680 "#sql-2faa_216", db_stat=7, prgflag=44, ha_open_flags=0, outparam=0x7ffff01665d0,
is_create_table=false) at /mysql/mysql-5.5-security/sql/table.cc:2003
#10 0x00000000005a69c9 in open_table_uncached (thd=0x20387a0, path=0x7fffd5073690 "./test/#sql-2faa_216",
db=0x7ffff0043c40 "test", table_name=0x7fffd5074680 "#sql-2faa_216", add_to_temporary_tables_list=true)
at /mysql/mysql-5.5-security/sql/sql_base.cc:5816
#11 0x00000000006794b9 in mysql_alter_table (thd=0x20387a0, new_db=0x7ffff0043c40 "test",
new_name=0x7ffff0069570 "t34", create_info=0x7fffd5074e50, table_list=0x7ffff0127f80,
alter_info=0x7fffd5074f30, order_num=0, order=0x0, ignore=false)
at /mysql/mysql-5.5-security/sql/sql_table.cc:6370
#12 0x000000000089552f in Alter_table_statement::execute (this=0x7ffff002e570, thd=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_alter.cc:106
#13 0x00000000005f7462 in mysql_execute_command (thd=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_parse.cc:4321
#14 0x00000000005f9eac in mysql_parse (thd=0x20387a0,
rawbuf=0x7ffff00f7040 "alter table `test`.`t34` add column `col61` blob not null", length=57,
parser_state=0x7fffd5076630) at /mysql/mysql-5.5-security/sql/sql_parse.cc:5509
#15 0x00000000005ee3ef in dispatch_command (command=COM_QUERY, thd=0x20387a0,
packet=0x1fa2e21 "alter table `test`.`t34` add column `col61` blob not null ", packet_length=58)
at /mysql/mysql-5.5-security/sql/sql_parse.cc:1035
#16 0x00000000005ed725 in do_command (thd=0x20387a0) at /mysql/mysql-5.5-security/sql/sql_parse.cc:772
#17 0x00000000006c9ad7 in do_handle_one_connection (thd_arg=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_connect.cc:748
#18 0x00000000006c9668 in handle_one_connection (arg=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_connect.cc:684
#19 0x00000032c2406d5b in start_thread () from /lib64/libpthread.so.0
#20 0x00000032c1ce4a7d in clone () from /lib64/libc.so.6
(gdb) continue
Continuing.
[New Thread 0x7fffe44ec700 (LWP 19418)]
[New Thread 0x7fffd65b3700 (LWP 19419)]
Program received signal SIGABRT, Aborted.
0x00000032c240c94c in pthread_kill () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00000032c240c94c in pthread_kill () from /lib64/libpthread.so.0
#1 0x00000000008be856 in my_write_core (sig=6) at /mysql/mysql-5.5-security/mysys/stacktrace.c:423
#2 0x0000000000554f26 in handle_segfault (sig=6) at /mysql/mysql-5.5-security/sql/mysqld.cc:2503
#3 <signal handler called>
#4 0x00000032c1c34085 in raise () from /lib64/libc.so.6
#5 0x00000032c1c35a36 in abort () from /lib64/libc.so.6
#6 0x00000032c1c2c8c5 in __assert_fail () from /lib64/libc.so.6
#7 0x00000000008bac20 in my_seek (fd=-1, pos=0, whence=0, MyFlags=0)
at /mysql/mysql-5.5-security/mysys/my_seek.c:57
#8 0x0000000000a65935 in inline_mysql_file_seek (
src_file=0xc31c70 "/mysql/mysql-5.5-security/storage/csv/ha_tina.cc", src_line=355, file=-1, pos=0,
whence=0, flags=0) at /mysql/mysql-5.5-security/include/mysql/psi/mysql_file.h:1234
#9 0x0000000000a66451 in write_meta_file (meta_file=-1, rows=0, dirty=true)
at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:355
#10 0x0000000000a66798 in free_share (share=0x7ffff012ac50)
at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:419
#11 0x0000000000a67a59 in ha_tina::open (this=0x7ffff017d3a0, name=0x7ffff01671f0 "./test/#sql-2faa_216",
mode=2, open_options=2) at /mysql/mysql-5.5-security/storage/csv/ha_tina.cc:937
#12 0x0000000000752a66 in handler::ha_open (this=0x7ffff017d3a0, table_arg=0x7ffff01665d0,
name=0x7ffff01671f0 "./test/#sql-2faa_216", mode=2, test_if_locked=2)
at /mysql/mysql-5.5-security/sql/handler.cc:2141
#13 0x000000000069a6e1 in open_table_from_share (thd=0x20387a0, share=0x7ffff0166e88,
alias=0x7fffd5074680 "#sql-2faa_216", db_stat=7, prgflag=44, ha_open_flags=0, outparam=0x7ffff01665d0,
is_create_table=false) at /mysql/mysql-5.5-security/sql/table.cc:2003
#14 0x00000000005a69c9 in open_table_uncached (thd=0x20387a0, path=0x7fffd5073690 "./test/#sql-2faa_216",
db=0x7ffff0043c40 "test", table_name=0x7fffd5074680 "#sql-2faa_216", add_to_temporary_tables_list=true)
at /mysql/mysql-5.5-security/sql/sql_base.cc:5816
#15 0x00000000006794b9 in mysql_alter_table (thd=0x20387a0, new_db=0x7ffff0043c40 "test",
new_name=0x7ffff0069570 "t34", create_info=0x7fffd5074e50, table_list=0x7ffff0127f80,
alter_info=0x7fffd5074f30, order_num=0, order=0x0, ignore=false)
at /mysql/mysql-5.5-security/sql/sql_table.cc:6370
#16 0x000000000089552f in Alter_table_statement::execute (this=0x7ffff002e570, thd=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_alter.cc:106
#17 0x00000000005f7462 in mysql_execute_command (thd=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_parse.cc:4321
#18 0x00000000005f9eac in mysql_parse (thd=0x20387a0,
rawbuf=0x7ffff00f7040 "alter table `test`.`t34` add column `col61` blob not null", length=57,
parser_state=0x7fffd5076630) at /mysql/mysql-5.5-security/sql/sql_parse.cc:5509
#19 0x00000000005ee3ef in dispatch_command (command=COM_QUERY, thd=0x20387a0,
packet=0x1fa2e21 "alter table `test`.`t34` add column `col61` blob not null ", packet_length=58)
at /mysql/mysql-5.5-security/sql/sql_parse.cc:1035
#20 0x00000000005ed725 in do_command (thd=0x20387a0) at /mysql/mysql-5.5-security/sql/sql_parse.cc:772
#21 0x00000000006c9ad7 in do_handle_one_connection (thd_arg=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_connect.cc:748
#22 0x00000000006c9668 in handle_one_connection (arg=0x20387a0)
at /mysql/mysql-5.5-security/sql/sql_connect.cc:684
#23 0x00000032c2406d5b in start_thread () from /lib64/libpthread.so.0
#24 0x00000032c1ce4a7d in clone () from /lib64/libc.so.6
[10 Jan 2011 7:57]
Roel Van de Paar
Btw, the InnoDB "has waited ... seconds the semaphore" can likely be ignored as they may have been the result of the gdb pause (after which continue was executed)
[10 Jan 2011 8:27]
Roel Van de Paar
From Marko: The “110110 10:16:57 [ERROR] Error in accept: Too many open files” is a MySQL error, not an InnoDB error. Accept is the call to accept a TCP/IP socket connection, and that one will attempt allocate a file descriptor as well. The “110110 10:16:58 InnoDB: Operating system error number 24 in a file operation.” a second later is the same error reported by InnoDB (i.e., InnoDB complains when it attempts to open the *.ibd file a second after the accept() failed in MYSQL).
[11 Jan 2011 22:23]
Omer Barnir
triage: setting tag to SR55MRU (w3 - increase the limit of open files
[10 Feb 2011 9:39]
Marko Mäkelä
As far as InnoDB is concerned, the problem is that InnoDB does not do graceful error handling when a read fails. (Write failures should probably result in a server crash, but read failures could be handled by denying access to the affected data.) The InnoDB fix for this could be a small part of Bug #10132 Crashing the server on corrupt InnoDB page is unhelpful
