Logging: ./mysql-test-run.pl rpl.rpl_innodb_bug28430 120605 11:11:28 [Note] Plugin 'FEDERATED' is disabled. MySQL Version 5.5.25 Checking supported features... - skipping ndbcluster - SSL connections supported - binaries are debug compiled Collecting tests... - adding combinations for rpl vardir: /opt/data/install/mysql/mysql-5.5.25/mysql-test/var Checking leftover processes... - found old pid 20318 in 'mysqld.2.pid', killing it... process did not exist! - found old pid 20315 in 'mysqld.1.pid', killing it... process did not exist! Removing old var directory... Creating var directory '/opt/data/install/mysql/mysql-5.5.25/mysql-test/var'... Installing system database... Using server port 52332 ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 worker[1] Trying to dump core for [mysqltest - pid: 20444, winpid: 20444] worker[1] Trying to dump core for [mysqld.1 - pid: 20406, winpid: 20406] worker[1] Trying to dump core for [mysqld.2 - pid: 20409, winpid: 20409] rpl.rpl_innodb_bug28430 'mix' [ fail ] timeout after 900 seconds Test ended at 2012-06-05 11:27:31 Test case timeout after 900 seconds == /opt/data/install/mysql/mysql-5.5.25/mysql-test/var/log/rpl_innodb_bug28430.log == SET local_time= NOW(); SET cur_user = CURRENT_USER(); SET local_uuid=UUID(); WHILE ins_count > 0 DO INSERT INTO test.byrange_tbl VALUES (NULL, NOW(), USER(), UUID(), ins_count,'Going to test MBR for MySQL'); SET ins_count = ins_count - 1; END WHILE; SELECT MAX(id) FROM test.byrange_tbl INTO del_count; WHILE del_count > 0 DO DELETE FROM test.byrange_tbl WHERE id = del_count; SET del_count = del_count - 2; END WHILE; END| CALL test.proc_norm(); SELECT count(*) as "Master regular" FROM test.regular_tbl; Master regular 500 CALL test.proc_bykey(); SELECT count(*) as "Master bykey" FROM test.bykey_tbl; Master bykey 500 == /opt/data/install/mysql/mysql-5.5.25/mysql-test/var/tmp/analyze-timeout-mysqld.2.err == mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/opt/data/install/mysql/mysql-5.5.25/mysql-test/var/tmp/mysqld.2.sock' (111) == /opt/data/install/mysql/mysql-5.5.25/mysql-test/var/tmp/analyze-timeout-mysqld.1.err == mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/opt/data/install/mysql/mysql-5.5.25/mysql-test/var/tmp/mysqld.1.sock' (111) - saving '/opt/data/install/mysql/mysql-5.5.25/mysql-test/var/log/rpl.rpl_innodb_bug28430-mix/' to '/opt/data/install/mysql/mysql-5.5.25/mysql-test/var/log/rpl.rpl_innodb_bug28430-mix/' - found 'core.20407' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace Core generated by '/opt/data/install/mysql/mysql-5.5.25/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- Missing separate debuginfo for Try: yum --disablerepo='*' --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/3b/5709b4dddefce3dd0b4bae733298285d52c602 [New Thread 20407] [New Thread 20412] [New Thread 20413] [New Thread 20426] [New Thread 20455] [New Thread 20417] [New Thread 20427] [New Thread 20416] [New Thread 20428] [New Thread 20415] [New Thread 20434] [New Thread 20414] [New Thread 20425] [New Thread 20456] [Thread debugging using libthread_db enabled] Core was generated by `/opt/data/install/mysql/mysql-5.5.25/sql/mysqld --defaults-group-suffix=.1 --de'. Program terminated with signal 6, Aborted. #0 0x0000003a3ee0c63c in pthread_kill () from /lib64/libpthread.so.0 #0 0x0000003a3ee0c63c in pthread_kill () from /lib64/libpthread.so.0 #1 0x00000000007d69d9 in my_write_core (sig=6) at /opt/data/install/mysql/mysql-5.5.25/mysys/stacktrace.c:433 #2 0x00000000006a0b6e in handle_fatal_signal (sig=6) at /opt/data/install/mysql/mysql-5.5.25/sql/signal_handler.cc:247 #3 #4 0x0000003a3e2dc613 in poll () from /lib64/libc.so.6 #5 0x0000000000515020 in handle_connections_sockets () at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:5147 #6 0x000000000051b25a in mysqld_main (argc=50, argv=0x2124ef8) at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:4595 #7 0x000000000051007d in main (argc=, argv=) at /opt/data/install/mysql/mysql-5.5.25/sql/main.cc:25 Thread 14 (Thread 0x7f1d18c77700 (LWP 20456)): #0 0x0000003a3ee0ebad in fsync () from /lib64/libpthread.so.0 #1 0x000000000097023f in os_file_fsync (file=9) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:2058 #2 os_file_flush_func (file=9) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:2156 #3 0x0000000000922654 in pfs_os_file_flush_func (space_id=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/include/os0file.ic:381 #4 fil_flush (space_id=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4685 #5 0x000000000095c5d6 in log_write_up_to (lsn=, wait=, flush_to_disk=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/log/log0log.c:1514 #6 0x00000000008b5d02 in trx_prepare_off_kernel (trx=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/trx/trx0trx.c:2005 #7 0x00000000008b7993 in trx_prepare_for_mysql (trx=0x7f1d0400e598) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/trx/trx0trx.c:2041 #8 0x000000000085be69 in innobase_xa_prepare (hton=, thd=0x28e10f0, all=false) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/handler/ha_innodb.cc:10511 #9 0x00000000006a69c1 in ha_commit_trans (thd=0x28e10f0, all=false) at /opt/data/install/mysql/mysql-5.5.25/sql/handler.cc:1309 #10 0x000000000064cb9c in trans_commit_stmt (thd=0x28e10f0) at /opt/data/install/mysql/mysql-5.5.25/sql/transaction.cc:299 #11 0x000000000059ad64 in mysql_execute_command (thd=0x28e10f0) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_parse.cc:4484 #12 0x000000000077a202 in sp_instr_stmt::exec_core (this=0x7f1d040e5f88, thd=, nextp=0x7f1d18c75708) at /opt/data/install/mysql/mysql-5.5.25/sql/sp_head.cc:3191 #13 0x000000000077f041 in sp_lex_keeper::reset_lex_and_exec_core (this=0x7f1d040e5fc8, thd=0x28e10f0, nextp=0x7f1d18c75708, open_tables=false, instr=0x7f1d040e5f88) at /opt/data/install/mysql/mysql-5.5.25/sql/sp_head.cc:2988 #14 0x000000000077f6d9 in sp_instr_stmt::execute (this=0x7f1d040e5f88, thd=0x28e10f0, nextp=0x7f1d18c75708) at /opt/data/install/mysql/mysql-5.5.25/sql/sp_head.cc:3123 #15 0x000000000077c951 in sp_head::execute (this=0x7f1d040d66d0, thd=0x28e10f0, merge_da_on_success=true) at /opt/data/install/mysql/mysql-5.5.25/sql/sp_head.cc:1433 #16 0x000000000077e485 in sp_head::execute_procedure (this=0x7f1d040d66d0, thd=, args=0x28e36c0) at /opt/data/install/mysql/mysql-5.5.25/sql/sp_head.cc:2189 #17 0x0000000000599f31 in mysql_execute_command (thd=0x28e10f0) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_parse.cc:4043 #18 0x000000000059b3d1 in mysql_parse (thd=0x28e10f0, rawbuf=0x7f1d04004c50 "CALL test.proc_byrange()", length=, parser_state=0x7f1d18c76ad0) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_parse.cc:5626 #19 0x000000000059c551 in dispatch_command (command=COM_QUERY, thd=0x28e10f0, packet=, packet_length=24) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_parse.cc:1036 #20 0x000000000059da6d in do_command (thd=0x28e10f0) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_parse.cc:773 #21 0x000000000063e8de in do_handle_one_connection (thd_arg=) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_connect.cc:814 #22 0x000000000063e9a4 in handle_one_connection (arg=0x28e10f0) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_connect.cc:733 #23 0x00000000007f1c2e in pfs_spawn_thread (arg=0x28a9780) at /opt/data/install/mysql/mysql-5.5.25/storage/perfschema/pfs.cc:1015 #24 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #25 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f1d17ef2700 (LWP 20425)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x254e558, mp=0x254e4e0, abstime=0x7f1d17ef1e00, file=0xb1b708 "/opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c", line=171) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x0000000000972530 in os_cond_wait_timed (event=0x254e4e0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:171 #3 os_event_wait_time_low (event=0x254e4e0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:716 #4 0x0000000000891602 in srv_lock_timeout_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2287 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f1d1b4fd700 (LWP 20414)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=2, message1=0x7f1d1b4fce58, message2=0x7f1d1b4fce50, type=0x7f1d1b4fce48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=2, message1=0x7f1d1b4fce58, message2=0x7f1d1b4fce50, type=0x7f1d1b4fce48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=2) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f1d200c3700 (LWP 20434)): #0 0x0000003a3ee0f245 in sigwait () from /lib64/libpthread.so.0 #1 0x0000000000514052 in signal_hand (arg=) at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:2682 #2 0x00000000007f1c2e in pfs_spawn_thread (arg=0x283e110) at /opt/data/install/mysql/mysql-5.5.25/storage/perfschema/pfs.cc:1015 #3 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #4 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f1d1aafc700 (LWP 20415)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=3, message1=0x7f1d1aafbe58, message2=0x7f1d1aafbe50, type=0x7f1d1aafbe48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=3, message1=0x7f1d1aafbe58, message2=0x7f1d1aafbe50, type=0x7f1d1aafbe48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=3) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f1d188f3700 (LWP 20428)): #0 0x0000003a3e2de8b3 in select () from /lib64/libc.so.6 #1 0x0000000000972beb in os_thread_sleep (tm=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0thread.c:259 #2 0x0000000000893ca0 in srv_master_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2805 #3 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #4 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f1d1a0fb700 (LWP 20416)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=4, message1=0x7f1d1a0fae58, message2=0x7f1d1a0fae50, type=0x7f1d1a0fae48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=4, message1=0x7f1d1a0fae58, message2=0x7f1d1a0fae50, type=0x7f1d1a0fae48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=4) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f1d16af0700 (LWP 20427)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x254e618, mp=0x254e5a0, abstime=0x7f1d16aefe00, file=0xb1b708 "/opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c", line=171) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x0000000000972530 in os_cond_wait_timed (event=0x254e5a0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:171 #3 os_event_wait_time_low (event=0x254e5a0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:716 #4 0x0000000000892fda in srv_monitor_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2137 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f1d196fa700 (LWP 20417)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=5, message1=0x7f1d196f9e58, message2=0x7f1d196f9e50, type=0x7f1d196f9e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=5, message1=0x7f1d196f9e58, message2=0x7f1d196f9e50, type=0x7f1d196f9e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=5) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f1d18cb8700 (LWP 20455)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x11229a8, mp=0x11223c8, abstime=0x7f1d18cb73c0, file=0xa1eda8 "/opt/data/install/mysql/mysql-5.5.25/include/mysql/psi/mysql_thread.h", line=1009) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x000000000072de1b in inline_mysql_cond_timedwait (this=0x11223c0, thd=, timeout=0x7f1d18cb73c0) at /opt/data/install/mysql/mysql-5.5.25/include/mysql/psi/mysql_thread.h:1009 #3 MYSQL_BIN_LOG::wait_for_update_bin_log (this=0x11223c0, thd=, timeout=0x7f1d18cb73c0) at /opt/data/install/mysql/mysql-5.5.25/sql/log.cc:5631 #4 0x00000000005b32d2 in mysql_binlog_send (thd=0x28da250, log_ident=0x7f1d00004c30 "master-bin.000001", pos=, flags=0) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_repl.cc:881 #5 0x000000000059cc7b in dispatch_command (command=COM_BINLOG_DUMP, thd=0x28da250, packet=0x28dd0e1 "", packet_length=4) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_parse.cc:1213 #6 0x000000000059da6d in do_command (thd=0x28da250) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_parse.cc:773 #7 0x000000000063e8de in do_handle_one_connection (thd_arg=) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_connect.cc:814 #8 0x000000000063e9a4 in handle_one_connection (arg=0x28da250) at /opt/data/install/mysql/mysql-5.5.25/sql/sql_connect.cc:733 #9 0x00000000007f1c2e in pfs_spawn_thread (arg=0x289be80) at /opt/data/install/mysql/mysql-5.5.25/storage/perfschema/pfs.cc:1015 #10 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #11 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f1d174f1700 (LWP 20426)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x254e498, mp=0x254e420, abstime=0x7f1d174f0dd0, file=0xb1b708 "/opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c", line=171) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x0000000000972530 in os_cond_wait_timed (event=0x254e420, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:171 #3 os_event_wait_time_low (event=0x254e420, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:716 #4 0x0000000000891bbf in srv_error_monitor_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2470 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f1d1befe700 (LWP 20413)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=1, message1=0x7f1d1befde58, message2=0x7f1d1befde50, type=0x7f1d1befde48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=1, message1=0x7f1d1befde58, message2=0x7f1d1befde50, type=0x7f1d1befde48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f1d232ee700 (LWP 20412)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=0, message1=0x7f1d232ede58, message2=0x7f1d232ede50, type=0x7f1d232ede48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=0, message1=0x7f1d232ede58, message2=0x7f1d232ede50, type=0x7f1d232ede48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=0) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f1d232f0720 (LWP 20407)): #0 0x0000003a3ee0c63c in pthread_kill () from /lib64/libpthread.so.0 #1 0x00000000007d69d9 in my_write_core (sig=6) at /opt/data/install/mysql/mysql-5.5.25/mysys/stacktrace.c:433 #2 0x00000000006a0b6e in handle_fatal_signal (sig=6) at /opt/data/install/mysql/mysql-5.5.25/sql/signal_handler.cc:247 #3 #4 0x0000003a3e2dc613 in poll () from /lib64/libc.so.6 #5 0x0000000000515020 in handle_connections_sockets () at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:5147 #6 0x000000000051b25a in mysqld_main (argc=50, argv=0x2124ef8) at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:4595 #7 0x000000000051007d in main (argc=, argv=) at /opt/data/install/mysql/mysql-5.5.25/sql/main.cc:25 - found 'core.20410' (1/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace Core generated by '/opt/data/install/mysql/mysql-5.5.25/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- Missing separate debuginfo for Try: yum --disablerepo='*' --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/3b/5709b4dddefce3dd0b4bae733298285d52c602 [New Thread 20410] [New Thread 20418] [New Thread 20422] [New Thread 20420] [New Thread 20435] [New Thread 20419] [New Thread 20453] [New Thread 20421] [New Thread 20433] [New Thread 20423] [New Thread 20432] [New Thread 20431] [New Thread 20430] [New Thread 20454] [Thread debugging using libthread_db enabled] Core was generated by `/opt/data/install/mysql/mysql-5.5.25/sql/mysqld --defaults-group-suffix=.2 --de'. Program terminated with signal 6, Aborted. #0 0x0000003a3ee0c63c in pthread_kill () from /lib64/libpthread.so.0 #0 0x0000003a3ee0c63c in pthread_kill () from /lib64/libpthread.so.0 #1 0x00000000007d69d9 in my_write_core (sig=6) at /opt/data/install/mysql/mysql-5.5.25/mysys/stacktrace.c:433 #2 0x00000000006a0b6e in handle_fatal_signal (sig=6) at /opt/data/install/mysql/mysql-5.5.25/sql/signal_handler.cc:247 #3 #4 0x0000003a3e2dc613 in poll () from /lib64/libc.so.6 #5 0x0000000000515020 in handle_connections_sockets () at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:5147 #6 0x000000000051b25a in mysqld_main (argc=59, argv=0x1483ef8) at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:4595 #7 0x000000000051007d in main (argc=, argv=) at /opt/data/install/mysql/mysql-5.5.25/sql/main.cc:25 Thread 14 (Thread 0x7f0910061700 (LWP 20454)): #0 0x0000003a3ee0ebad in fsync () from /lib64/libpthread.so.0 #1 0x000000000097023f in os_file_fsync (file=9) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:2058 #2 os_file_flush_func (file=9) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:2156 #3 0x0000000000922654 in pfs_os_file_flush_func (space_id=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/include/os0file.ic:381 #4 fil_flush (space_id=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4685 #5 0x000000000095c5d6 in log_write_up_to (lsn=, wait=, flush_to_disk=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/log/log0log.c:1514 #6 0x00000000008b3117 in trx_commit_complete_for_mysql (trx=0x7f08f0011f88) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/trx/trx0trx.c:1724 #7 0x000000000085ce96 in innobase_commit (hton=, thd=, all=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/handler/ha_innodb.cc:2847 #8 0x00000000006a64ed in ha_commit_one_phase (thd=0x7f08f0000a40, all=true) at /opt/data/install/mysql/mysql-5.5.25/sql/handler.cc:1392 #9 0x00000000006a6abc in ha_commit_trans (thd=0x7f08f0000a40, all=true) at /opt/data/install/mysql/mysql-5.5.25/sql/handler.cc:1326 #10 0x000000000064cc95 in trans_commit (thd=0x7f08f0000a40) at /opt/data/install/mysql/mysql-5.5.25/sql/transaction.cc:181 #11 0x000000000074438e in Xid_log_event::do_apply_event (this=0x7f08f00a14b0, rli=) at /opt/data/install/mysql/mysql-5.5.25/sql/log_event.cc:5624 #12 0x000000000052a6c1 in apply_event (ev=0x7f08f00a14b0, thd=0x7f08f0000a40, rli=0x1bee550) at /opt/data/install/mysql/mysql-5.5.25/sql/log_event.h:1132 #13 apply_event_and_update_pos (ev=0x7f08f00a14b0, thd=0x7f08f0000a40, rli=0x1bee550) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:2393 #14 0x000000000052f6a9 in exec_relay_log_event (thd=0x7f08f0000a40, rli=0x1bee550) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:2553 #15 0x0000000000530888 in handle_slave_sql (arg=0x1bed080) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:3377 #16 0x00000000007f1c2e in pfs_spawn_thread (arg=0x7f090000f5f0) at /opt/data/install/mysql/mysql-5.5.25/storage/perfschema/pfs.cc:1015 #17 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #18 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f09077af700 (LWP 20430)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x18ad558, mp=0x18ad4e0, abstime=0x7f09077aee00, file=0xb1b708 "/opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c", line=171) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x0000000000972530 in os_cond_wait_timed (event=0x18ad4e0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:171 #3 os_event_wait_time_low (event=0x18ad4e0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:716 #4 0x0000000000891602 in srv_lock_timeout_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2287 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f0906dae700 (LWP 20431)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x18ad498, mp=0x18ad420, abstime=0x7f0906daddd0, file=0xb1b708 "/opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c", line=171) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x0000000000972530 in os_cond_wait_timed (event=0x18ad420, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:171 #3 os_event_wait_time_low (event=0x18ad420, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:716 #4 0x0000000000891bbf in srv_error_monitor_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2470 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f09063ad700 (LWP 20432)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x18ad618, mp=0x18ad5a0, abstime=0x7f09063ace00, file=0xb1b708 "/opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c", line=171) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x0000000000972530 in os_cond_wait_timed (event=0x18ad5a0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:171 #3 os_event_wait_time_low (event=0x18ad5a0, time_in_usec=, reset_sig_count=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0sync.c:716 #4 0x0000000000892fda in srv_monitor_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2137 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f0908fb7700 (LWP 20423)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=5, message1=0x7f0908fb6e58, message2=0x7f0908fb6e50, type=0x7f0908fb6e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=5, message1=0x7f0908fb6e58, message2=0x7f0908fb6e50, type=0x7f0908fb6e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=5) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f09081b0700 (LWP 20433)): #0 0x0000003a3e2de8b3 in select () from /lib64/libc.so.6 #1 0x0000000000972beb in os_thread_sleep (tm=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0thread.c:259 #2 0x0000000000893ca0 in srv_master_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0srv.c:2805 #3 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #4 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f090a3b9700 (LWP 20421)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=3, message1=0x7f090a3b8e58, message2=0x7f090a3b8e50, type=0x7f090a3b8e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=3, message1=0x7f090a3b8e58, message2=0x7f090a3b8e50, type=0x7f090a3b8e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=3) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f09100a2700 (LWP 20453)): #0 0x0000003a3ee0b75b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00000000007db6b5 in safe_cond_timedwait (cond=0x1bee500, mp=0x1bee3e8, abstime=0x7f09100a1840, file=0xa1eda8 "/opt/data/install/mysql/mysql-5.5.25/include/mysql/psi/mysql_thread.h", line=1009) at /opt/data/install/mysql/mysql-5.5.25/mysys/thr_mutex.c:278 #2 0x000000000053234f in inline_mysql_cond_timedwait (thd=0x7f08fc000a40, mysql=0x7f08fc008160, mi=0x1bed080, reconnect=true, suppress_warnings=false) at /opt/data/install/mysql/mysql-5.5.25/include/mysql/psi/mysql_thread.h:1009 #3 slave_sleep (thd=0x7f08fc000a40, mysql=0x7f08fc008160, mi=0x1bed080, reconnect=true, suppress_warnings=false) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:2143 #4 connect_to_master (thd=0x7f08fc000a40, mysql=0x7f08fc008160, mi=0x1bed080, reconnect=true, suppress_warnings=false) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:4307 #5 0x0000000000532d9c in safe_reconnect (thd=0x7f08fc000a40, mysql=0x7f08fc008160, mi=0x1bed080, retry_count=, suppress_warnings=false, messages=) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:4350 #6 try_to_reconnect (thd=0x7f08fc000a40, mysql=0x7f08fc008160, mi=0x1bed080, retry_count=, suppress_warnings=false, messages=) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:2733 #7 0x000000000053495d in handle_slave_io (arg=0x1bed080) at /opt/data/install/mysql/mysql-5.5.25/sql/slave.cc:2988 #8 0x00000000007f1c2e in pfs_spawn_thread (arg=0x7f090000f5f0) at /opt/data/install/mysql/mysql-5.5.25/storage/perfschema/pfs.cc:1015 #9 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #10 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f090b7bb700 (LWP 20419)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=1, message1=0x7f090b7bae58, message2=0x7f090b7bae50, type=0x7f090b7bae48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=1, message1=0x7f090b7bae58, message2=0x7f090b7bae50, type=0x7f090b7bae48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=1) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f0910165700 (LWP 20435)): #0 0x0000003a3ee0f245 in sigwait () from /lib64/libpthread.so.0 #1 0x0000000000514052 in signal_hand (arg=) at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:2682 #2 0x00000000007f1c2e in pfs_spawn_thread (arg=0x1b9d350) at /opt/data/install/mysql/mysql-5.5.25/storage/perfschema/pfs.cc:1015 #3 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #4 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f090adba700 (LWP 20420)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=2, message1=0x7f090adb9e58, message2=0x7f090adb9e50, type=0x7f090adb9e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=2, message1=0x7f090adb9e58, message2=0x7f090adb9e50, type=0x7f090adb9e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=2) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f09099b8700 (LWP 20422)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=4, message1=0x7f09099b7e58, message2=0x7f09099b7e50, type=0x7f09099b7e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=4, message1=0x7f09099b7e58, message2=0x7f09099b7e50, type=0x7f09099b7e48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=4) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f0912c4e700 (LWP 20418)): #0 0x0000003f3bc00614 in ?? () from /lib64/libaio.so.1 #1 0x000000000096e951 in os_aio_linux_collect (global_seg=0, message1=0x7f0912c4de58, message2=0x7f0912c4de50, type=0x7f0912c4de48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4524 #2 os_aio_linux_handle (global_seg=0, message1=0x7f0912c4de58, message2=0x7f0912c4de50, type=0x7f0912c4de48) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/os/os0file.c:4668 #3 0x00000000009211bf in fil_aio_wait (segment=0) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/fil/fil0fil.c:4561 #4 0x0000000000894d3d in io_handler_thread (arg=) at /opt/data/install/mysql/mysql-5.5.25/storage/innobase/srv/srv0start.c:482 #5 0x0000003a3ee077f1 in start_thread () from /lib64/libpthread.so.0 #6 0x0000003a3e2e5ccd in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f0912c50720 (LWP 20410)): #0 0x0000003a3ee0c63c in pthread_kill () from /lib64/libpthread.so.0 #1 0x00000000007d69d9 in my_write_core (sig=6) at /opt/data/install/mysql/mysql-5.5.25/mysys/stacktrace.c:433 #2 0x00000000006a0b6e in handle_fatal_signal (sig=6) at /opt/data/install/mysql/mysql-5.5.25/sql/signal_handler.cc:247 #3 #4 0x0000003a3e2dc613 in poll () from /lib64/libc.so.6 #5 0x0000000000515020 in handle_connections_sockets () at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:5147 #6 0x000000000051b25a in mysqld_main (argc=59, argv=0x1483ef8) at /opt/data/install/mysql/mysql-5.5.25/sql/mysqld.cc:4595 #7 0x000000000051007d in main (argc=, argv=) at /opt/data/install/mysql/mysql-5.5.25/sql/main.cc:25 Only 1 of 3 completed. mysql-test-run: *** ERROR: Not all tests completed