### 5.6.24 - only debug builds are asserting, release build just breaks replication [umshastr@hod03]/export/umesh/server/binaries/mysql-5.6.24/mysql-test: ./mtr --mysqld=--binlog_format=row bug76727 Logging: ./mtr --mysqld=--binlog_format=row bug76727 2015-04-17 08:25:02 0 [Note] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld (mysqld 5.6.24-enterprise-commercial-advanced) starting as process 15633 ... 2015-04-17 08:25:02 15633 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 08:25:02 15633 [Note] Binlog end 2015-04-17 08:25:02 15633 [Note] Shutting down plugin 'MyISAM' 2015-04-17 08:25:02 15633 [Note] Shutting down plugin 'CSV' MySQL Version 5.6.24 Using binlog format 'row' Checking supported features... - SSL connections supported Collecting tests... Checking leftover processes... Removing old var directory... Creating var directory '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var'... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 include/master-slave.inc Warnings: Note #### Sending passwords in plain text without SSL/TLS is extremely insecure. Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. [connection master] CREATE TABLE t1 (f1 INTEGER PRIMARY KEY); CREATE TABLE t2 (f1 INTEGER PRIMARY KEY); CREATE TABLE t3 (f1 INTEGER PRIMARY KEY); CREATE TRIGGER tr1 AFTER INSERT ON t1 FOR EACH ROW CALL p1(NEW.f1); CREATE PROCEDURE p1 (IN x INT) BEGIN DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK TO event_logging; INSERT t3 VALUES (x); END; SAVEPOINT event_logging; INSERT INTO t2 VALUES (x); RELEASE SAVEPOINT event_logging; END| INSERT INTO t2 VALUES (1); INSERT INTO t1 VALUES (1); main.bug76727 [ fail ] Found warnings/errors in server log file! Test ended at 2015-04-17 08:25:05 line 2015-04-17 09:25:04 15678 [ERROR] Slave SQL: Could not execute Write_rows event on table test.t3; Unknown error, Error_code: 1105; handler error HA_ERR_GENERIC; the event's master log master-bin.000001, end_log_pos 1553, Error_code: 1105 ^ Found warnings in /export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/log/mysqld.2.err ok - saving '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/log/main.bug76727/' to '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/log/main.bug76727/' -------------------------------------------------------------------------- The servers were restarted 0 times Spent 0.000 of 4 seconds executing testcases mysql-test-run: WARNING: Got errors/warnings while running tests, please examine '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/log/warnings' for details. Completed: Failed 1/1 tests, 0.00% were successful. Failing test(s): main.bug76727 The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html mysql-test-run: *** ERROR: there were failing test cases [umshastr@hod03]/export/umesh/server/binaries/mysql-5.6.24/mysql-test: ./mtr --debug --mysqld=--binlog_format=row bug76727 [umshastr@hod03]/export/umesh/server/binaries/mysql-5.6.24/mysql-test: ./mtr --debug --mysqld=--binlog_format=row bug76727 Logging: ./mtr --debug --mysqld=--binlog_format=row bug76727 2015-04-17 08:25:15 0 [Note] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug (mysqld 5.6.24-enterprise-commercial-advanced-debug) starting as process 15721 ... 2015-04-17 08:25:15 15721 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 08:25:15 15721 [Note] Binlog end 2015-04-17 08:25:15 15721 [Note] Shutting down plugin 'CSV' 2015-04-17 08:25:15 15721 [Note] Shutting down plugin 'MyISAM' MySQL Version 5.6.24 Using binlog format 'row' Checking supported features... - SSL connections supported - binaries are debug compiled Collecting tests... Checking leftover processes... Removing old var directory... Creating var directory '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var'... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 include/master-slave.inc Warnings: Note #### Sending passwords in plain text without SSL/TLS is extremely insecure. Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. [connection master] CREATE TABLE t1 (f1 INTEGER PRIMARY KEY); CREATE TABLE t2 (f1 INTEGER PRIMARY KEY); CREATE TABLE t3 (f1 INTEGER PRIMARY KEY); CREATE TRIGGER tr1 AFTER INSERT ON t1 FOR EACH ROW CALL p1(NEW.f1); CREATE PROCEDURE p1 (IN x INT) BEGIN DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK TO event_logging; INSERT t3 VALUES (x); END; SAVEPOINT event_logging; INSERT INTO t2 VALUES (x); RELEASE SAVEPOINT event_logging; END| INSERT INTO t2 VALUES (1); INSERT INTO t1 VALUES (1); main.bug76727 [ fail ] Test ended at 2015-04-17 08:25:26 CURRENT_TEST: main.bug76727 /export/umesh/server/binaries/mysql-5.6.24/bin/mysqltest: WARNING: Option 'debug=d:t:A,/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/log/mysqltest.trace' used, but is disabled Server [mysqld.2 - pid: 15766, winpid: 15766, exit: 256] failed during test run Server log from this test: ----------SERVER LOG START----------- 2015-04-17 09:25:23 0 [Warning] The syntax '--master-retry-count' is deprecated and will be removed in a future release. Please use 'CHANGE MASTER TO master_retry_count = ' instead. 2015-04-17 09:25:23 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2015-04-17 09:25:23 0 [Note] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug (mysqld 5.6.24-enterprise-commercial-advanced-debug-log) starting as process 15767 ... 2015-04-17 09:25:23 15767 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 09:25:23 15767 [Warning] The option innodb (skip-innodb) is deprecated and will be removed in a future release 2015-04-17 09:25:23 7f290e401740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator. 2015-04-17 09:25:23 15767 [Note] InnoDB: Using atomics to ref count buffer pool pages 2015-04-17 09:25:23 7f290e401740 InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!! 2015-04-17 09:25:23 7f290e401740 InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!! 2015-04-17 09:25:23 15767 [Note] InnoDB: The InnoDB memory heap is disabled 2015-04-17 09:25:23 15767 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2015-04-17 09:25:23 15767 [Note] InnoDB: Memory barrier is not used 2015-04-17 09:25:23 15767 [Note] InnoDB: Compressed tables use zlib 1.2.3 2015-04-17 09:25:23 15767 [Note] InnoDB: Using Linux native AIO 2015-04-17 09:25:23 15767 [Note] InnoDB: Using CPU crc32 instructions 2015-04-17 09:25:23 15767 [Note] InnoDB: Initializing buffer pool, size = 8.0M 2015-04-17 09:25:23 15767 [Note] InnoDB: Completed initialization of buffer pool 2015-04-17 09:25:23 15767 [Note] InnoDB: Highest supported file format is Barracuda. 2015-04-17 09:25:24 15767 [Note] InnoDB: 128 rollback segment(s) are active. 2015-04-17 09:25:24 15767 [Note] InnoDB: Waiting for purge to start 2015-04-17 09:25:24 15767 [Note] InnoDB: 5.6.24 started; log sequence number 1625977 2015-04-17 09:25:24 15767 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 882db563-e4ca-11e4-9c19-0010e05f3e06. 2015-04-17 09:25:24 15767 [Note] RSA private key file not found: /export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/mysqld.2/data//private_key.pem. Some authentication plugins will not work. 2015-04-17 09:25:24 15767 [Note] RSA public key file not found: /export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/mysqld.2/data//public_key.pem. Some authentication plugins will not work. 2015-04-17 09:25:24 15767 [Note] Server hostname (bind-address): '*'; port: 13001 2015-04-17 09:25:24 15767 [Note] IPv6 is available. 2015-04-17 09:25:24 15767 [Note] - '::' resolves to '::'; 2015-04-17 09:25:24 15767 [Note] Server socket created on IP: '::'. 2015-04-17 09:25:24 15767 [Note] Event Scheduler: Loaded 0 events 2015-04-17 09:25:24 15767 [Note] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug: ready for connections. Version: '5.6.24-enterprise-commercial-advanced-debug-log' socket: '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/tmp/mysqld.2.sock' port: 13001 MySQL Enterprise Server - Advanced Edition Debug (Commercial) 2015-04-17 09:25:25 15767 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='127.0.0.1', master_port= 13000, master_log_file='master-bin.000001', master_log_pos= 4, master_bind=''. 2015-04-17 09:25:25 15767 [Warning] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. 2015-04-17 09:25:25 15767 [Warning] Slave SQL: If a crash happens this configuration does not guarantee that the relay log info will be consistent, Error_code: 0 2015-04-17 09:25:25 15767 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4 2015-04-17 09:25:25 15767 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:13000',replication started in log 'master-bin.000001' at position 4 mysqld-debug: /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_record.cc:246: int unpack_row(const Relay_log_info*, TABLE*, uint, const uchar*, const MY_BITMAP*, const uchar**, ulong*, const uchar*): Assertion `table_found' failed. 06:25:25 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=4 max_threads=151 thread_count=2 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 60941 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f28d4000a10 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f2904175e50 thread_stack 0x40000 /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(my_print_stacktrace+0x32)[0xae0f0c] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(handle_fatal_signal+0x47a)[0x745d8a] /lib64/libpthread.so.0(+0xf130)[0x7f290dffc130] /lib64/libc.so.6(gsignal+0x39)[0x7f290cbff5c9] /lib64/libc.so.6(abort+0x148)[0x7f290cc00cd8] /lib64/libc.so.6(+0x2e536)[0x7f290cbf8536] /lib64/libc.so.6(+0x2e5e2)[0x7f290cbf85e2] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_Z10unpack_rowPK14Relay_log_infoP5TABLEjPKhPK9st_bitmapPS5_PmS5_+0x284)[0xa7c8f8] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_ZN14Rows_log_event18unpack_current_rowEPK14Relay_log_infoPK9st_bitmap+0xdd)[0xa58f09] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_ZN20Write_rows_log_event9write_rowEPK14Relay_log_infob+0xf7)[0xa34ecd] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_ZN20Write_rows_log_event11do_exec_rowEPK14Relay_log_info+0x5e)[0xa35bd0] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_ZN14Rows_log_event12do_apply_rowEPK14Relay_log_info+0x8b)[0xa2f7bf] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xcd6)[0xa3ce6c] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_ZN9Log_event11apply_eventEP14Relay_log_info+0x39f)[0xa50dfb] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(_Z26apply_event_and_update_posPP9Log_eventP3THDP14Relay_log_info+0x3e1)[0xa96819] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug[0xaa1904] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(handle_slave_sql+0xa2d)[0xaa29f1] /export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug(pfs_spawn_thread+0x18c)[0xdd43e0] /lib64/libpthread.so.0(+0x7df3)[0x7f290dff4df3] /lib64/libc.so.6(clone+0x6d)[0x7f290ccc047d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0): Connection ID (thread ID): 5 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file ----------SERVER LOG END------------- - saving '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/log/main.bug76727/' to '/export/umesh/server/binaries/mysql-5.6.24/mysql-test/var/log/main.bug76727/' - found 'core.15767' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace Core generated by '/export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 15797] [New LWP 15772] [New LWP 15776] [New LWP 15779] [New LWP 15802] [New LWP 15801] [New LWP 15790] [New LWP 15793] [New LWP 15769] [New LWP 15782] [New LWP 15771] [New LWP 15796] [New LWP 15767] [New LWP 15768] [New LWP 15781] [New LWP 15784] [New LWP 15773] [New LWP 15778] [New LWP 15783] [New LWP 15775] [New LWP 15780] [New LWP 15770] [New LWP 15777] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/export/umesh/server/binaries/mysql-5.6.24/bin/mysqld-debug --defaults-group-su'. Program terminated with signal 6, Aborted. #0 0x00007f290dff9771 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007f290dff9771 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000ae0f7f in my_write_core (sig=6) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/mysys/stacktrace.c:422 #2 0x0000000000745f8a in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/signal_handler.cc:230 #3 #4 0x00007f290cbff5c9 in raise () from /lib64/libc.so.6 #5 0x00007f290cc00cd8 in abort () from /lib64/libc.so.6 #6 0x00007f290cbf8536 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f290cbf85e2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000000000a7c8f8 in unpack_row (rli=0x36f3780, table=0x7f28d4057530, colcnt=1, row_data=0x7f28d400bf30 "\376\001", cols=0x7f28d4044d78, current_row_end=0x7f28d4044e80, master_reclength=0x7f28d4044e28, row_end=0x7f28d400bf35 "\177") at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_record.cc:246 #9 0x0000000000a58f09 in Rows_log_event::unpack_current_row (this=0x7f28d4044c60, rli=0x36f3780, cols=0x7f28d4044d78) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.h:4281 #10 0x0000000000a34ecd in Write_rows_log_event::write_row (this=0x7f28d4044c60, rli=0x36f3780, overwrite=false) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:12659 #11 0x0000000000a35bd0 in Write_rows_log_event::do_exec_row (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:12877 #12 0x0000000000a2f7bf in Rows_log_event::do_apply_row (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:10311 #13 0x0000000000a3ce6c in Rows_log_event::do_apply_event (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:11418 #14 0x0000000000a50dfb in Log_event::apply_event (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:3127 #15 0x0000000000a96819 in apply_event_and_update_pos (ptr_ev=0x7f2904175900, thd=0x7f28d4000a10, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:3718 #16 0x0000000000aa1904 in exec_relay_log_event (thd=0x7f28d4000a10, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:4215 #17 0x0000000000aa29f1 in handle_slave_sql (arg=0x36a1160) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:6051 #18 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x7f28e800cd60) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #19 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #20 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7f2902a50700 (LWP 15777)): #0 0x00007f290dff8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000b8956f in os_cond_wait_timed (cond=0x34fcf60, fast_mutex=0x34fcf20, abstime=0x7f2902a4fe80) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:177 #2 0x0000000000b897a4 in os_event_wait_time_low (event=0x34fcf20, time_in_usec=5000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:705 #3 0x0000000000c4587c in srv_monitor_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0srv.cc:1534 #4 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7f2906a7e700 (LWP 15770)): #0 0x00007f290ddeb644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000b83271 in os_aio_linux_collect (array=0x34fdb20, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:4956 #2 0x0000000000b83a36 in os_aio_linux_handle (global_seg=2, message1=0x7f2906a7ded0, message2=0x7f2906a7dec8, type=0x7f2906a7dec0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:5100 #3 0x0000000000d2c41a in fil_aio_wait (segment=2) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000c4caae in io_handler_thread (arg=0x194a4f0 ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0start.cc:485 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f2900c4d700 (LWP 15780)): #0 0x00007f290ccb7c33 in select () from /lib64/libc.so.6 #1 0x0000000000b8a128 in os_thread_sleep (tm=1000000) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0thread.cc:261 #2 0x0000000000cd83f9 in page_cleaner_sleep_if_needed (next_loop_time=1429251926717) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/buf/buf0flu.cc:2375 #3 0x0000000000cdc93c in buf_flush_page_cleaner_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/buf/buf0flu.cc:2416 #4 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f2903e52700 (LWP 15775)): #0 0x00007f290dff8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000b8956f in os_cond_wait_timed (cond=0x3611200, fast_mutex=0x36111c0, abstime=0x7f2903e51eb0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:177 #2 0x0000000000b897a4 in os_event_wait_time_low (event=0x36111c0, time_in_usec=1000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:705 #3 0x0000000000b67745 in lock_wait_timeout_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/lock/lock0wait.cc:503 #4 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f28f2bfd700 (LWP 15783)): #0 0x00007f290dff8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000b8956f in os_cond_wait_timed (cond=0x362ef50, fast_mutex=0x362ef10, abstime=0x7f28f2bfce10) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:177 #2 0x0000000000b897a4 in os_event_wait_time_low (event=0x362ef10, time_in_usec=5000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:705 #3 0x0000000000c8620c in ib_wqueue_timedwait (wq=0x3636098, wait_in_usecs=5000000) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/ut/ut0wqueue.cc:154 #4 0x0000000000d601c6 in fts_optimize_thread (arg=0x3636098) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/fts/fts0opt.cc:3000 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f290204f700 (LWP 15778)): #0 0x00007f290ccb7c33 in select () from /lib64/libc.so.6 #1 0x0000000000b8a128 in os_thread_sleep (tm=1000000) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0thread.cc:261 #2 0x0000000000c436ba in srv_master_sleep () at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0srv.cc:2296 #3 0x0000000000c44277 in srv_master_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0srv.cc:2340 #4 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f2904c7b700 (LWP 15773)): #0 0x00007f290ddeb644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000b83271 in os_aio_linux_collect (array=0x35543f0, segment=1, seg_size=256) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:4956 #2 0x0000000000b83a36 in os_aio_linux_handle (global_seg=5, message1=0x7f2904c7aed0, message2=0x7f2904c7aec8, type=0x7f2904c7aec0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:5100 #3 0x0000000000d2c41a in fil_aio_wait (segment=5) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000c4caae in io_handler_thread (arg=0x194a508 ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0start.cc:485 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f290427a700 (LWP 15784)): #0 0x00007f290dffbed1 in sigwait () from /lib64/libpthread.so.0 #1 0x0000000000632568 in signal_hand (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:3191 #2 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x369be70) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #3 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f28f3fff700 (LWP 15781)): #0 0x00007f290dff8705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000b89857 in os_cond_wait (cond=0x34fcff0, fast_mutex=0x34fcfb0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:214 #2 0x0000000000b898d3 in os_event_wait_low (event=0x34fcfb0, reset_sig_count=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:592 #3 0x0000000000cd6105 in buf_dump_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/buf/buf0dump.cc:594 #4 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f2907e80700 (LWP 15768)): #0 0x00007f290ddeb644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000b83271 in os_aio_linux_collect (array=0x3547130, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:4956 #2 0x0000000000b83a36 in os_aio_linux_handle (global_seg=0, message1=0x7f2907e7fed0, message2=0x7f2907e7fec8, type=0x7f2907e7fec0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:5100 #3 0x0000000000d2c41a in fil_aio_wait (segment=0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000c4caae in io_handler_thread (arg=0x194a4e0 ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0start.cc:485 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f290e401740 (LWP 15767)): #0 0x00007f290ccb5edd in poll () from /lib64/libc.so.6 #1 0x00000000006336c5 in handle_connections_sockets () at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:6260 #2 0x0000000000639f1e in mysqld_main (argc=86, argv=0x33d0998) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:5679 #3 0x00000000006298f3 in main (argc=8, argv=0x7ffff01a18d8) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/main.cc:25 Thread 12 (Thread 0x7f29041b7700 (LWP 15796)): #0 0x00007f290ccb5edd in poll () from /lib64/libc.so.6 #1 0x0000000000ea06db in vio_io_wait (vio=0x7f28e000b310, event=VIO_IO_EVENT_READ, timeout=120000) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/vio/viosocket.c:771 #2 0x0000000000e9efc4 in vio_socket_io_wait (vio=0x7f28e000b310, event=VIO_IO_EVENT_READ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/vio/viosocket.c:68 #3 0x0000000000e9f09f in vio_read (vio=0x7f28e000b310, buf=0x7f28e000b500 "I", size=16384) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/vio/viosocket.c:123 #4 0x0000000000e9f359 in vio_read_buff (vio=0x7f28e000b310, buf=0x7f28e000f510 "", size=4) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/vio/viosocket.c:157 #5 0x000000000073792a in net_read_raw_loop (net=0x7f28e0008b20, count=4) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/net_serv.cc:669 #6 0x0000000000737b0f in net_read_packet_header (net=0x7f28e0008b20) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/net_serv.cc:757 #7 0x00000000007388d1 in net_read_packet (net=0x7f28e0008b20, complen=0x7f29041b6c38) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/net_serv.cc:816 #8 0x0000000000738a4e in my_net_read (net=0x7f28e0008b20) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/net_serv.cc:894 #9 0x00000000008d4896 in cli_safe_read (mysql=0x7f28e0008b20) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql-common/client.c:620 #10 0x0000000000a949b9 in read_event (mysql=0x7f28e0008b20, mi=0x36a1160, suppress_warnings=0x7f29041b6e3b) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:3474 #11 0x0000000000a9e737 in handle_slave_io (arg=0x36a1160) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:4671 #12 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x7f28e800cd60) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #13 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f290607d700 (LWP 15771)): #0 0x00007f290ddeb644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000b83271 in os_aio_linux_collect (array=0x34fdb20, segment=1, seg_size=256) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:4956 #2 0x0000000000b83a36 in os_aio_linux_handle (global_seg=3, message1=0x7f290607ced0, message2=0x7f290607cec8, type=0x7f290607cec0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:5100 #3 0x0000000000d2c41a in fil_aio_wait (segment=3) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000c4caae in io_handler_thread (arg=0x194a4f8 ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0start.cc:485 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f28f35fe700 (LWP 15782)): #0 0x00007f290dff8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000b8956f in os_cond_wait_timed (cond=0x361b900, fast_mutex=0x361b8c0, abstime=0x7f28f35fded0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:177 #2 0x0000000000b897a4 in os_event_wait_time_low (event=0x361b8c0, time_in_usec=10000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:705 #3 0x0000000000d232c2 in dict_stats_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/dict/dict0stats_bg.cc:348 #4 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f290747f700 (LWP 15769)): #0 0x00007f290ddeb644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000b83271 in os_aio_linux_collect (array=0x34f1e70, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:4956 #2 0x0000000000b83a36 in os_aio_linux_handle (global_seg=1, message1=0x7f290747eed0, message2=0x7f290747eec8, type=0x7f290747eec0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:5100 #3 0x0000000000d2c41a in fil_aio_wait (segment=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000c4caae in io_handler_thread (arg=0x194a4e8 ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0start.cc:485 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f29041f8700 (LWP 15793)): #0 0x00007f290dff8705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000000062f1d6 in inline_mysql_cond_wait (that=0x1914ee0 , mutex=0x19128c0 , src_file=0xf98d38 "/export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc", src_line=2688) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/include/mysql/psi/mysql_thread.h:1151 #2 0x000000000063a9ab in block_until_new_connection () at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:2688 #3 0x000000000063ac68 in one_thread_per_connection_end (thd=0x37dac20, block_pthread=true) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:2776 #4 0x00000000007bb284 in do_handle_one_connection (thd_arg=0x37dac20) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:989 #5 0x00000000007bb2e7 in handle_one_connection (arg=0x37dac20) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:898 #6 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x37a4570) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #7 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f2904239700 (LWP 15790)): #0 0x0000000000ea5b6e in nch (p=0x7f2904234810, cs=0x7f28e80232a0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/regex/regcomp.c:1196 #1 0x0000000000ea479f in p_bracket (p=0x7f2904234810) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/regex/regcomp.c:664 #2 0x0000000000ea51de in bothcases (p=0x7f2904234810, ch=98) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/regex/regcomp.c:908 #3 0x0000000000ea5273 in ordinary (p=0x7f2904234810, ch=98) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/regex/regcomp.c:927 #4 0x0000000000ea333e in p_ere_exp (p=0x7f2904234810) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/regex/regcomp.c:336 #5 0x0000000000ea2e86 in p_ere (p=0x7f2904234810, stop=128) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/regex/regcomp.c:235 #6 0x0000000000ea2ccf in my_regcomp (preg=0x7f28e8093a50, pattern=0x7f28e803e8c0 "Enabling keys got errno", cflags=7, charset=0x17acce0 ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/regex/regcomp.c:180 #7 0x0000000000691e69 in Item_func_regex::regcomp (this=0x7f28e8093980, send_error=false) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/item_cmpfunc.cc:5479 #8 0x0000000000692005 in Item_func_regex::val_int (this=0x7f28e8093980) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/item_cmpfunc.cc:5557 #9 0x00000000007c9966 in evaluate_join_record (join=0x7f28e804f8b8, join_tab=0x7f28e8019bc0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_executor.cc:1461 #10 0x00000000007ce060 in sub_select (join=0x7f28e804f8b8, join_tab=0x7f28e8019bc0, end_of_records=false) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_executor.cc:1276 #11 0x00000000007c9e10 in evaluate_join_record (join=0x7f28e804f8b8, join_tab=0x7f28e80198c0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_executor.cc:1601 #12 0x00000000007ce060 in sub_select (join=0x7f28e804f8b8, join_tab=0x7f28e80198c0, end_of_records=false) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_executor.cc:1276 #13 0x00000000007cf90b in do_select (join=0x7f28e804f8b8) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_executor.cc:933 #14 0x00000000007d0217 in JOIN::exec (this=0x7f28e804f8b8) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_executor.cc:194 #15 0x0000000000832cac in mysql_execute_select (thd=0x3703120, select_lex=0x7f28e8098628, free_join=true) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_select.cc:1100 #16 0x000000000083366b in mysql_select (thd=0x3703120, tables=0x7f28e808d420, wild_num=0, fields=..., conds=0x7f28e8093ae0, order=0x0, group=0x0, having=0x0, select_options=1342177408, result=0x7f28e804f7e8, unit=0x7f28e8097fe0, select_lex=0x7f28e8098628) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_select.cc:1221 #17 0x000000000088a79a in mysql_multi_update (thd=0x3703120, table_list=0x7f28e808d420, fields=0x7f28e8098748, values=0x7f28e8098b78, conds=0x7f28e8093ae0, options=0, handle_duplicates=DUP_ERROR, ignore=false, unit=0x7f28e8097fe0, select_lex=0x7f28e8098628, result=0x7f2904236548) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_update.cc:1526 #18 0x00000000007fca2b in mysql_execute_command (thd=0x3703120) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_parse.cc:3385 #19 0x00000000009b4b06 in sp_instr_stmt::exec_core (this=0x7f28e8093c78, thd=0x3703120, nextp=0x7f2904236be8) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sp_instr.cc:904 #20 0x00000000009b5df3 in sp_lex_instr::reset_lex_and_exec_core (this=0x7f28e8093c78, thd=0x3703120, nextp=0x7f2904236be8, open_tables=false) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sp_instr.cc:390 #21 0x00000000009b60c2 in sp_lex_instr::validate_lex_and_execute_core (this=0x7f28e8093c78, thd=0x3703120, nextp=0x7f2904236be8, open_tables=false) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sp_instr.cc:641 #22 0x00000000009b64bc in sp_instr_stmt::execute (this=0x7f28e8093c78, thd=0x3703120, nextp=0x7f2904236be8) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sp_instr.cc:811 #23 0x000000000075037a in sp_head::execute (this=0x7f28e808bf30, thd=0x3703120, merge_da_on_success=true) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sp_head.cc:648 #24 0x000000000075131a in sp_head::execute_procedure (this=0x7f28e808bf30, thd=0x3703120, args=0x3705b30) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sp_head.cc:1320 #25 0x000000000080068b in mysql_execute_command (thd=0x3703120) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_parse.cc:4605 #26 0x0000000000802212 in mysql_parse (thd=0x3703120, rawbuf=0x7f28e8004fd0 "call mtr.check_warnings(@result)", length=32, parser_state=0x7f2904238700) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_parse.cc:6391 #27 0x0000000000803e36 in dispatch_command (command=COM_QUERY, thd=0x3703120, packet=0x37d68f1 "call mtr.check_warnings(@result)", packet_length=32) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_parse.cc:1340 #28 0x00000000008056c0 in do_command (thd=0x3703120) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_parse.cc:1037 #29 0x00000000007bb221 in do_handle_one_connection (thd_arg=0x3703120) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:982 #30 0x00000000007bb2e7 in handle_one_connection (arg=0x3703120) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:898 #31 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x3763b20) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #32 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #33 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f2904135700 (LWP 15801)): #0 0x00007f290dff8705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000000062f1d6 in inline_mysql_cond_wait (that=0x1914ee0 , mutex=0x19128c0 , src_file=0xf98d38 "/export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc", src_line=2688) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/include/mysql/psi/mysql_thread.h:1151 #2 0x000000000063a9ab in block_until_new_connection () at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:2688 #3 0x000000000063ac68 in one_thread_per_connection_end (thd=0x37e2ce0, block_pthread=true) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:2776 #4 0x00000000007bb284 in do_handle_one_connection (thd_arg=0x37e2ce0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:989 #5 0x00000000007bb2e7 in handle_one_connection (arg=0x37e2ce0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:898 #6 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x3751320) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #7 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f29040f4700 (LWP 15802)): #0 0x00007f290dff8705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000000062f1d6 in inline_mysql_cond_wait (that=0x1914ee0 , mutex=0x19128c0 , src_file=0xf98d38 "/export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc", src_line=2688) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/include/mysql/psi/mysql_thread.h:1151 #2 0x000000000063a9ab in block_until_new_connection () at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:2688 #3 0x000000000063ac68 in one_thread_per_connection_end (thd=0x37ea030, block_pthread=true) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/mysqld.cc:2776 #4 0x00000000007bb284 in do_handle_one_connection (thd_arg=0x37ea030) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:989 #5 0x00000000007bb2e7 in handle_one_connection (arg=0x37ea030) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/sql_connect.cc:898 #6 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x37a4570) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #7 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f290164e700 (LWP 15779)): #0 0x00007f290dff8705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000b89857 in os_cond_wait (cond=0x34fce40, fast_mutex=0x34fce00) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:214 #2 0x0000000000b898d3 in os_event_wait_low (event=0x34fce00, reset_sig_count=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:592 #3 0x0000000000c42b65 in srv_purge_coordinator_suspend (slot=0x34fcc10, rseg_history_len=0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0srv.cc:2635 #4 0x0000000000c43075 in srv_purge_coordinator_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0srv.cc:2754 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f2903451700 (LWP 15776)): #0 0x00007f290dff8ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000b8956f in os_cond_wait_timed (cond=0x34fced0, fast_mutex=0x34fce90, abstime=0x7f2903450e70) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:177 #2 0x0000000000b897a4 in os_event_wait_time_low (event=0x34fce90, time_in_usec=1000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0sync.cc:705 #3 0x0000000000c4470c in srv_error_monitor_thread (arg=0x0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0srv.cc:1765 #4 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f290567c700 (LWP 15772)): #0 0x00007f290ddeb644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000b83271 in os_aio_linux_collect (array=0x35543f0, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:4956 #2 0x0000000000b83a36 in os_aio_linux_handle (global_seg=4, message1=0x7f290567bed0, message2=0x7f290567bec8, type=0x7f290567bec0) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/os/os0file.cc:5100 #3 0x0000000000d2c41a in fil_aio_wait (segment=4) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000c4caae in io_handler_thread (arg=0x194a500 ) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/innobase/srv/srv0start.cc:485 #5 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f290ccc047d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f2904176700 (LWP 15797)): #0 0x00007f290dff9771 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000ae0f7f in my_write_core (sig=6) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/mysys/stacktrace.c:422 #2 0x0000000000745f8a in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/signal_handler.cc:230 #3 #4 0x00007f290cbff5c9 in raise () from /lib64/libc.so.6 #5 0x00007f290cc00cd8 in abort () from /lib64/libc.so.6 #6 0x00007f290cbf8536 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f290cbf85e2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000000000a7c8f8 in unpack_row (rli=0x36f3780, table=0x7f28d4057530, colcnt=1, row_data=0x7f28d400bf30 "\376\001", cols=0x7f28d4044d78, current_row_end=0x7f28d4044e80, master_reclength=0x7f28d4044e28, row_end=0x7f28d400bf35 "\177") at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_record.cc:246 #9 0x0000000000a58f09 in Rows_log_event::unpack_current_row (this=0x7f28d4044c60, rli=0x36f3780, cols=0x7f28d4044d78) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.h:4281 #10 0x0000000000a34ecd in Write_rows_log_event::write_row (this=0x7f28d4044c60, rli=0x36f3780, overwrite=false) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:12659 #11 0x0000000000a35bd0 in Write_rows_log_event::do_exec_row (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:12877 #12 0x0000000000a2f7bf in Rows_log_event::do_apply_row (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:10311 #13 0x0000000000a3ce6c in Rows_log_event::do_apply_event (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:11418 #14 0x0000000000a50dfb in Log_event::apply_event (this=0x7f28d4044c60, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/log_event.cc:3127 #15 0x0000000000a96819 in apply_event_and_update_pos (ptr_ev=0x7f2904175900, thd=0x7f28d4000a10, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:3718 #16 0x0000000000aa1904 in exec_relay_log_event (thd=0x7f28d4000a10, rli=0x36f3780) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:4215 #17 0x0000000000aa29f1 in handle_slave_sql (arg=0x36a1160) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/sql/rpl_slave.cc:6051 #18 0x0000000000dd43e0 in pfs_spawn_thread (arg=0x7f28e800cd60) at /export/home/pb2/build/sb_0-14808650-1427301633.73/mysqlcom-pro-5.6.24/storage/perfschema/pfs.cc:1860 #19 0x00007f290dff4df3 in start_thread () from /lib64/libpthread.so.0 #20 0x00007f290ccc047d in clone () from /lib64/libc.so.6 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 0.000 of 12 seconds executing testcases Completed: Failed 1/1 tests, 0.00% were successful. Failing test(s): main.bug76727 The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html mysql-test-run: *** ERROR: there were failing test cases [umshastr@hod03]/export/umesh/server/binaries/mysql-5.6.24/mysql-test: ### 5.6.25 - only debug builds are asserting, release build just breaks replication commit: 3cd6fd44125938e9844d08dca9f5f2c293e942e1 date: 2015-04-09 08:15:18 +0300 build-date: 2015-04-09 08:09:18 +0200 short: 3cd6fd4 branch: mysql-5.6 MySQL source 5.6.25 [umshastr@hod03]/export/umesh/server/binaries/mysql-5.6.25/mysql-test: ./mtr --mysqld=--binlog_format=row bug76727 Logging: ./mtr --mysqld=--binlog_format=row bug76727 2015-04-17 08:18:22 0 [Note] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld (mysqld 5.6.25-enterprise-commercial-advanced) starting as process 15084 ... 2015-04-17 08:18:22 15084 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 08:18:22 15084 [Note] Binlog end 2015-04-17 08:18:22 15084 [Note] Shutting down plugin 'CSV' 2015-04-17 08:18:22 15084 [Note] Shutting down plugin 'MyISAM' MySQL Version 5.6.25 Using binlog format 'row' Checking supported features... - SSL connections supported Collecting tests... Checking leftover processes... Removing old var directory... Creating var directory '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var'... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 include/master-slave.inc Warnings: Note #### Sending passwords in plain text without SSL/TLS is extremely insecure. Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. [connection master] CREATE TABLE t1 (f1 INTEGER PRIMARY KEY); CREATE TABLE t2 (f1 INTEGER PRIMARY KEY); CREATE TABLE t3 (f1 INTEGER PRIMARY KEY); CREATE TRIGGER tr1 AFTER INSERT ON t1 FOR EACH ROW CALL p1(NEW.f1); CREATE PROCEDURE p1 (IN x INT) BEGIN DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK TO event_logging; INSERT t3 VALUES (x); END; SAVEPOINT event_logging; INSERT INTO t2 VALUES (x); RELEASE SAVEPOINT event_logging; END| INSERT INTO t2 VALUES (1); INSERT INTO t1 VALUES (1); main.bug76727 [ fail ] Found warnings/errors in server log file! Test ended at 2015-04-17 08:18:26 line 2015-04-17 09:18:25 15129 [ERROR] Slave SQL: Could not execute Write_rows event on table test.t3; Unknown error, Error_code: 1105; handler error HA_ERR_GENERIC; the event's master log master-bin.000001, end_log_pos 1553, Error_code: 1105 ^ Found warnings in /export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/log/mysqld.2.err ok - saving '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/log/main.bug76727/' to '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/log/main.bug76727/' -------------------------------------------------------------------------- The servers were restarted 0 times Spent 0.000 of 4 seconds executing testcases mysql-test-run: WARNING: Got errors/warnings while running tests, please examine '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/log/warnings' for details. Completed: Failed 1/1 tests, 0.00% were successful. Failing test(s): main.bug76727 The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html mysql-test-run: *** ERROR: there were failing test cases [umshastr@hod03]/export/umesh/server/binaries/mysql-5.6.25/mysql-test: ./mtr --debug --mysqld=--binlog_format=row bug76727 Logging: ./mtr --debug --mysqld=--binlog_format=row bug76727 2015-04-17 08:19:41 0 [Note] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug (mysqld 5.6.25-enterprise-commercial-advanced-debug) starting as process 15177 ... 2015-04-17 08:19:41 15177 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 08:19:41 15177 [Note] Binlog end 2015-04-17 08:19:41 15177 [Note] Shutting down plugin 'CSV' 2015-04-17 08:19:41 15177 [Note] Shutting down plugin 'MyISAM' MySQL Version 5.6.25 Using binlog format 'row' Checking supported features... - SSL connections supported - binaries are debug compiled Collecting tests... Checking leftover processes... Removing old var directory... Creating var directory '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var'... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 include/master-slave.inc Warnings: Note #### Sending passwords in plain text without SSL/TLS is extremely insecure. Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. [connection master] CREATE TABLE t1 (f1 INTEGER PRIMARY KEY); CREATE TABLE t2 (f1 INTEGER PRIMARY KEY); CREATE TABLE t3 (f1 INTEGER PRIMARY KEY); CREATE TRIGGER tr1 AFTER INSERT ON t1 FOR EACH ROW CALL p1(NEW.f1); CREATE PROCEDURE p1 (IN x INT) BEGIN DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK TO event_logging; INSERT t3 VALUES (x); END; SAVEPOINT event_logging; INSERT INTO t2 VALUES (x); RELEASE SAVEPOINT event_logging; END| INSERT INTO t2 VALUES (1); INSERT INTO t1 VALUES (1); main.bug76727 [ fail ] Test ended at 2015-04-17 08:19:52 CURRENT_TEST: main.bug76727 /export/umesh/server/binaries/mysql-5.6.25/bin/mysqltest: WARNING: Option 'debug=d:t:A,/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/log/mysqltest.trace' used, but is disabled The server [mysqld.2 - pid: 15221, winpid: 15221, exit: 256] crashed while running 'check warnings' Server log from this test: ----------SERVER LOG START----------- 2015-04-17 09:19:49 0 [Warning] The syntax '--master-retry-count' is deprecated and will be removed in a future release. Please use 'CHANGE MASTER TO master_retry_count = ' instead. 2015-04-17 09:19:49 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2015-04-17 09:19:49 0 [Note] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug (mysqld 5.6.25-enterprise-commercial-advanced-debug-log) starting as process 15222 ... 2015-04-17 09:19:49 15222 [Warning] The option innodb (skip-innodb) is deprecated and will be removed in a future release 2015-04-17 09:19:49 15222 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 09:19:49 7f783b9a6740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator. 2015-04-17 09:19:49 15222 [Note] InnoDB: Using atomics to ref count buffer pool pages 2015-04-17 09:19:49 7f783b9a6740 InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!! 2015-04-17 09:19:49 7f783b9a6740 InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!! 2015-04-17 09:19:49 15222 [Note] InnoDB: The InnoDB memory heap is disabled 2015-04-17 09:19:49 15222 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2015-04-17 09:19:49 15222 [Note] InnoDB: Memory barrier is not used 2015-04-17 09:19:49 15222 [Note] InnoDB: Compressed tables use zlib 1.2.3 2015-04-17 09:19:49 15222 [Note] InnoDB: Using Linux native AIO 2015-04-17 09:19:49 15222 [Note] InnoDB: Using CPU crc32 instructions 2015-04-17 09:19:49 15222 [Note] InnoDB: Initializing buffer pool, size = 8.0M 2015-04-17 09:19:49 15222 [Note] InnoDB: Completed initialization of buffer pool 2015-04-17 09:19:49 15222 [Note] InnoDB: Highest supported file format is Barracuda. 2015-04-17 09:19:50 15222 [Note] InnoDB: 128 rollback segment(s) are active. 2015-04-17 09:19:50 15222 [Note] InnoDB: Waiting for purge to start 2015-04-17 09:19:50 15222 [Note] InnoDB: 5.6.25 started; log sequence number 1625977 2015-04-17 09:19:50 15222 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: c12a84c8-e4c9-11e4-9c14-0010e05f3e06. 2015-04-17 09:19:50 15222 [Note] RSA private key file not found: /export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/mysqld.2/data//private_key.pem. Some authentication plugins will not work. 2015-04-17 09:19:50 15222 [Note] RSA public key file not found: /export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/mysqld.2/data//public_key.pem. Some authentication plugins will not work. 2015-04-17 09:19:50 15222 [Note] Server hostname (bind-address): '*'; port: 13001 2015-04-17 09:19:50 15222 [Note] IPv6 is available. 2015-04-17 09:19:50 15222 [Note] - '::' resolves to '::'; 2015-04-17 09:19:50 15222 [Note] Server socket created on IP: '::'. 2015-04-17 09:19:50 15222 [Note] Event Scheduler: Loaded 0 events 2015-04-17 09:19:50 15222 [Note] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug: ready for connections. Version: '5.6.25-enterprise-commercial-advanced-debug-log' socket: '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/tmp/mysqld.2.sock' port: 13001 MySQL Enterprise Server - Advanced Edition Debug (Commercial) 2015-04-17 09:19:51 15222 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='127.0.0.1', master_port= 13000, master_log_file='master-bin.000001', master_log_pos= 4, master_bind=''. 2015-04-17 09:19:51 15222 [Warning] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. 2015-04-17 09:19:51 15222 [Warning] Slave SQL: If a crash happens this configuration does not guarantee that the relay log info will be consistent, Error_code: 0 2015-04-17 09:19:51 15222 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4 2015-04-17 09:19:51 15222 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:13000',replication started in log 'master-bin.000001' at position 4 mysqld-debug: /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_record.cc:246: int unpack_row(const Relay_log_info*, TABLE*, uint, const uchar*, const MY_BITMAP*, const uchar**, ulong*, const uchar*): Assertion `table_found' failed. 06:19:52 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=4 max_threads=151 thread_count=3 connection_count=1 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 60942 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f7800000a10 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f783171ae50 thread_stack 0x40000 /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(my_print_stacktrace+0x32)[0xae156c] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(handle_fatal_signal+0x47a)[0x745e6a] /lib64/libpthread.so.0(+0xf130)[0x7f783b5a1130] /lib64/libc.so.6(gsignal+0x39)[0x7f783a1a45c9] /lib64/libc.so.6(abort+0x148)[0x7f783a1a5cd8] /lib64/libc.so.6(+0x2e536)[0x7f783a19d536] /lib64/libc.so.6(+0x2e5e2)[0x7f783a19d5e2] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_Z10unpack_rowPK14Relay_log_infoP5TABLEjPKhPK9st_bitmapPS5_PmS5_+0x284)[0xa7ccb4] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_ZN14Rows_log_event18unpack_current_rowEPK14Relay_log_infoPK9st_bitmap+0xdd)[0xa592c5] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_ZN20Write_rows_log_event9write_rowEPK14Relay_log_infob+0xf7)[0xa35289] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_ZN20Write_rows_log_event11do_exec_rowEPK14Relay_log_info+0x5e)[0xa35f8c] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_ZN14Rows_log_event12do_apply_rowEPK14Relay_log_info+0x8b)[0xa2fb7b] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xcd6)[0xa3d228] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_ZN9Log_event11apply_eventEP14Relay_log_info+0x39f)[0xa511b7] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(_Z26apply_event_and_update_posPP9Log_eventP3THDP14Relay_log_info+0x3e1)[0xa96e1f] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug[0xaa1f28] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(handle_slave_sql+0xa2d)[0xaa3015] /export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug(pfs_spawn_thread+0x18c)[0xb93008] /lib64/libpthread.so.0(+0x7df3)[0x7f783b599df3] /lib64/libc.so.6(clone+0x6d)[0x7f783a26547d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0): Connection ID (thread ID): 5 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file ----------SERVER LOG END------------- - saving '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/log/main.bug76727/' to '/export/umesh/server/binaries/mysql-5.6.25/mysql-test/var/log/main.bug76727/' - found 'core.15222' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace Core generated by '/export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 15252] [New LWP 15256] [New LWP 15257] [New LWP 15222] [New LWP 15223] [New LWP 15225] [New LWP 15226] [New LWP 15232] [New LWP 15230] [New LWP 15231] [New LWP 15233] [New LWP 15224] [New LWP 15227] [New LWP 15245] [New LWP 15251] [New LWP 15248] [New LWP 15234] [New LWP 15228] [New LWP 15235] [New LWP 15237] [New LWP 15238] [New LWP 15236] [New LWP 15239] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/export/umesh/server/binaries/mysql-5.6.25/bin/mysqld-debug --defaults-group-su'. Program terminated with signal 6, Aborted. #0 0x00007f783b59e771 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007f783b59e771 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000ae15df in my_write_core (sig=6) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/mysys/stacktrace.c:422 #2 0x000000000074606a in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/signal_handler.cc:230 #3 #4 0x00007f783a1a45c9 in raise () from /lib64/libc.so.6 #5 0x00007f783a1a5cd8 in abort () from /lib64/libc.so.6 #6 0x00007f783a19d536 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f783a19d5e2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000000000a7ccb4 in unpack_row (rli=0x3655ae0, table=0x7f7800056d00, colcnt=1, row_data=0x7f780000bbd0 "\376\001", cols=0x7f780003e158, current_row_end=0x7f780003e260, master_reclength=0x7f780003e208, row_end=0x7f780000bbd5 "\177") at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_record.cc:246 #9 0x0000000000a592c5 in Rows_log_event::unpack_current_row (this=0x7f780003e040, rli=0x3655ae0, cols=0x7f780003e158) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.h:4281 #10 0x0000000000a35289 in Write_rows_log_event::write_row (this=0x7f780003e040, rli=0x3655ae0, overwrite=false) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:12659 #11 0x0000000000a35f8c in Write_rows_log_event::do_exec_row (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:12877 #12 0x0000000000a2fb7b in Rows_log_event::do_apply_row (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:10311 #13 0x0000000000a3d228 in Rows_log_event::do_apply_event (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:11418 #14 0x0000000000a511b7 in Log_event::apply_event (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:3127 #15 0x0000000000a96e1f in apply_event_and_update_pos (ptr_ev=0x7f783171a900, thd=0x7f7800000a10, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:3769 #16 0x0000000000aa1f28 in exec_relay_log_event (thd=0x7f7800000a10, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:4266 #17 0x0000000000aa3015 in handle_slave_sql (arg=0x3600cb0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:6110 #18 0x0000000000b93008 in pfs_spawn_thread (arg=0x7f7814014700) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #19 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #20 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7f783181f700 (LWP 15239)): #0 0x00007f783b5a0ed1 in sigwait () from /lib64/libpthread.so.0 #1 0x0000000000632548 in signal_hand (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:3191 #2 0x0000000000b93008 in pfs_spawn_thread (arg=0x35fb9c0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #3 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7f7828dfa700 (LWP 15236)): #0 0x00007f783b59d705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c238bb in os_cond_wait (cond=0x345c670, fast_mutex=0x345c630) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:214 #2 0x0000000000c23937 in os_event_wait_low (event=0x345c630, reset_sig_count=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:592 #3 0x0000000000d702f9 in buf_dump_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/buf/buf0dump.cc:594 #4 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f78235fe700 (LWP 15238)): #0 0x00007f783b59dab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c235d3 in os_cond_wait_timed (cond=0x358f2f0, fast_mutex=0x358f2b0, abstime=0x7f78235fde10) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c23808 in os_event_wait_time_low (event=0x358f2b0, time_in_usec=5000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:705 #3 0x0000000000d2037c in ib_wqueue_timedwait (wq=0x35986f8, wait_in_usecs=5000000) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/ut/ut0wqueue.cc:154 #4 0x0000000000dfa3ba in fts_optimize_thread (arg=0x35986f8) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/fts/fts0opt.cc:3000 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f7823fff700 (LWP 15237)): #0 0x00007f783b59dab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c235d3 in os_cond_wait_timed (cond=0x357c4e0, fast_mutex=0x357c4a0, abstime=0x7f7823ffeed0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c23808 in os_event_wait_time_low (event=0x357c4a0, time_in_usec=10000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:705 #3 0x0000000000dbd4b6 in dict_stats_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/dict/dict0stats_bg.cc:348 #4 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f78297fb700 (LWP 15235)): #0 0x00007f783a25cc33 in select () from /lib64/libc.so.6 #1 0x0000000000c2418c in os_thread_sleep (tm=1000000) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0thread.cc:261 #2 0x0000000000d725ed in page_cleaner_sleep_if_needed (next_loop_time=1429251592829) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/buf/buf0flu.cc:2375 #3 0x0000000000d76b30 in buf_flush_page_cleaner_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/buf/buf0flu.cc:2416 #4 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f7832220700 (LWP 15228)): #0 0x00007f783b390644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c1d2ed in os_aio_linux_collect (array=0x34b3aa0, segment=1, seg_size=256) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:4957 #2 0x0000000000c1dab2 in os_aio_linux_handle (global_seg=5, message1=0x7f783221fed0, message2=0x7f783221fec8, type=0x7f783221fec0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:5101 #3 0x0000000000dc660e in fil_aio_wait (segment=5) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ce6c1c in io_handler_thread (arg=0x1950588 ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0start.cc:489 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f782a1fc700 (LWP 15234)): #0 0x00007f783b59d705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c238bb in os_cond_wait (cond=0x345c4c0, fast_mutex=0x345c480) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:214 #2 0x0000000000c23937 in os_event_wait_low (event=0x345c480, reset_sig_count=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:592 #3 0x0000000000cdcc79 in srv_purge_coordinator_suspend (slot=0x345c290, rseg_history_len=0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0srv.cc:2635 #4 0x0000000000cdd189 in srv_purge_coordinator_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0srv.cc:2754 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f783179d700 (LWP 15248)): #0 0x00007f783b59d705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000000062f1b6 in inline_mysql_cond_wait (that=0x1914ea0 , mutex=0x1912880 , src_file=0xf98df8 "/export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc", src_line=2688) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/include/mysql/psi/mysql_thread.h:1151 #2 0x000000000063a975 in block_until_new_connection () at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:2688 #3 0x000000000063ac32 in one_thread_per_connection_end (thd=0x373c2b0, block_pthread=true) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:2776 #4 0x00000000007bb404 in do_handle_one_connection (thd_arg=0x373c2b0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:989 #5 0x00000000007bb467 in handle_one_connection (arg=0x373c2b0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:898 #6 0x0000000000b93008 in pfs_spawn_thread (arg=0x36b3aa0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #7 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f783175c700 (LWP 15251)): #0 0x00007f783a25aedd in poll () from /lib64/libc.so.6 #1 0x0000000000ea0f67 in vio_io_wait (vio=0x7f780c00b320, event=VIO_IO_EVENT_READ, timeout=120000) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/vio/viosocket.c:771 #2 0x0000000000e9f850 in vio_socket_io_wait (vio=0x7f780c00b320, event=VIO_IO_EVENT_READ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/vio/viosocket.c:68 #3 0x0000000000e9f92b in vio_read (vio=0x7f780c00b320, buf=0x7f780c00b510 "I", size=16384) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/vio/viosocket.c:123 #4 0x0000000000e9fbe5 in vio_read_buff (vio=0x7f780c00b320, buf=0x7f780c00f520 "", size=4) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/vio/viosocket.c:157 #5 0x0000000000737a0a in net_read_raw_loop (net=0x7f780c008b30, count=4) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/net_serv.cc:669 #6 0x0000000000737bef in net_read_packet_header (net=0x7f780c008b30) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/net_serv.cc:757 #7 0x00000000007389b1 in net_read_packet (net=0x7f780c008b30, complen=0x7f783175bc38) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/net_serv.cc:816 #8 0x0000000000738b2e in my_net_read (net=0x7f780c008b30) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/net_serv.cc:894 #9 0x00000000008d4c5a in cli_safe_read (mysql=0x7f780c008b30) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql-common/client.c:620 #10 0x0000000000a94fbf in read_event (mysql=0x7f780c008b30, mi=0x3600cb0, suppress_warnings=0x7f783175be3b) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:3525 #11 0x0000000000a9ed49 in handle_slave_io (arg=0x3600cb0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:4722 #12 0x0000000000b93008 in pfs_spawn_thread (arg=0x7f7814014700) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #13 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f78317de700 (LWP 15245)): #0 0x00007f783b59d705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000000062f1b6 in inline_mysql_cond_wait (that=0x1914ea0 , mutex=0x1912880 , src_file=0xf98df8 "/export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc", src_line=2688) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/include/mysql/psi/mysql_thread.h:1151 #2 0x000000000063a975 in block_until_new_connection () at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:2688 #3 0x000000000063ac32 in one_thread_per_connection_end (thd=0x36653a0, block_pthread=true) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:2776 #4 0x00000000007bb404 in do_handle_one_connection (thd_arg=0x36653a0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:989 #5 0x00000000007bb467 in handle_one_connection (arg=0x36653a0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:898 #6 0x0000000000b93008 in pfs_spawn_thread (arg=0x36b2e80) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #7 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f7832c21700 (LWP 15227)): #0 0x00007f783b390644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c1d2ed in os_aio_linux_collect (array=0x34b3aa0, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:4957 #2 0x0000000000c1dab2 in os_aio_linux_handle (global_seg=4, message1=0x7f7832c20ed0, message2=0x7f7832c20ec8, type=0x7f7832c20ec0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:5101 #3 0x0000000000dc660e in fil_aio_wait (segment=4) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ce6c1c in io_handler_thread (arg=0x1950580 ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0start.cc:489 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f7834a24700 (LWP 15224)): #0 0x00007f783b390644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c1d2ed in os_aio_linux_collect (array=0x3499540, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:4957 #2 0x0000000000c1dab2 in os_aio_linux_handle (global_seg=1, message1=0x7f7834a23ed0, message2=0x7f7834a23ec8, type=0x7f7834a23ec0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:5101 #3 0x0000000000dc660e in fil_aio_wait (segment=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ce6c1c in io_handler_thread (arg=0x1950568 ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0start.cc:489 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f782abfd700 (LWP 15233)): #0 0x00007f783a25cc33 in select () from /lib64/libc.so.6 #1 0x0000000000c2418c in os_thread_sleep (tm=1000000) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0thread.cc:261 #2 0x0000000000cdd7ce in srv_master_sleep () at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0srv.cc:2296 #3 0x0000000000cde38b in srv_master_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0srv.cc:2340 #4 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f782bfff700 (LWP 15231)): #0 0x00007f783b59dab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c235d3 in os_cond_wait_timed (cond=0x345c550, fast_mutex=0x345c510, abstime=0x7f782bffee70) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c23808 in os_event_wait_time_low (event=0x345c510, time_in_usec=1000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:705 #3 0x0000000000cde820 in srv_error_monitor_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0srv.cc:1765 #4 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f78313f7700 (LWP 15230)): #0 0x00007f783b59dab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c235d3 in os_cond_wait_timed (cond=0x35705e0, fast_mutex=0x35705a0, abstime=0x7f78313f6eb0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c23808 in os_event_wait_time_low (event=0x35705a0, time_in_usec=1000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:705 #3 0x0000000000c017c1 in lock_wait_timeout_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/lock/lock0wait.cc:503 #4 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f782b5fe700 (LWP 15232)): #0 0x00007f783b59dab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c235d3 in os_cond_wait_timed (cond=0x345c5e0, fast_mutex=0x345c5a0, abstime=0x7f782b5fde80) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c23808 in os_event_wait_time_low (event=0x345c5a0, time_in_usec=5000000, reset_sig_count=1) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0sync.cc:705 #3 0x0000000000cdf990 in srv_monitor_thread (arg=0x0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0srv.cc:1534 #4 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f7833622700 (LWP 15226)): #0 0x00007f783b390644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c1d2ed in os_aio_linux_collect (array=0x345d1a0, segment=1, seg_size=256) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:4957 #2 0x0000000000c1dab2 in os_aio_linux_handle (global_seg=3, message1=0x7f7833621ed0, message2=0x7f7833621ec8, type=0x7f7833621ec0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:5101 #3 0x0000000000dc660e in fil_aio_wait (segment=3) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ce6c1c in io_handler_thread (arg=0x1950578 ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0start.cc:489 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f7834023700 (LWP 15225)): #0 0x00007f783b390644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c1d2ed in os_aio_linux_collect (array=0x345d1a0, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:4957 #2 0x0000000000c1dab2 in os_aio_linux_handle (global_seg=2, message1=0x7f7834022ed0, message2=0x7f7834022ec8, type=0x7f7834022ec0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:5101 #3 0x0000000000dc660e in fil_aio_wait (segment=2) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ce6c1c in io_handler_thread (arg=0x1950570 ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0start.cc:489 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f7835425700 (LWP 15223)): #0 0x00007f783b390644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c1d2ed in os_aio_linux_collect (array=0x34a67e0, segment=0, seg_size=256) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:4957 #2 0x0000000000c1dab2 in os_aio_linux_handle (global_seg=0, message1=0x7f7835424ed0, message2=0x7f7835424ec8, type=0x7f7835424ec0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/os/os0file.cc:5101 #3 0x0000000000dc660e in fil_aio_wait (segment=0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ce6c1c in io_handler_thread (arg=0x1950560 ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/innobase/srv/srv0start.cc:489 #5 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f783b9a6740 (LWP 15222)): #0 0x00007f783a25aedd in poll () from /lib64/libc.so.6 #1 0x00000000006336a5 in handle_connections_sockets () at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:6254 #2 0x0000000000639ee8 in mysqld_main (argc=86, argv=0x3331998) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:5673 #3 0x00000000006298d3 in main (argc=8, argv=0x7fffbc7a6e78) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/main.cc:25 Thread 3 (Thread 0x7f7831699700 (LWP 15257)): #0 0x00007f783b59d705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000000000062f1b6 in inline_mysql_cond_wait (that=0x1914ea0 , mutex=0x1912880 , src_file=0xf98df8 "/export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc", src_line=2688) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/include/mysql/psi/mysql_thread.h:1151 #2 0x000000000063a975 in block_until_new_connection () at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:2688 #3 0x000000000063ac32 in one_thread_per_connection_end (thd=0x374b6e0, block_pthread=true) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/mysqld.cc:2776 #4 0x00000000007bb404 in do_handle_one_connection (thd_arg=0x374b6e0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:989 #5 0x00000000007bb467 in handle_one_connection (arg=0x374b6e0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:898 #6 0x0000000000b93008 in pfs_spawn_thread (arg=0x3684930) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #7 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f78316da700 (LWP 15256)): #0 0x0000000000ea4dfc in p_bracket (p=0x7f78316d5810) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/regex/regcomp.c:640 #1 0x0000000000ea5a6a in bothcases (p=0x7f78316d5810, ch=101) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/regex/regcomp.c:908 #2 0x0000000000ea5aff in ordinary (p=0x7f78316d5810, ch=101) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/regex/regcomp.c:927 #3 0x0000000000ea3bca in p_ere_exp (p=0x7f78316d5810) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/regex/regcomp.c:336 #4 0x0000000000ea3712 in p_ere (p=0x7f78316d5810, stop=128) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/regex/regcomp.c:235 #5 0x0000000000ea355b in my_regcomp (preg=0x7f77fc015470, pattern=0x7f77fc043500 "Slave: Column [0-9]* type mismatch", cflags=7, charset=0x17acce0 ) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/regex/regcomp.c:180 #6 0x0000000000691e35 in Item_func_regex::regcomp (this=0x7f77fc0153a0, send_error=false) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/item_cmpfunc.cc:5479 #7 0x0000000000691fd1 in Item_func_regex::val_int (this=0x7f77fc0153a0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/item_cmpfunc.cc:5557 #8 0x00000000007c9ae6 in evaluate_join_record (join=0x7f77fc02c618, join_tab=0x7f77fc039f80) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_executor.cc:1461 #9 0x00000000007ce1e0 in sub_select (join=0x7f77fc02c618, join_tab=0x7f77fc039f80, end_of_records=false) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_executor.cc:1276 #10 0x00000000007c9f90 in evaluate_join_record (join=0x7f77fc02c618, join_tab=0x7f77fc039c80) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_executor.cc:1601 #11 0x00000000007ce1e0 in sub_select (join=0x7f77fc02c618, join_tab=0x7f77fc039c80, end_of_records=false) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_executor.cc:1276 #12 0x00000000007cfa8b in do_select (join=0x7f77fc02c618) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_executor.cc:933 #13 0x00000000007d0397 in JOIN::exec (this=0x7f77fc02c618) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_executor.cc:194 #14 0x0000000000832f5c in mysql_execute_select (thd=0x36653a0, select_lex=0x7f77fc01a048, free_join=true) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_select.cc:1100 #15 0x000000000083391b in mysql_select (thd=0x36653a0, tables=0x7f77fc00e6c0, wild_num=0, fields=..., conds=0x7f77fc015500, order=0x0, group=0x0, having=0x0, select_options=1342177408, result=0x7f77fc02c548, unit=0x7f77fc019a00, select_lex=0x7f77fc01a048) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_select.cc:1221 #16 0x000000000088aa9e in mysql_multi_update (thd=0x36653a0, table_list=0x7f77fc00e6c0, fields=0x7f77fc01a168, values=0x7f77fc01a598, conds=0x7f77fc015500, options=0, handle_duplicates=DUP_ERROR, ignore=false, unit=0x7f77fc019a00, select_lex=0x7f77fc01a048, result=0x7f78316d7548) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_update.cc:1526 #17 0x00000000007fccd7 in mysql_execute_command (thd=0x36653a0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_parse.cc:3380 #18 0x00000000009b4ee6 in sp_instr_stmt::exec_core (this=0x7f77fc015698, thd=0x36653a0, nextp=0x7f78316d7be8) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sp_instr.cc:904 #19 0x00000000009b61d3 in sp_lex_instr::reset_lex_and_exec_core (this=0x7f77fc015698, thd=0x36653a0, nextp=0x7f78316d7be8, open_tables=false) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sp_instr.cc:390 #20 0x00000000009b64a2 in sp_lex_instr::validate_lex_and_execute_core (this=0x7f77fc015698, thd=0x36653a0, nextp=0x7f78316d7be8, open_tables=false) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sp_instr.cc:641 #21 0x00000000009b689c in sp_instr_stmt::execute (this=0x7f77fc015698, thd=0x36653a0, nextp=0x7f78316d7be8) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sp_instr.cc:811 #22 0x000000000075045a in sp_head::execute (this=0x7f77fc00d1d0, thd=0x36653a0, merge_da_on_success=true) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sp_head.cc:648 #23 0x00000000007513fa in sp_head::execute_procedure (this=0x7f77fc00d1d0, thd=0x36653a0, args=0x3667db8) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sp_head.cc:1320 #24 0x0000000000800937 in mysql_execute_command (thd=0x36653a0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_parse.cc:4600 #25 0x00000000008024be in mysql_parse (thd=0x36653a0, rawbuf=0x7f77fc004c40 "call mtr.check_warnings(@result)", length=32, parser_state=0x7f78316d9700) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_parse.cc:6386 #26 0x00000000008040e2 in dispatch_command (command=COM_QUERY, thd=0x36653a0, packet=0x3737f81 "call mtr.check_warnings(@result)", packet_length=32) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_parse.cc:1340 #27 0x000000000080596e in do_command (thd=0x36653a0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_parse.cc:1037 #28 0x00000000007bb3a1 in do_handle_one_connection (thd_arg=0x3744380) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:982 #29 0x00000000007bb467 in handle_one_connection (arg=0x3744380) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/sql_connect.cc:898 #30 0x0000000000b93008 in pfs_spawn_thread (arg=0x3700c50) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #31 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #32 0x00007f783a26547d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f783171b700 (LWP 15252)): #0 0x00007f783b59e771 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000ae15df in my_write_core (sig=6) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/mysys/stacktrace.c:422 #2 0x000000000074606a in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/signal_handler.cc:230 #3 #4 0x00007f783a1a45c9 in raise () from /lib64/libc.so.6 #5 0x00007f783a1a5cd8 in abort () from /lib64/libc.so.6 #6 0x00007f783a19d536 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f783a19d5e2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000000000a7ccb4 in unpack_row (rli=0x3655ae0, table=0x7f7800056d00, colcnt=1, row_data=0x7f780000bbd0 "\376\001", cols=0x7f780003e158, current_row_end=0x7f780003e260, master_reclength=0x7f780003e208, row_end=0x7f780000bbd5 "\177") at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_record.cc:246 #9 0x0000000000a592c5 in Rows_log_event::unpack_current_row (this=0x7f780003e040, rli=0x3655ae0, cols=0x7f780003e158) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.h:4281 #10 0x0000000000a35289 in Write_rows_log_event::write_row (this=0x7f780003e040, rli=0x3655ae0, overwrite=false) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:12659 #11 0x0000000000a35f8c in Write_rows_log_event::do_exec_row (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:12877 #12 0x0000000000a2fb7b in Rows_log_event::do_apply_row (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:10311 #13 0x0000000000a3d228 in Rows_log_event::do_apply_event (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:11418 #14 0x0000000000a511b7 in Log_event::apply_event (this=0x7f780003e040, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/log_event.cc:3127 #15 0x0000000000a96e1f in apply_event_and_update_pos (ptr_ev=0x7f783171a900, thd=0x7f7800000a10, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:3769 #16 0x0000000000aa1f28 in exec_relay_log_event (thd=0x7f7800000a10, rli=0x3655ae0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:4266 #17 0x0000000000aa3015 in handle_slave_sql (arg=0x3600cb0) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/sql/rpl_slave.cc:6110 #18 0x0000000000b93008 in pfs_spawn_thread (arg=0x7f7814014700) at /export/home/pb2/build/sb_0-14944936-1428563642.5/mysqlcom-pro-5.6.25/storage/perfschema/pfs.cc:1860 #19 0x00007f783b599df3 in start_thread () from /lib64/libpthread.so.0 #20 0x00007f783a26547d in clone () from /lib64/libc.so.6 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 0.000 of 13 seconds executing testcases Completed: Failed 1/1 tests, 0.00% were successful. Failing test(s): main.bug76727 The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html mysql-test-run: *** ERROR: there were failing test cases [umshastr@hod03]/export/umesh/server/binaries/mysql-5.6.25/mysql-test: // 5.6.23 debug build [umshastr@hod03]/export/umesh/server/source/mysql-5.6.23/mysql-test: ./mtr --mysqld=--binlog_format=row bug76727 Logging: ./mtr --mysqld=--binlog_format=row bug76727 2015-04-17 07:01:49 8978 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 07:01:49 8978 [Note] Binlog end 2015-04-17 07:01:49 8978 [Note] Shutting down plugin 'CSV' 2015-04-17 07:01:49 8978 [Note] Shutting down plugin 'MyISAM' MySQL Version 5.6.23 Using binlog format 'row' Checking supported features... - SSL connections supported - binaries are debug compiled Collecting tests... Checking leftover processes... Removing old var directory... Creating var directory '/export/umesh/server/source/mysql-5.6.23/mysql-test/var'... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 include/master-slave.inc Warnings: Note #### Sending passwords in plain text without SSL/TLS is extremely insecure. Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. [connection master] CREATE TABLE t1 (f1 INTEGER PRIMARY KEY); CREATE TABLE t2 (f1 INTEGER PRIMARY KEY); CREATE TABLE t3 (f1 INTEGER PRIMARY KEY); CREATE TRIGGER tr1 AFTER INSERT ON t1 FOR EACH ROW CALL p1(NEW.f1); CREATE PROCEDURE p1 (IN x INT) BEGIN DECLARE EXIT HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK TO event_logging; INSERT t3 VALUES (x); END; SAVEPOINT event_logging; INSERT INTO t2 VALUES (x); RELEASE SAVEPOINT event_logging; END| INSERT INTO t2 VALUES (1); INSERT INTO t1 VALUES (1); main.bug76727 [ fail ] Test ended at 2015-04-17 07:01:55 CURRENT_TEST: main.bug76727 The server [mysqld.2 - pid: 9024, winpid: 9024, exit: 256] crashed while running 'check warnings' Server log from this test: ----------SERVER LOG START----------- 2015-04-17 08:01:53 0 [Warning] The syntax '--master-retry-count' is deprecated and will be removed in a future release. Please use 'CHANGE MASTER TO master_retry_count = ' instead. 2015-04-17 08:01:53 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2015-04-17 08:01:53 9025 [Note] Plugin 'FEDERATED' is disabled. 2015-04-17 08:01:53 9025 [Warning] The option innodb (skip-innodb) is deprecated and will be removed in a future release 2015-04-17 08:01:53 7f96c1e0e740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator. 2015-04-17 08:01:53 9025 [Note] InnoDB: Using atomics to ref count buffer pool pages 2015-04-17 08:01:53 7f96c1e0e740 InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!! 2015-04-17 08:01:53 7f96c1e0e740 InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!! 2015-04-17 08:01:53 9025 [Note] InnoDB: The InnoDB memory heap is disabled 2015-04-17 08:01:53 9025 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2015-04-17 08:01:53 9025 [Note] InnoDB: Memory barrier is not used 2015-04-17 08:01:53 9025 [Note] InnoDB: Compressed tables use zlib 1.2.3 2015-04-17 08:01:53 9025 [Note] InnoDB: Using Linux native AIO 2015-04-17 08:01:53 9025 [Note] InnoDB: Using CPU crc32 instructions 2015-04-17 08:01:53 9025 [Note] InnoDB: Initializing buffer pool, size = 8.0M 2015-04-17 08:01:53 9025 [Note] InnoDB: Completed initialization of buffer pool 2015-04-17 08:01:53 9025 [Note] InnoDB: Highest supported file format is Barracuda. 2015-04-17 08:01:54 9025 [Note] InnoDB: 128 rollback segment(s) are active. 2015-04-17 08:01:54 9025 [Note] InnoDB: Waiting for purge to start 2015-04-17 08:01:55 9025 [Note] InnoDB: 5.6.23 started; log sequence number 1625977 2015-04-17 08:01:55 9025 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: de2ac859-e4be-11e4-9bcd-0010e05f3e06. 2015-04-17 08:01:55 9025 [Note] Server hostname (bind-address): '*'; port: 13001 2015-04-17 08:01:55 9025 [Note] IPv6 is available. 2015-04-17 08:01:55 9025 [Note] - '::' resolves to '::'; 2015-04-17 08:01:55 9025 [Note] Server socket created on IP: '::'. 2015-04-17 08:01:55 9025 [Note] Event Scheduler: Loaded 0 events 2015-04-17 08:01:55 9025 [Note] /export/umesh/server/source/mysql-5.6.23/sql/mysqld: ready for connections. Version: '5.6.23-debug-log' socket: '/export/umesh/server/source/mysql-5.6.23/mysql-test/var/tmp/mysqld.2.sock' port: 13001 Source distribution 2015-04-17 08:01:55 9025 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='127.0.0.1', master_port= 13000, master_log_file='master-bin.000001', master_log_pos= 4, master_bind=''. 2015-04-17 08:01:55 9025 [Warning] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. 2015-04-17 08:01:55 9025 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:13000',replication started in log 'master-bin.000001' at position 4 2015-04-17 08:01:55 9025 [Warning] Slave SQL: If a crash happens this configuration does not guarantee that the relay log info will be consistent, Error_code: 0 2015-04-17 08:01:55 9025 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4 mysqld: /export/umesh/server/source/mysql-5.6.23/sql/rpl_record.cc:246: int unpack_row(const Relay_log_info*, TABLE*, uint, const uchar*, const MY_BITMAP*, const uchar**, ulong*, const uchar*): Assertion `table_found' failed. 05:01:55 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=4 max_threads=151 thread_count=2 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61111 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f9688000a50 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f96b7d91e50 thread_stack 0x40000 /export/umesh/server/source/mysql-5.6.23/sql/mysqld(my_print_stacktrace+0x35)[0xa9fa4c] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(handle_fatal_signal+0x3f3)[0x72a02b] /lib64/libpthread.so.0(+0xf130)[0x7f96c1a08130] /lib64/libc.so.6(gsignal+0x39)[0x7f96c08135c9] /lib64/libc.so.6(abort+0x148)[0x7f96c0814cd8] /lib64/libc.so.6(+0x2e536)[0x7f96c080c536] /lib64/libc.so.6(+0x2e5e2)[0x7f96c080c5e2] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_Z10unpack_rowPK14Relay_log_infoP5TABLEjPKhPK9st_bitmapPS5_PmS5_+0x283)[0xa3de8a] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_ZN14Rows_log_event18unpack_current_rowEPK14Relay_log_infoPK9st_bitmap+0xdd)[0xa195b5] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_ZN20Write_rows_log_event9write_rowEPK14Relay_log_infob+0x10e)[0xa13d6e] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_ZN20Write_rows_log_event11do_exec_rowEPK14Relay_log_info+0x64)[0xa14a66] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_ZN14Rows_log_event12do_apply_rowEPK14Relay_log_info+0x94)[0xa0e01c] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xd1c)[0xa10d64] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_ZN9Log_event11apply_eventEP14Relay_log_info+0x3b7)[0x9fc121] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(_Z26apply_event_and_update_posPP9Log_eventP3THDP14Relay_log_info+0x3c2)[0xa564f6] /export/umesh/server/source/mysql-5.6.23/sql/mysqld[0xa57a82] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(handle_slave_sql+0xaf8)[0xa5cfd5] /export/umesh/server/source/mysql-5.6.23/sql/mysqld(pfs_spawn_thread+0x159)[0xb68104] /lib64/libpthread.so.0(+0x7df3)[0x7f96c1a00df3] /lib64/libc.so.6(clone+0x6d)[0x7f96c08d447d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0): Connection ID (thread ID): 5 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file ----------SERVER LOG END------------- mysqltest failed but provided no output - saving '/export/umesh/server/source/mysql-5.6.23/mysql-test/var/log/main.bug76727/' to '/export/umesh/server/source/mysql-5.6.23/mysql-test/var/log/main.bug76727/' - found 'core.9025' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace Core generated by '/export/umesh/server/source/mysql-5.6.23/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 9055] [New LWP 9060] [New LWP 9059] [New LWP 9038] [New LWP 9025] [New LWP 9029] [New LWP 9039] [New LWP 9027] [New LWP 9031] [New LWP 9051] [New LWP 9033] [New LWP 9041] [New LWP 9028] [New LWP 9030] [New LWP 9042] [New LWP 9035] [New LWP 9040] [New LWP 9048] [New LWP 9037] [New LWP 9034] [New LWP 9036] [New LWP 9026] [New LWP 9054] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/export/umesh/server/source/mysql-5.6.23/sql/mysqld --defaults-group-suffix=.2'. Program terminated with signal 6, Aborted. #0 0x00007f96c1a05771 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007f96c1a05771 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000a9face in my_write_core (sig=6) at /export/umesh/server/source/mysql-5.6.23/mysys/stacktrace.c:422 #2 0x000000000072a210 in handle_fatal_signal (sig=6) at /export/umesh/server/source/mysql-5.6.23/sql/signal_handler.cc:230 #3 #4 0x00007f96c08135c9 in raise () from /lib64/libc.so.6 #5 0x00007f96c0814cd8 in abort () from /lib64/libc.so.6 #6 0x00007f96c080c536 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f96c080c5e2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000000000a3de8a in unpack_row (rli=0x2053c70, table=0x7f9688056360, colcnt=1, row_data=0x7f9688038ff0 "\376\001", cols=0x7f96880046b8, current_row_end=0x7f96880047c0, master_reclength=0x7f9688004768, row_end=0x7f9688038ff5 "\177") at /export/umesh/server/source/mysql-5.6.23/sql/rpl_record.cc:246 #9 0x0000000000a195b5 in Rows_log_event::unpack_current_row (this=0x7f96880045a0, rli=0x2053c70, cols=0x7f96880046b8) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.h:4281 #10 0x0000000000a13d6e in Write_rows_log_event::write_row (this=0x7f96880045a0, rli=0x2053c70, overwrite=false) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:12650 #11 0x0000000000a14a66 in Write_rows_log_event::do_exec_row (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:12868 #12 0x0000000000a0e01c in Rows_log_event::do_apply_row (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:10302 #13 0x0000000000a10d64 in Rows_log_event::do_apply_event (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:11409 #14 0x00000000009fc121 in Log_event::apply_event (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:3125 #15 0x0000000000a564f6 in apply_event_and_update_pos (ptr_ev=0x7f96b7d91930, thd=0x7f9688000a50, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:3718 #16 0x0000000000a57a82 in exec_relay_log_event (thd=0x7f9688000a50, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:4215 #17 0x0000000000a5cfd5 in handle_slave_sql (arg=0x20516d0) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:6051 #18 0x0000000000b68104 in pfs_spawn_thread (arg=0x7f969c060870) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #19 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #20 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7f96b7dd3700 (LWP 9054)): #0 0x00007f96c08c9edd in poll () from /lib64/libc.so.6 #1 0x0000000000e6f258 in vio_io_wait (vio=0x7f969400b3c0, event=VIO_IO_EVENT_READ, timeout=120000) at /export/umesh/server/source/mysql-5.6.23/vio/viosocket.c:771 #2 0x0000000000e6e06e in vio_socket_io_wait (vio=0x7f969400b3c0, event=VIO_IO_EVENT_READ) at /export/umesh/server/source/mysql-5.6.23/vio/viosocket.c:68 #3 0x0000000000e6e14e in vio_read (vio=0x7f969400b3c0, buf=0x7f969400b5b0 "I", size=16384) at /export/umesh/server/source/mysql-5.6.23/vio/viosocket.c:123 #4 0x0000000000e6e305 in vio_read_buff (vio=0x7f969400b3c0, buf=0x7f969400f5c0 "", size=4) at /export/umesh/server/source/mysql-5.6.23/vio/viosocket.c:157 #5 0x000000000071c57c in net_read_raw_loop (net=0x7f9694008bd0, count=4) at /export/umesh/server/source/mysql-5.6.23/sql/net_serv.cc:669 #6 0x000000000071c77c in net_read_packet_header (net=0x7f9694008bd0) at /export/umesh/server/source/mysql-5.6.23/sql/net_serv.cc:757 #7 0x000000000071c849 in net_read_packet (net=0x7f9694008bd0, complen=0x7f96b7dd2c80) at /export/umesh/server/source/mysql-5.6.23/sql/net_serv.cc:816 #8 0x000000000071c9e9 in my_net_read (net=0x7f9694008bd0) at /export/umesh/server/source/mysql-5.6.23/sql/net_serv.cc:894 #9 0x00000000008afd2b in cli_safe_read (mysql=0x7f9694008bd0) at /export/umesh/server/source/mysql-5.6.23/sql-common/client.c:620 #10 0x0000000000a55cc0 in read_event (mysql=0x7f9694008bd0, mi=0x20516d0, suppress_warnings=0x7f96b7dd2e3b) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:3474 #11 0x0000000000a58fec in handle_slave_io (arg=0x20516d0) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:4671 #12 0x0000000000b68104 in pfs_spawn_thread (arg=0x7f969c060870) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #13 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7f96bba9c700 (LWP 9026)): #0 0x00007f96c17f7644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c04dd2 in os_aio_linux_collect (array=0x1ec1050, segment=0, seg_size=256) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:4956 #2 0x0000000000c051af in os_aio_linux_handle (global_seg=0, message1=0x7f96bba9bed0, message2=0x7f96bba9bec8, type=0x7f96bba9bec0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:5100 #3 0x0000000000dc2d27 in fil_aio_wait (segment=0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ccd8d2 in io_handler_thread (arg=0x184b060 ) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0start.cc:485 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f96b5c6b700 (LWP 9036)): #0 0x00007f96c08cbc33 in select () from /lib64/libc.so.6 #1 0x0000000000c085ee in os_thread_sleep (tm=1000000) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0thread.cc:261 #2 0x0000000000cc9f67 in srv_master_sleep () at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0srv.cc:2296 #3 0x0000000000cca091 in srv_master_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0srv.cc:2340 #4 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f96b706d700 (LWP 9034)): #0 0x00007f96c1a04ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c07203 in os_cond_wait_timed (cond=0x1e76a80, fast_mutex=0x1e76a40, abstime=0x7f96b706ce70) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c07a3e in os_event_wait_time_low (event=0x1e76a40, time_in_usec=1000000, reset_sig_count=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:723 #3 0x0000000000cc8de5 in srv_error_monitor_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0srv.cc:1765 #4 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f96b526a700 (LWP 9037)): #0 0x00007f96c1a04705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c072fd in os_cond_wait (cond=0x1e769f0, fast_mutex=0x1e769b0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:214 #2 0x0000000000c0786d in os_event_wait_low (event=0x1e769b0, reset_sig_count=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:610 #3 0x0000000000cca9ed in srv_purge_coordinator_suspend (slot=0x1e767c0, rseg_history_len=0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0srv.cc:2635 #4 0x0000000000ccaf24 in srv_purge_coordinator_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0srv.cc:2754 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f96b7e55700 (LWP 9048)): #0 0x00007f96c1a04705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000aa3ed6 in safe_cond_wait (cond=0x180d0e0 , mp=0x180a5a0 , file=0xec8a20 "/export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h", line=1151) at /export/umesh/server/source/mysql-5.6.23/mysys/thr_mutex.c:240 #2 0x000000000061ab99 in inline_mysql_cond_wait (that=0x180d0e0 , mutex=0x180a5a0 , src_file=0xec9830 "/export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc", src_line=2682) at /export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h:1151 #3 0x0000000000620324 in block_until_new_connection () at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2682 #4 0x0000000000620623 in one_thread_per_connection_end (thd=0x2063020, block_pthread=true) at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2770 #5 0x000000000079b671 in do_handle_one_connection (thd_arg=0x2063020) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:989 #6 0x000000000079b0df in handle_one_connection (arg=0x2063020) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:898 #7 0x0000000000b68104 in pfs_spawn_thread (arg=0x1fb6d10) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #8 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #9 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f96a6bfd700 (LWP 9040)): #0 0x00007f96c1a04ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c07203 in os_cond_wait_timed (cond=0x1f978d0, fast_mutex=0x1f97890, abstime=0x7f96a6bfced0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c07a3e in os_event_wait_time_low (event=0x1f97890, time_in_usec=10000000, reset_sig_count=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:723 #3 0x0000000000db128a in dict_stats_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/dict/dict0stats_bg.cc:349 #4 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f96b666c700 (LWP 9035)): #0 0x00007f96c1a04ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c07203 in os_cond_wait_timed (cond=0x1e76b10, fast_mutex=0x1e76ad0, abstime=0x7f96b666be80) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c07a3e in os_event_wait_time_low (event=0x1e76ad0, time_in_usec=5000000, reset_sig_count=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:723 #3 0x0000000000cc87e9 in srv_monitor_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0srv.cc:1534 #4 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f96b7e96700 (LWP 9042)): #0 0x00007f96c1a07ed1 in sigwait () from /lib64/libpthread.so.0 #1 0x0000000000621118 in signal_hand (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:3185 #2 0x0000000000b68104 in pfs_spawn_thread (arg=0x1fadf60) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #3 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f96b9298700 (LWP 9030)): #0 0x00007f96c17f7644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c04dd2 in os_aio_linux_collect (array=0x1ece2f0, segment=0, seg_size=256) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:4956 #2 0x0000000000c051af in os_aio_linux_handle (global_seg=4, message1=0x7f96b9297ed0, message2=0x7f96b9297ec8, type=0x7f96b9297ec0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:5100 #3 0x0000000000dc2d27 in fil_aio_wait (segment=4) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ccd8d2 in io_handler_thread (arg=0x184b080 ) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0start.cc:485 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f96ba69a700 (LWP 9028)): #0 0x00007f96c17f7644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c04dd2 in os_aio_linux_collect (array=0x1e74fc0, segment=0, seg_size=256) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:4956 #2 0x0000000000c051af in os_aio_linux_handle (global_seg=2, message1=0x7f96ba699ed0, message2=0x7f96ba699ec8, type=0x7f96ba699ec0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:5100 #3 0x0000000000dc2d27 in fil_aio_wait (segment=2) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ccd8d2 in io_handler_thread (arg=0x184b070 ) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0start.cc:485 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f96a61fc700 (LWP 9041)): #0 0x00007f96c1a04ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c07203 in os_cond_wait_timed (cond=0x1fb4750, fast_mutex=0x1fb4710, abstime=0x7f96a61fbe20) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c07a3e in os_event_wait_time_low (event=0x1fb4710, time_in_usec=5000000, reset_sig_count=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:723 #3 0x0000000000d0df75 in ib_wqueue_timedwait (wq=0x1fb25c8, wait_in_usecs=5000000) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/ut/ut0wqueue.cc:154 #4 0x0000000000df0d59 in fts_optimize_thread (arg=0x1fb25c8) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/fts/fts0opt.cc:3001 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f96b7a6e700 (LWP 9033)): #0 0x00007f96c1a04ab2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c07203 in os_cond_wait_timed (cond=0x1f8b100, fast_mutex=0x1f8b0c0, abstime=0x7f96b7a6deb0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:177 #2 0x0000000000c07a3e in os_event_wait_time_low (event=0x1f8b0c0, time_in_usec=1000000, reset_sig_count=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:723 #3 0x0000000000be5a19 in lock_wait_timeout_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/lock/lock0wait.cc:503 #4 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f96b7e14700 (LWP 9051)): #0 0x00007f96c1a04705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000aa3ed6 in safe_cond_wait (cond=0x180d0e0 , mp=0x180a5a0 , file=0xec8a20 "/export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h", line=1151) at /export/umesh/server/source/mysql-5.6.23/mysys/thr_mutex.c:240 #2 0x000000000061ab99 in inline_mysql_cond_wait (that=0x180d0e0 , mutex=0x180a5a0 , src_file=0xec9830 "/export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc", src_line=2682) at /export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h:1151 #3 0x0000000000620324 in block_until_new_connection () at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2682 #4 0x0000000000620623 in one_thread_per_connection_end (thd=0x2136600, block_pthread=true) at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2770 #5 0x000000000079b671 in do_handle_one_connection (thd_arg=0x2136600) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:989 #6 0x000000000079b0df in handle_one_connection (arg=0x2136600) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:898 #7 0x0000000000b68104 in pfs_spawn_thread (arg=0x20c73c0) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #8 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #9 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f96b8897700 (LWP 9031)): #0 0x00007f96c17f7644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c04dd2 in os_aio_linux_collect (array=0x1ece2f0, segment=1, seg_size=256) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:4956 #2 0x0000000000c051af in os_aio_linux_handle (global_seg=5, message1=0x7f96b8896ed0, message2=0x7f96b8896ec8, type=0x7f96b8896ec0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:5100 #3 0x0000000000dc2d27 in fil_aio_wait (segment=5) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ccd8d2 in io_handler_thread (arg=0x184b088 ) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0start.cc:485 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f96bb09b700 (LWP 9027)): #0 0x00007f96c17f7644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c04dd2 in os_aio_linux_collect (array=0x1e6c6f0, segment=0, seg_size=256) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:4956 #2 0x0000000000c051af in os_aio_linux_handle (global_seg=1, message1=0x7f96bb09aed0, message2=0x7f96bb09aec8, type=0x7f96bb09aec0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:5100 #3 0x0000000000dc2d27 in fil_aio_wait (segment=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ccd8d2 in io_handler_thread (arg=0x184b068 ) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0start.cc:485 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f96a75fe700 (LWP 9039)): #0 0x00007f96c1a04705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000c072fd in os_cond_wait (cond=0x1e76ba0, fast_mutex=0x1e76b60) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:214 #2 0x0000000000c0786d in os_event_wait_low (event=0x1e76b60, reset_sig_count=1) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0sync.cc:610 #3 0x0000000000d60b8d in buf_dump_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/buf/buf0dump.cc:594 #4 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f96b9c99700 (LWP 9029)): #0 0x00007f96c17f7644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000000000c04dd2 in os_aio_linux_collect (array=0x1e74fc0, segment=1, seg_size=256) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:4956 #2 0x0000000000c051af in os_aio_linux_handle (global_seg=3, message1=0x7f96b9c98ed0, message2=0x7f96b9c98ec8, type=0x7f96b9c98ec0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0file.cc:5100 #3 0x0000000000dc2d27 in fil_aio_wait (segment=3) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/fil/fil0fil.cc:5753 #4 0x0000000000ccd8d2 in io_handler_thread (arg=0x184b078 ) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/srv/srv0start.cc:485 #5 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f96c1e0e740 (LWP 9025)): #0 0x00007f96c08c9edd in poll () from /lib64/libc.so.6 #1 0x000000000062676d in handle_connections_sockets () at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:6249 #2 0x00000000006258ce in mysqld_main (argc=85, argv=0x1d64998) at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:5668 #3 0x000000000061a5d0 in main (argc=7, argv=0x7fff659634f8) at /export/umesh/server/source/mysql-5.6.23/sql/main.cc:25 Thread 4 (Thread 0x7f96a7fff700 (LWP 9038)): #0 0x00007f96c08cbc33 in select () from /lib64/libc.so.6 #1 0x0000000000c085ee in os_thread_sleep (tm=1000000) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/os/os0thread.cc:261 #2 0x0000000000d684f9 in page_cleaner_sleep_if_needed (next_loop_time=1429246916019) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/buf/buf0flu.cc:2375 #3 0x0000000000d685ef in buf_flush_page_cleaner_thread (arg=0x0) at /export/umesh/server/source/mysql-5.6.23/storage/innobase/buf/buf0flu.cc:2416 #4 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f96b7d51700 (LWP 9059)): #0 0x00007f96c1a04705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000aa3ed6 in safe_cond_wait (cond=0x180d0e0 , mp=0x180a5a0 , file=0xec8a20 "/export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h", line=1151) at /export/umesh/server/source/mysql-5.6.23/mysys/thr_mutex.c:240 #2 0x000000000061ab99 in inline_mysql_cond_wait (that=0x180d0e0 , mutex=0x180a5a0 , src_file=0xec9830 "/export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc", src_line=2682) at /export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h:1151 #3 0x0000000000620324 in block_until_new_connection () at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2682 #4 0x0000000000620623 in one_thread_per_connection_end (thd=0x213e840, block_pthread=true) at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2770 #5 0x000000000079b671 in do_handle_one_connection (thd_arg=0x213e840) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:989 #6 0x000000000079b0df in handle_one_connection (arg=0x213e840) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:898 #7 0x0000000000b68104 in pfs_spawn_thread (arg=0x20f6810) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #8 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #9 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f96b7d10700 (LWP 9060)): #0 0x00007f96c1a04705 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000000000aa3ed6 in safe_cond_wait (cond=0x180d0e0 , mp=0x180a5a0 , file=0xec8a20 "/export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h", line=1151) at /export/umesh/server/source/mysql-5.6.23/mysys/thr_mutex.c:240 #2 0x000000000061ab99 in inline_mysql_cond_wait (that=0x180d0e0 , mutex=0x180a5a0 , src_file=0xec9830 "/export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc", src_line=2682) at /export/umesh/server/source/mysql-5.6.23/include/mysql/psi/mysql_thread.h:1151 #3 0x0000000000620324 in block_until_new_connection () at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2682 #4 0x0000000000620623 in one_thread_per_connection_end (thd=0x2146010, block_pthread=true) at /export/umesh/server/source/mysql-5.6.23/sql/mysqld.cc:2770 #5 0x000000000079b671 in do_handle_one_connection (thd_arg=0x2146010) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:989 #6 0x000000000079b0df in handle_one_connection (arg=0x2146010) at /export/umesh/server/source/mysql-5.6.23/sql/sql_connect.cc:898 #7 0x0000000000b68104 in pfs_spawn_thread (arg=0x20ff590) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #8 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #9 0x00007f96c08d447d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f96b7d92700 (LWP 9055)): #0 0x00007f96c1a05771 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000a9face in my_write_core (sig=6) at /export/umesh/server/source/mysql-5.6.23/mysys/stacktrace.c:422 #2 0x000000000072a210 in handle_fatal_signal (sig=6) at /export/umesh/server/source/mysql-5.6.23/sql/signal_handler.cc:230 #3 #4 0x00007f96c08135c9 in raise () from /lib64/libc.so.6 #5 0x00007f96c0814cd8 in abort () from /lib64/libc.so.6 #6 0x00007f96c080c536 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f96c080c5e2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000000000a3de8a in unpack_row (rli=0x2053c70, table=0x7f9688056360, colcnt=1, row_data=0x7f9688038ff0 "\376\001", cols=0x7f96880046b8, current_row_end=0x7f96880047c0, master_reclength=0x7f9688004768, row_end=0x7f9688038ff5 "\177") at /export/umesh/server/source/mysql-5.6.23/sql/rpl_record.cc:246 #9 0x0000000000a195b5 in Rows_log_event::unpack_current_row (this=0x7f96880045a0, rli=0x2053c70, cols=0x7f96880046b8) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.h:4281 #10 0x0000000000a13d6e in Write_rows_log_event::write_row (this=0x7f96880045a0, rli=0x2053c70, overwrite=false) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:12650 #11 0x0000000000a14a66 in Write_rows_log_event::do_exec_row (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:12868 #12 0x0000000000a0e01c in Rows_log_event::do_apply_row (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:10302 #13 0x0000000000a10d64 in Rows_log_event::do_apply_event (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:11409 #14 0x00000000009fc121 in Log_event::apply_event (this=0x7f96880045a0, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/log_event.cc:3125 #15 0x0000000000a564f6 in apply_event_and_update_pos (ptr_ev=0x7f96b7d91930, thd=0x7f9688000a50, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:3718 #16 0x0000000000a57a82 in exec_relay_log_event (thd=0x7f9688000a50, rli=0x2053c70) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:4215 #17 0x0000000000a5cfd5 in handle_slave_sql (arg=0x20516d0) at /export/umesh/server/source/mysql-5.6.23/sql/rpl_slave.cc:6051 #18 0x0000000000b68104 in pfs_spawn_thread (arg=0x7f969c060870) at /export/umesh/server/source/mysql-5.6.23/storage/perfschema/pfs.cc:1860 #19 0x00007f96c1a00df3 in start_thread () from /lib64/libpthread.so.0 #20 0x00007f96c08d447d in clone () from /lib64/libc.so.6 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 0.000 of 8 seconds executing testcases Completed: Failed 1/1 tests, 0.00% were successful. Failing test(s): main.bug76727 The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html mysql-test-run: *** ERROR: there were failing test cases [umshastr@hod03]/export/umesh/server/source/mysql-5.6.23/mysql-test: