Description:
Server crashes found during RQG concurrency testing with replication enabled.
My MTR based replay testcase has the following properties:
1. bin logging is enabled
2. Connection con1 executes a stored procedure running
DROP SCHEMA testdb_S
in an "endless" loop.
3. Connection con2 executes a stored procedure running the sequence
CREATE DATABASE testdb_S;
CREATE TABLE testdb_S . r1 ENGINE = InnoDB AS SELECT 1 AS col1;
DROP TABLE testdb_S . r1;
in an "endless" loop.
4. After some period of time connection "default" kills both
connections via
KILL <id of session taken from processlist>
and polls if/till the sessions disappear from the processlist.
The crashes happen usually in that timespan.
5. Some fraction of test runs does not crash.
Some other fraction fails because of warnings.
I guess that we would see more different backtraces in case
the connection con2 would run other DDL.
The problem was observed on
- mysql-trunk May 2015
- mysql-5.7 2015-05-19 commit ba957f682055b63d11a2bfb84f855cae39485c88
with and without the not yet pushed fix for Bug#21069721.
Just two examples of the crashes hit
mysql-5.7 commit ba957f682055b63d11a2bfb84f855cae39485c88
with patch for Bug#21069721
compiled from source with debug
=============================================================
tmp/4.prt .... 14:17:14 UTC - mysqld got signal 11 ;
#3 <signal handler called>
#4 0x00007f8f7a6a835a in __strcmp_ssse3 () from /lib64/libc.so.6
#5 0x000000000173dda3 in ha_innobase::delete_table (this=0x7f8f04028d30, name=0x7f8f62158d70 "./testdb_S/r1")
at <tree>/storage/innobase/handler/ha_innodb.cc:11143
#6 0x0000000000df577a in handler::ha_delete_table (this=0x7f8f04028d30, name=0x7f8f62158d70 "./testdb_S/r1")
at <tree>/sql/handler.cc:4566
#7 0x0000000000def3f6 in ha_delete_table (thd=0x7f8f04000bb0, table_type=0x3836ca0, path=0x7f8f62158d70 "./testdb_S/r1", db=0x7f8f0401fef0 "testdb_S", alias=0x7f8f0401ff20 "r1", generate_warning=false)
at <tree>/sql/handler.cc:2309
#8 0x000000000140698b in quick_rm_table (thd=0x7f8f04000bb0, base=0x3836ca0, db=0x7f8f0401fef0 "testdb_S", table_name=0x7f8f0401ff20 "r1", flags=0)
at <tree>/sql/sql_table.cc:2820
#9 0x00000000012fdf47 in drop_open_table (thd=0x7f8f04000bb0, table=0x7f8f040a1270, db_name=0x7f8f0401fef0 "testdb_S", table_name=0x7f8f0401ff20 "r1")
at <tree>/sql/sql_base.cc:2560
#10 0x00000000015873b7 in Query_result_create::prepare2 (this=0x7f8f040282c8)
at <tree>/sql/sql_insert.cc:2761
#11 0x00000000013d1cae in JOIN::prepare_result (this=0x7f8f04028950)
at <tree>/sql/sql_select.cc:899
#12 0x0000000001338e2e in JOIN::exec (this=0x7f8f04028950)
at <tree>/sql/sql_executor.cc:123
#13 0x00000000013d0891 in handle_query (thd=0x7f8f04000bb0, lex=0x7f8f04020718, result=0x7f8f040282c8, added_options=268435456, removed_options=0)
at <tree>/sql/sql_select.cc:183
#14 0x0000000001381460 in mysql_execute_command (thd=0x7f8f04000bb0)
at <tree>/sql/sql_parse.cc:2867
#15 0x00000000012de1ad in sp_instr_stmt::exec_core (this=0x7f8f04022160, thd=0x7f8f04000bb0, nextp=0x7f8f6215aa38)
at <tree>/sql/sp_instr.cc:923
Query (7f8f040280d0): CREATE TABLE testdb_S . r1 ENGINE = InnoDB AS SELECT 1 AS col1
tmp/9.prt ... 14:19:46 UTC - mysqld got signal 6 ;
2015-05-19 17:19:46 0x7fb2353d3700 InnoDB: Assertion failure in thread 140403374110464 in file ut0lst.h line 280
InnoDB: Failing assertion: ((list).init == UT_LIST_INITIALISED)
#3 <signal handler called>
#4 0x00007fb24d7de4c9 in raise () from /lib64/libc.so.6
#5 0x00007fb24d7df958 in abort () from /lib64/libc.so.6
#6 0x000000000192f93f in ut_dbg_assertion_failed (expr=0x1ecddc0 "((list).init == UT_LIST_INITIALISED)", file=0x1ecdd78 "<tree>/storage/innobase/include/ut0lst.h", line=280) at <tree>/storage/innobase/ut/ut0dbg.cc:67
#7 0x00000000017d46f3 in ut_list_remove<ut_list_base<mem_block_info_t, ut_list_node<mem_block_info_t> mem_block_info_t::*>, GenericGetNode<mem_block_info_t> > (list=..., node=..., get_node=...) at <tree>/storage/innobase/include/ut0lst.h:280
#8 0x00000000017d465c in ut_list_remove<ut_list_base<mem_block_info_t, ut_list_node<mem_block_info_t> mem_block_info_t::*> > (list=..., elem=0x7fb1d80c5030) at <tree>/storage/innobase/include/ut0lst.h:331
#9 0x00000000017d4392 in mem_heap_block_free (heap=0x7fb1d80c5030, block=0x7fb1d80c5030) at <tree>/storage/innobase/mem/mem0mem.cc:439
#10 0x0000000001865aa3 in mem_heap_free (heap=0x7fb1d80c5030) at <tree>/storage/innobase/include/mem0mem.ic:539
#11 0x000000000186bb20 in row_prebuilt_free (prebuilt=0x7fb1d80c50b8, dict_locked=0) at <tree>/storage/innobase/row/row0mysql.cc:1027
#12 0x00000000017338c8 in ha_innobase::close (this=0x7fb1d80c4830) at <tree>/storage/innobase/handler/ha_innodb.cc:5600
#13 0x0000000000df00a2 in handler::ha_close (this=0x7fb1d80c4830) at <tree>/sql/handler.cc:2548
#14 0x000000000146aa8c in closefrm (table=0x7fb1d80c3eb0, free_share=true) at <tree>/sql/table.cc:3120
#15 0x00000000012fb092 in intern_close_table (table=0x7fb1d80c3eb0) at <tree>/sql/sql_base.cc:1117
Query (7fb1d40240f0): DROP SCHEMA testdb_S
How to repeat:
I will upload an archive with the replay testcase,
protocols etc. soon.
Description: Server crashes found during RQG concurrency testing with replication enabled. My MTR based replay testcase has the following properties: 1. bin logging is enabled 2. Connection con1 executes a stored procedure running DROP SCHEMA testdb_S in an "endless" loop. 3. Connection con2 executes a stored procedure running the sequence CREATE DATABASE testdb_S; CREATE TABLE testdb_S . r1 ENGINE = InnoDB AS SELECT 1 AS col1; DROP TABLE testdb_S . r1; in an "endless" loop. 4. After some period of time connection "default" kills both connections via KILL <id of session taken from processlist> and polls if/till the sessions disappear from the processlist. The crashes happen usually in that timespan. 5. Some fraction of test runs does not crash. Some other fraction fails because of warnings. I guess that we would see more different backtraces in case the connection con2 would run other DDL. The problem was observed on - mysql-trunk May 2015 - mysql-5.7 2015-05-19 commit ba957f682055b63d11a2bfb84f855cae39485c88 with and without the not yet pushed fix for Bug#21069721. Just two examples of the crashes hit mysql-5.7 commit ba957f682055b63d11a2bfb84f855cae39485c88 with patch for Bug#21069721 compiled from source with debug ============================================================= tmp/4.prt .... 14:17:14 UTC - mysqld got signal 11 ; #3 <signal handler called> #4 0x00007f8f7a6a835a in __strcmp_ssse3 () from /lib64/libc.so.6 #5 0x000000000173dda3 in ha_innobase::delete_table (this=0x7f8f04028d30, name=0x7f8f62158d70 "./testdb_S/r1") at <tree>/storage/innobase/handler/ha_innodb.cc:11143 #6 0x0000000000df577a in handler::ha_delete_table (this=0x7f8f04028d30, name=0x7f8f62158d70 "./testdb_S/r1") at <tree>/sql/handler.cc:4566 #7 0x0000000000def3f6 in ha_delete_table (thd=0x7f8f04000bb0, table_type=0x3836ca0, path=0x7f8f62158d70 "./testdb_S/r1", db=0x7f8f0401fef0 "testdb_S", alias=0x7f8f0401ff20 "r1", generate_warning=false) at <tree>/sql/handler.cc:2309 #8 0x000000000140698b in quick_rm_table (thd=0x7f8f04000bb0, base=0x3836ca0, db=0x7f8f0401fef0 "testdb_S", table_name=0x7f8f0401ff20 "r1", flags=0) at <tree>/sql/sql_table.cc:2820 #9 0x00000000012fdf47 in drop_open_table (thd=0x7f8f04000bb0, table=0x7f8f040a1270, db_name=0x7f8f0401fef0 "testdb_S", table_name=0x7f8f0401ff20 "r1") at <tree>/sql/sql_base.cc:2560 #10 0x00000000015873b7 in Query_result_create::prepare2 (this=0x7f8f040282c8) at <tree>/sql/sql_insert.cc:2761 #11 0x00000000013d1cae in JOIN::prepare_result (this=0x7f8f04028950) at <tree>/sql/sql_select.cc:899 #12 0x0000000001338e2e in JOIN::exec (this=0x7f8f04028950) at <tree>/sql/sql_executor.cc:123 #13 0x00000000013d0891 in handle_query (thd=0x7f8f04000bb0, lex=0x7f8f04020718, result=0x7f8f040282c8, added_options=268435456, removed_options=0) at <tree>/sql/sql_select.cc:183 #14 0x0000000001381460 in mysql_execute_command (thd=0x7f8f04000bb0) at <tree>/sql/sql_parse.cc:2867 #15 0x00000000012de1ad in sp_instr_stmt::exec_core (this=0x7f8f04022160, thd=0x7f8f04000bb0, nextp=0x7f8f6215aa38) at <tree>/sql/sp_instr.cc:923 Query (7f8f040280d0): CREATE TABLE testdb_S . r1 ENGINE = InnoDB AS SELECT 1 AS col1 tmp/9.prt ... 14:19:46 UTC - mysqld got signal 6 ; 2015-05-19 17:19:46 0x7fb2353d3700 InnoDB: Assertion failure in thread 140403374110464 in file ut0lst.h line 280 InnoDB: Failing assertion: ((list).init == UT_LIST_INITIALISED) #3 <signal handler called> #4 0x00007fb24d7de4c9 in raise () from /lib64/libc.so.6 #5 0x00007fb24d7df958 in abort () from /lib64/libc.so.6 #6 0x000000000192f93f in ut_dbg_assertion_failed (expr=0x1ecddc0 "((list).init == UT_LIST_INITIALISED)", file=0x1ecdd78 "<tree>/storage/innobase/include/ut0lst.h", line=280) at <tree>/storage/innobase/ut/ut0dbg.cc:67 #7 0x00000000017d46f3 in ut_list_remove<ut_list_base<mem_block_info_t, ut_list_node<mem_block_info_t> mem_block_info_t::*>, GenericGetNode<mem_block_info_t> > (list=..., node=..., get_node=...) at <tree>/storage/innobase/include/ut0lst.h:280 #8 0x00000000017d465c in ut_list_remove<ut_list_base<mem_block_info_t, ut_list_node<mem_block_info_t> mem_block_info_t::*> > (list=..., elem=0x7fb1d80c5030) at <tree>/storage/innobase/include/ut0lst.h:331 #9 0x00000000017d4392 in mem_heap_block_free (heap=0x7fb1d80c5030, block=0x7fb1d80c5030) at <tree>/storage/innobase/mem/mem0mem.cc:439 #10 0x0000000001865aa3 in mem_heap_free (heap=0x7fb1d80c5030) at <tree>/storage/innobase/include/mem0mem.ic:539 #11 0x000000000186bb20 in row_prebuilt_free (prebuilt=0x7fb1d80c50b8, dict_locked=0) at <tree>/storage/innobase/row/row0mysql.cc:1027 #12 0x00000000017338c8 in ha_innobase::close (this=0x7fb1d80c4830) at <tree>/storage/innobase/handler/ha_innodb.cc:5600 #13 0x0000000000df00a2 in handler::ha_close (this=0x7fb1d80c4830) at <tree>/sql/handler.cc:2548 #14 0x000000000146aa8c in closefrm (table=0x7fb1d80c3eb0, free_share=true) at <tree>/sql/table.cc:3120 #15 0x00000000012fb092 in intern_close_table (table=0x7fb1d80c3eb0) at <tree>/sql/sql_base.cc:1117 Query (7fb1d40240f0): DROP SCHEMA testdb_S How to repeat: I will upload an archive with the replay testcase, protocols etc. soon.