Bug #52856 | concurrent show columns or show full columns causes a crash!!!1 | ||
---|---|---|---|
Submitted: | 15 Apr 2010 10:10 | Modified: | 7 Jul 2010 19:47 |
Reporter: | Shane Bester (Platinum Quality Contributor) | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Server: Locking | Severity: | S1 (Critical) |
Version: | 5.5.3,5.5.4, 5.6.99 | OS: | Any (XP64, linux) |
Assigned to: | Sergei Glukhov | CPU Architecture: | Any |
Tags: | crash, regression, show full columns |
[15 Apr 2010 10:10]
Shane Bester
[15 Apr 2010 17:43]
Sveta Smirnova
Thank you for the report. Crash verified with different trace: stack_bottom = 0x4826f078 thread_stack 0x20000 ./sbin/mysqld(my_print_stacktrace+0x35)[0x9fcecc] ./sbin/mysqld(handle_segfault+0x2a7)[0x51dc98] /lib64/libpthread.so.0[0x3429e0dd40] ./sbin/mysqld(_ZNK12Field_string8sql_typeER6String+0x15a)[0x6f40c6] ./sbin/mysqld[0x618ed3] ./sbin/mysqld[0x61535c] ./sbin/mysqld(_Z14get_all_tablesP3THDP10TABLE_LISTP4Item+0x246)[0x61605f] ./sbin/mysqld(_Z24get_schema_tables_resultP4JOIN23enum_schema_table_state+0x2f4)[0x623fe1] ./sbin/mysqld(_ZN4JOIN4execEv+0x6da)[0x5e04e0] ./sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x2f9)[0x5e298f] ./sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x1a4)[0x5dad3a] ./sbin/mysqld[0x5b817c] ./sbin/mysqld(_Z21mysql_execute_commandP3THD+0x966)[0x5b07f2] ./sbin/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x280)[0x5ba283] ./sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xb08)[0x5adfd0] ./sbin/mysqld(_Z10do_commandP3THD+0x252)[0x5ad2b3] ./sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x14d)[0x68df2d] ./sbin/mysqld(handle_one_connection+0x33)[0x68ddd9] ./sbin/mysqld(_Z16pfs_spawn_threadPv+0xb4)[0x9d042c] /lib64/libpthread.so.0[0x3429e061b5] /lib64/libc.so.6(clone+0x6d)[0x34292cd39d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at 0x4a6c2f8 = show full columns from t1 thd->thread_id=41176 thd->killed=NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. To repeat run `while true; do ./bin/mysql -uroot -S /tmp/mysql_ssmirnova.sock test -e "show full columns from t1"; done` in 2 terminals
[16 Apr 2010 12:01]
Jon Olav Hauglid
The crash is repeatable by adding the following sync point: === modified file 'sql/sql_show.cc' --- sql/sql_show.cc 2010-04-14 07:40:45 +0000 +++ sql/sql_show.cc 2010-04-16 11:53:49 +0000 @@ -17,6 +17,7 @@ /* Function with list databases, tables or fields */ #include "mysql_priv.h" +#include "debug_sync.h" #include "sql_select.h" // For select_describe #include "sql_show.h" #include "repl_failsafe.h" @@ -4068,6 +4069,7 @@ static int get_schema_column_record(THD /* to satisfy 'field->val_str' ASSERTs */ field->table= show_table; show_table->in_use= thd; + DEBUG_SYNC(thd, "get_schema_column"); if (wild && wild[0] && wild_case_compare(system_charset_info, field->field_name,wild)) and then running the following MTR test case: --source include/have_debug_sync.inc CREATE TABLE t1(a CHAR(255)); connect(con1, localhost, root); SET DEBUG_SYNC= "get_schema_column SIGNAL waiting WAIT_FOR completed"; --send SHOW FULL COLUMNS FROM t1 connection default; SET DEBUG_SYNC= "now WAIT_FOR waiting"; SHOW FULL COLUMNS FROM t1; SET DEBUG_SYNC= "now SIGNAL completed"; connection con1; --reap
[27 Apr 2010 12:31]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/106655 3008 Jon Olav Hauglid 2010-04-27 Bug #52856 concurrent show columns or show full columns causes a crash!!!1 This crash could occur if two connections concurrenly tried to execute SHOW (FULL) COLUMNS. During processing of the statments, details about the table columns are used to populate an information schema table. The problem was that this included temporarily modifying table column data as this was needed by table column code. Since the two connections then modified the same data structures, it was possible to get a thread timing where one connection made table column data set by the other connection, invalid. In more detail, the problem was that get_schema_column_record() runs through all Fields in TABLE_SHARE::field to get column details. For each field, field->table (and field->table->in_use) are set as they are needed inside the field code. The crash would occur if the the timing between two connections were as follows: con1: set field->table and table->in_use con2: set field->table and table->in_use con2: access field->table and table->in_use con2: close table and set table->in_use = NULL con1: access field->table and table->in_use => CRASH This patch fixes the problem by making sure LOCK_open is taken before trying to populate the I_S table with data from TABLE_SHARE fields. This is similar to what is already done when executing the similar "SELECT * FROM information_schema.columns" statement. No test case added. A very exact timing of the execution of the two SHOW (FULL) COLUMNS statements were needed to trigger the error. This was only possible to reliably reproduce using a sync point. See the bug page for details about the sync point and a possible test case. With this patch, the thread timing needed is no longer possible to achieve due to LOCK_open preventing two threads from modifying the TABLE_SHARE fields at the same time.
[5 May 2010 6:51]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/107404 3142 Sergey Glukhov 2010-05-05 Bug#52856 concurrent show columns or show full columns causes a crash!!!1 We should avoid any SHARE fields assignments as this is shared structure and assignments may affect other therads. To avoid this copy of SHARE struct is created and stored into TABLE struct which is used in get_schema_coulumns_record later. @ mysql-test/r/mdl_sync.result test case @ mysql-test/t/mdl_sync.test test case @ sql/sql_show.cc We should avoid any SHARE fields assignments as this is shared structure and assignments may affect other therads. To avoid this copy of SHARE struct is created and stored into TABLE struct which is used in get_schema_coulumns_record later.
[27 May 2010 15:54]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/109399 3068 Sergey Glukhov 2010-05-27 Bug#52856 concurrent show columns or show full columns causes a crash!!!1 We should avoid any SHARE fields assignments as this is shared structure and assignments may affect other therads. To avoid this copy of SHARE struct is created and stored into TABLE struct which is used in get_schema_coulumns_record later. @ mysql-test/r/mdl_sync.result test case @ mysql-test/t/mdl_sync.test test case @ sql/sql_show.cc We should avoid any SHARE fields assignments as this is shared structure and assignments may affect other therads. To avoid this copy of SHARE struct is created and stored into TABLE struct which is used in get_schema_coulumns_record later.
[15 Jun 2010 8:17]
Bugs System
Pushed into 5.5.5-m3 (revid:alik@sun.com-20100615080459-smuswd9ooeywcxuc) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (merge vers: 5.1.47) (pib:16)
[15 Jun 2010 8:34]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100615080558-cw01bzdqr1bdmmec) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (pib:16)
[7 Jul 2010 19:47]
Paul DuBois
Noted in 5.5.5 changelog. Concurrent SHOW COLUMNS statements could cause a server crash.