MySQL Bugs: #46321: Deadlock in tdc_wait_for_old_versions() on non-concurrent DDL on partitions

Bug #46321	Deadlock in tdc_wait_for_old_versions() on non-concurrent DDL on partitions
Submitted:	21 Jul 2009 12:38	Modified:	20 Aug 2009 9:50
Reporter:	Philip Stoev	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	MySQL Server: Locking	Severity:	S2 (Serious)
Version:	5.4	OS:	Any
Assigned to:	Assigned Account	CPU Architecture:	Any
Tags:	locking, mdl, partitioning, regression

Description:
When executing non-concurrent DDL ( CREATE TABLE ... PARTITION BY , DROP TABLE, ALTER DROP PARTITION), all connections hanged as follows:

DDL:

#0  0x000000315b00b309 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000a146a1 in safe_cond_wait (cond=0x1057160, mp=0x1056480, file=0xba686d "sql_base.cc", line=7794) at thr_mutex.c:423
#2  0x00000000006cac9e in tdc_wait_for_old_versions (thd=0x7f0954cb2908, mdl_context=0x7f0954cb29e0) at sql_base.cc:7794
#3  0x00000000006d5c27 in recover_from_failed_open_table_attempt (thd=0x7f0954cb2908, table=0x27686b8, action=OT_BACK_OFF_AND_RETRY) at sql_base.cc:3516
#4  0x00000000006d7bbe in open_tables (thd=0x7f0954cb2908, start=0x7f0950b704e0, counter=0x7f0950b70514, flags=0) at sql_base.cc:3805
#5  0x00000000006d8292 in open_and_lock_tables_derived (thd=0x7f0954cb2908, tables=0x27686b8, derived=true, flags=0) at sql_base.cc:4227
---Type <return> to continue, or q <return> to quit---
#6  0x00000000006878d4 in open_and_lock_tables (thd=0x7f0954cb2908, tables=0x27686b8) at ../../sql/mysql_priv.h:1519
#7  0x000000000067d124 in mysql_execute_command (thd=0x7f0954cb2908) at sql_parse.cc:2636
#8  0x0000000000684920 in mysql_parse (thd=0x7f0954cb2908,
    inBuf=0x2768350 "CREATE TABLE IF NOT EXISTS x ( `int_nokey` INTEGER, `int_key` INTEGER NOT NULL, KEY (`int_key`) ) ENGINE = MEMORY /*!50100  PARTITION BY LIST ( `int_nokey` ) ( PARTITION p0 VALUES IN ( 8, NULL ), PART"..., length=348, found_semicolon=0x7f0950b71f10) at sql_parse.cc:5942
#9  0x000000000068553b in dispatch_command (command=COM_QUERY, thd=0x7f0954cb2908,
    packet=0x7f0954cbd269 "CREATE TABLE IF NOT EXISTS x ( `int_nokey` INTEGER, `int_key` INTEGER NOT NULL, KEY (`int_key`) ) ENGINE = MEMORY /*!50100  PARTITION BY LIST ( `int_nokey` ) ( PARTITION p0 VALUES IN ( 8, NULL ), PART"..., packet_length=348) at sql_parse.cc:1061
#10 0x0000000000686a41 in do_command (thd=0x7f0954cb2908) at sql_parse.cc:743
#11 0x0000000000673b3f in handle_one_connection (arg=0x7f0954cb2908) at sql_connect.cc:1158
#12 0x000000315b0073da in start_thread () from /lib64/libpthread.so.0
#13 0x000000315a4e627d in clone () from /lib64/libc.so.6

DML:

#0  0x000000315b00b309 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000a146a1 in safe_cond_wait (cond=0x1057160, mp=0x1056480, file=0xba686d "sql_base.cc", line=7794) at thr_mutex.c:423
#2  0x00000000006cac9e in tdc_wait_for_old_versions (thd=0x7f0954cc12a8, mdl_context=0x7f0954cc1380) at sql_base.cc:7794
#3  0x00000000006d5c27 in recover_from_failed_open_table_attempt (thd=0x7f0954cc12a8, table=0x29145e0, action=OT_BACK_OFF_AND_RETRY) at sql_base.cc:3516
#4  0x00000000006d7bbe in open_tables (thd=0x7f0954cc12a8, start=0x7f0950b2f210, counter=0x7f0950b2f4b8, flags=0) at sql_base.cc:3805
#5  0x000000000074277c in mysql_update (thd=0x7f0954cc12a8, table_list=0x29145e0, fields=@0x7f0954cc3188, values=@0x7f0954cc3598, conds=0x2914ee8,
    order_num=0, order=0x0, limit=18446744073709551615, handle_duplicates=DUP_ERROR, ignore=false) at sql_update.cc:211
#6  0x000000000067e847 in mysql_execute_command (thd=0x7f0954cc12a8) at sql_parse.cc:3099
#7  0x0000000000684920 in mysql_parse (thd=0x7f0954cc12a8, inBuf=0x29144d0 "UPDATE x SET `int_nokey` = 1 WHERE `int_nokey` = 3", length=50,
    found_semicolon=0x7f0950b30f10) at sql_parse.cc:5942
#8  0x000000000068553b in dispatch_command (command=COM_QUERY, thd=0x7f0954cc12a8,
    packet=0x7f0954ccbc09 "UPDATE x SET `int_nokey` = 1 WHERE `int_nokey` = 3", packet_length=50) at sql_parse.cc:1061
#9  0x0000000000686a41 in do_command (thd=0x7f0954cc12a8) at sql_parse.cc:743
#10 0x0000000000673b3f in handle_one_connection (arg=0x7f0954cc12a8) at sql_connect.cc:1158
#11 0x000000315b0073da in start_thread () from /lib64/libpthread.so.0
#12 0x000000315a4e627d in clone () from /lib64/libc.so.6

Curiously, running SHOW TABLE STATUS causes the deadlock to go away.

How to repeat:
A test case will be uploaded shortly.

Grammar file for bug 46321

Attachment: bug46321.yy (application/octet-stream, text), 2.17 KiB.

To reproduce with the random query generator:

$ perl runall.pl \
  --basedir=/build/bzr/azalea \
  --grammar=conf/bug46321.yy \
  --threads=2 \
  --queries=100000 \
  --mysqld=--log-output=none \
  --mem

The last two options are there only to increase concurrency, the bug is otherwise repeatable without them. If it does not work with 2 threads for you, try 10 threads.

The grammar is structured so that the DDL statements are only issued by the first client, so no concurrent execution of DDL ever happens.

Hello Philip!

After investigating causes behind this deadlock I believe that this is yet another manifestation of already known bug #40181 "Partitions: hang if create index" fix for which was recently queued in one of team trees.

This is confirmed by the fact that I was not able to repeat this deadlock on mysql-next-bugfixing tree after I have applied patch for bug #40181 to it.

So I am closing this bug as "Duplicate".