Bug #54811 assert in mysql_lock_have_duplicate
Submitted: 25 Jun 2010 14:35 Modified: 12 Aug 2010 19:24
Reporter: Matthias Leich Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Locking Severity:S3 (Non-critical)
Version:5.5.5-m3 OS:Any
Assigned to: Konstantin Osipov CPU Architecture:Any
Tags: locking

[25 Jun 2010 14:35] Matthias Leich
Description:
The assert happens in lock.cc:704 :
    /* All tables in list must be in lock. */
    DBUG_ASSERT((table2->lock_position < mylock->table_count) &&  <---
                (table2 == lock_tables[table2->lock_position]));

RQG test with concurrent DDL/DML workload
- 32 threads
- current conf/runtime/WL5004_* grammars

Result on 5.5.5-m3 mysql-trunk-runtime revno: 3062 2010-06-25
-------------------------------------------------------------
Thread 1 (Thread 31949):
#0  0x000000382860bd02 in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000000000b5b191 in my_write_core (sig=6) at stacktrace.c:326
#2  0x0000000000699e71 in handle_segfault (sig=6) at mysqld.cc:2791
#3  <signal handler called>
#4  0x0000003827a30265 in raise () from /lib64/libc.so.6
#5  0x0000003827a31d10 in abort () from /lib64/libc.so.6
#6  0x0000003827a296e6 in __assert_fail () from /lib64/libc.so.6
#7  0x000000000068d150 in mysql_lock_have_duplicate (thd=0x2aaabc112248,
    needle=0x10ff7f68, haystack=0x10ff8b70) at lock.cc:704
#8  0x0000000000705571 in unique_table (thd=0x2aaabc112248, table=0x10ff7f68,
    table_list=0x10ff8b70, check_alias=false) at sql_base.cc:2011
#9  0x000000000075591d in select_insert::prepare (this=0x10ff98c8, values=...,
    u=0x2aaabc113be0) at sql_insert.cc:3145
#10 0x00000000007455f9 in JOIN::prepare (this=0x2aaabc2d7288,
    rref_pointer_array=0x2aaabc1143b0, tables_init=0x10ff8b70, wild_num=0,
    conds_init=0x10ff93c8, og_num=0, order_init=0x0, group_init=0x0,
    having_init=0x0, proc_param_init=0x0, select_lex_arg=0x2aaabc1141e0,
    unit_arg=0x2aaabc113be0) at sql_select.cc:706
#11 0x0000000000745a26 in mysql_select (thd=0x2aaabc112248,
    rref_pointer_array=0x2aaabc1143b0, tables=0x10ff8b70, wild_num=0,
    fields=..., conds=0x10ff93c8, og_num=0, order=0x0, group=0x0, having=0x0,
    proc_param=0x0, select_options=3492023040, result=0x10ff98c8,
    unit=0x2aaabc113be0, select_lex=0x2aaabc1141e0) at sql_select.cc:2528
#12 0x000000000074b207 in handle_select (thd=0x2aaabc112248,
    lex=0x2aaabc113b38, result=0x10ff98c8, setup_tables_done_option=1073741824)
    at sql_select.cc:290
#13 0x00000000006ae7c9 in mysql_execute_command (thd=0x2aaabc112248)
    at sql_parse.cc:3306
#14 0x00000000006b3577 in mysql_parse (thd=0x2aaabc112248,
    inBuf=0x10ff7cd8 "INSERT  IGNORE INTO testdb_S . t1_base1_N  ( `pk` , `col_int` , `col_int_key`  ) SELECT   `col_int` , `pk` , `col_int_key`  FROM testdb_S . t1_temp2_N   A WHERE `pk` BETWEEN 0 AND 0 + 1 LIMIT 1 ON DUP"...,
    length=226, parser_state=0x49e5e9b0) at sql_parse.cc:5911
#15 0x00000000006b4b2a in dispatch_command (command=COM_QUERY,
    thd=0x2aaabc112248,
    packet=0x2aaabc174109 "INSERT  IGNORE INTO testdb_S . t1_base1_N  ( `pk` , `col_int` , `col_int_key`  ) SELECT   `col_int` , `pk` , `col_int_key`  FROM testdb_S . t1_temp2_N   A WHERE `pk` BETWEEN 0 AND 0 + 1 LIMIT 1 ON DUP"...,
    packet_length=226) at sql_parse.cc:1135
#16 0x00000000006b6072 in do_command (thd=0x2aaabc112248) at sql_parse.cc:807
#17 0x00000000006a3f81 in do_handle_one_connection (thd_arg=0x2aaabc112248)
    at sql_connect.cc:1196
#18 0x00000000006a4047 in handle_one_connection (arg=0x2aaabc112248)
    at sql_connect.cc:1135
#19 0x000000382860673d in start_thread () from /lib64/libpthread.so.0
#20 0x0000003827ad3d1d in clone () from /lib64/libc.so.6

How to repeat:
I will upload an archive with log and grammars.
The crash has a low probability.
Please wait till I come up with a simplified
replay testcase.
[25 Jun 2010 14:41] Matthias Leich
Archive with log and grammars

Attachment: problem.tgz (application/x-compressed-tar, text), 119.74 KiB.

[29 Jun 2010 11:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/112443

3065 Konstantin Osipov	2010-06-29
      A fix for Bug#54811 "Assert in mysql_lock_have_duplicate()".
      Remove mysql_lock_have_duplicate(), since now we always
      have TABLE_LIST objects for MyISAMMRG children
      in lex->query_tables and keep it till the end of the
      statement (sub-statement).
     @ sql/lock.cc
        Remove a function that is now unused.
     @ sql/lock.h
        Remove a function that is now unused.
     @ sql/sql_base.cc
        Don't try to search for duplicate table among THR_LOCK objects, TABLE_LIST list contains all used tables.
[29 Jun 2010 14:03] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/112486

3065 Konstantin Osipov	2010-06-29
      A fix for Bug#54811 "Assert in mysql_lock_have_duplicate()".
      Remove mysql_lock_have_duplicate(), since now we always
      have TABLE_LIST objects for MyISAMMRG children
      in lex->query_tables and keep it till the end of the
      statement (sub-statement).
     @ mysql-test/r/merge.result
        Update results (Bug#54811).
     @ mysql-test/t/merge-big.test
        Update to new wait state.
     @ mysql-test/t/merge.test
        Add a test case for Bug#54811.
     @ sql/lock.cc
        Remove a function that is now unused.
     @ sql/lock.h
        Remove a function that is now unused.
     @ sql/sql_base.cc
        Don't try to search for duplicate table among THR_LOCK objects, TABLE_LIST list contains all used tables.
[30 Jun 2010 11:01] Konstantin Osipov
Fixed based on the backtrace.
The patch is queued into trunk-runtime (Currently 5.5.6).
[2 Jul 2010 15:18] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/112771

3074 Konstantin Osipov	2010-07-02
      A test case for Bug#50788 "main.merge fails on HPUX",
      and a backport of relevant changes from the 6.0
      version of the fix done by Ingo Struewing.
      The bug itself was fixed by the patch for Bug#54811.
      
      
      MyISAMMRG engine would try to use MMAP on its children
      even on platforms that don't support it and even if
      myisam_use_mmap option was off.
      This lead to an infinite hang in INSERT ... SELECT into 
      a MyISAMMRG table when the destination MyISAM table
      was also selected from.
      
      A bug in duplicate detection fixed by 54811 was essential to 
      the hang - when a duplicate is detected, the optimizer 
      disables the use of memory mapped files, and it wasn't the case.
      
      The patch below is also to not turn on MMAP on children tables
      if myisam_use_mmap is off.
      A test case is added to cover MyISAMMRG and myisam_use_mmap
      option.
     @ mysql-test/r/merge_mmap.result
        Result file - Bug#50788.
     @ mysql-test/t/merge_mmap-master.opt
        An option file for the test for Bug#50788 -- use mmap.
     @ mysql-test/t/merge_mmap.test
        Try INSERT ... SELECT into a merge table when myisam_use_mmap is on (Bug#50788).
     @ storage/myisam/mi_statrec.c
        Fixed misinterpretation of the return value of my_b_read().
     @ storage/myisammrg/ha_myisammrg.cc
        Skip HA_EXTRA_MMAP if MyISAM memory mapping is disabled.
[2 Jul 2010 15:21] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/112773

3074 Konstantin Osipov	2010-07-02
      A test case for Bug#50788 "main.merge fails on HPUX",
      and a backport of relevant changes from the 6.0
      version of the fix done by Ingo Struewing.
      The bug itself was fixed by the patch for Bug#54811.
      
      
      MyISAMMRG engine would try to use MMAP on its children
      even on platforms that don't support it and even if
      myisam_use_mmap option was off.
      This lead to an infinite hang in INSERT ... SELECT into 
      a MyISAMMRG table when the destination MyISAM table
      was also selected from.
      
      A bug in duplicate detection fixed by 54811 was essential to 
      the hang - when a duplicate is detected, the optimizer 
      disables the use of memory mapped files, and it wasn't the case.
      
      The patch below is also to not turn on MMAP on children tables
      if myisam_use_mmap is off.
      A test case is added to cover MyISAMMRG and myisam_use_mmap
      option.
     @ mysql-test/r/merge_mmap.result
        Result file - Bug#50788.
     @ mysql-test/t/merge_mmap-master.opt
        An option file for the test for Bug#50788 -- use mmap.
     @ mysql-test/t/merge_mmap.test
        Try INSERT ... SELECT into a merge table when myisam_use_mmap is on (Bug#50788).
     @ storage/myisam/mi_statrec.c
        Fixed misinterpretation of the return value of my_b_read().
     @ storage/myisammrg/ha_myisammrg.cc
        Skip HA_EXTRA_MMAP if MyISAM memory mapping is disabled.
[4 Aug 2010 7:51] Bugs System
Pushed into mysql-trunk 5.5.6-m3 (revid:alik@sun.com-20100731131027-1n61gseejyxsqk5d) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 8:08] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804080001-bny5271e65xo34ig) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 8:23] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804081533-c1d3rbipo9e8rt1s) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 9:03] Bugs System
Pushed into mysql-next-mr (revid:alik@ibmvm-20100804081630-ntapn8bf9pko9vj3) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (pib:20)
[12 Aug 2010 19:24] Paul DuBois
Noted in 5.5.6 changelog.

The MERGE storage engine tried to use memory mapping on the
underlying MyISAM tables even on platforms that do not support it and
even when myisam_use_mmap was disabled. This led to a hang for INSERT
INTO ... SELECT FROM statements that selected from a MyISAM table
into a a MERGE table that contained the same MyISAM table.