Bug #51084 Batched key access crashes for SELECT with derived table and LEFT JOIN
Submitted: 11 Feb 2010 10:26 Modified: 23 Nov 2010 3:16
Reporter: Guilhem Bichot Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S3 (Non-critical)
Version:6.0-codebase-bugfixing OS:Linux
Assigned to: Jørgen Løland CPU Architecture:Any
Tags: BKA, join_cache_level, LooseScan, optimizer_switch, semijoin

[11 Feb 2010 10:26] Guilhem Bichot
Description:
I have alik@sun.com-20100209120530-02iqcoowu8yfn2z6
I create the test in "how-to-repeat", I run it with --mysqld=--optimizer_join_cache_level=8 and it crashes:
SELECT COUNT(*) 
  FROM ((t2 JOIN t1 ON t2.package_id = t1.id) 
        JOIN t3 ON t3.package_id = t1.id)
       LEFT JOIN 
       (t5 JOIN t4 ON t5.carrier_id = t4.id)
       ON t4.carrier = t1.carrier;
#2  0x000000000070c61f in handle_segfault (sig=11) at mysqld.cc:2830
#3  <signal handler called>
#4  memset () at ../sysdeps/x86_64/memset.S:65
#5  0x000000000076001c in JOIN_CACHE_BKA_UNIQUE::cleanup_hash_table (this=0x2c10dd8) at sql_join_cache.cc:2890
#6  0x000000000075fa35 in JOIN_CACHE_BKA_UNIQUE::reset (this=0x2c10dd8, for_writing=true) at sql_join_cache.cc:2633
#7  0x000000000075e1cb in JOIN_CACHE::join_records (this=0x2c10dd8, skip_last=false) at sql_join_cache.cc:1698
#8  0x000000000075e0dc in JOIN_CACHE::join_records (this=0x2bf0be8, skip_last=false) at sql_join_cache.cc:1671
#9  0x00000000007c2445 in sub_select_cache (join=0x2bea278, join_tab=0x2bccce8, end_of_records=true) at sql_select.cc:16327
#10 0x00000000007c2603 in sub_select (join=0x2bea278, join_tab=0x2bcca38, end_of_records=true) at sql_select.cc:16490
#11 0x00000000007c1ca5 in do_select (join=0x2bea278, fields=0x2beff18, table=0x0, procedure=0x0) at sql_select.cc:16081
#12 0x00000000007a25a0 in JOIN::exec (this=0x2bea278) at sql_select.cc:2962
#13 0x00000000007a2c48 in mysql_select (thd=0x2a25140, rref_pointer_array=0x2a27328, tables=0x2a27fe8, wild_num=0, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, 
    result=0x2a57958, unit=0x2a26a50, select_lex=0x2a27110) at sql_select.cc:3153
I found this by running join_nested.test with --mysqld=--optimizer_join_cache_level=8

How to repeat:
This comes from the "BUG#29604" piece of join_nested.test.

CREATE TABLE t1 (
  carrier char(2) default NULL,
  id int NOT NULL auto_increment PRIMARY KEY
);
INSERT INTO t1 VALUES
  ('CO',235371754),('CO',235376554),('CO',235376884),('CO',235377874),
  ('CO',231060394),('CO',231059224),('CO',231059314),('CO',231060484),
  ('CO',231060274),('CO',231060124),('CO',231060244),('CO',231058594),
  ('CO',231058924),('CO',231058504),('CO',231059344),('CO',231060424),
  ('CO',231059554),('CO',231060304),('CO',231059644),('CO',231059464),
  ('CO',231059764),('CO',231058294),('CO',231058624),('CO',231058864),
  ('CO',231059374),('CO',231059584),('CO',231059734),('CO',231059014),
  ('CO',231059854),('CO',231059494),('CO',231059794),('CO',231058534),
  ('CO',231058324),('CO',231058684),('CO',231059524),('CO',231059974);

CREATE TABLE t2 (
  scan_date date default NULL,
  package_id int default NULL,
  INDEX scan_date(scan_date),
  INDEX package_id(package_id)
);
INSERT INTO t2 VALUES
  ('2008-12-29',231062944),('2008-12-29',231065764),('2008-12-29',231066124),
  ('2008-12-29',231060094),('2008-12-29',231061054),('2008-12-29',231065644),
  ('2008-12-29',231064384),('2008-12-29',231064444),('2008-12-29',231073774),
  ('2008-12-29',231058594),('2008-12-29',231059374),('2008-12-29',231066004),
  ('2008-12-29',231068494),('2008-12-29',231070174),('2008-12-29',231071884),
  ('2008-12-29',231063274),('2008-12-29',231063754),('2008-12-29',231064144),
  ('2008-12-29',231069424),('2008-12-29',231073714),('2008-12-29',231058414),
  ('2008-12-29',231060994),('2008-12-29',231069154),('2008-12-29',231068614),
  ('2008-12-29',231071464),('2008-12-29',231074014),('2008-12-29',231059614),
  ('2008-12-29',231059074),('2008-12-29',231059464),('2008-12-29',231069094),
  ('2008-12-29',231067294),('2008-12-29',231070144),('2008-12-29',231073804),
  ('2008-12-29',231072634),('2008-12-29',231058294),('2008-12-29',231065344),
  ('2008-12-29',231066094),('2008-12-29',231069034),('2008-12-29',231058594),
  ('2008-12-29',231059854),('2008-12-29',231059884),('2008-12-29',231059914),
  ('2008-12-29',231063664),('2008-12-29',231063814),('2008-12-29',231063904);

CREATE TABLE t3 (
  package_id int default NULL,
  INDEX package_id(package_id)
);
INSERT INTO t3 VALUES
  (231058294),(231058324),(231058354),(231058384),(231058414),(231058444),
  (231058474),(231058504),(231058534),(231058564),(231058594),(231058624),
  (231058684),(231058744),(231058804),(231058864),(231058924),(231058954),
  (231059014),(231059074),(231059104),(231059134),(231059164),(231059194),
  (231059224),(231059254),(231059284),(231059314),(231059344),(231059374),
  (231059404),(231059434),(231059464),(231059494),(231059524),(231059554),
  (231059584),(231059614),(231059644),(231059674),(231059704),(231059734),
  (231059764),(231059794),(231059824),(231059854),(231059884),(231059914),
  (231059944),(231059974),(231060004),(231060034),(231060064),(231060094),
  (231060124),(231060154),(231060184),(231060214),(231060244),(231060274),
  (231060304),(231060334),(231060364),(231060394),(231060424),(231060454),
  (231060484),(231060514),(231060544),(231060574),(231060604),(231060634),
  (231060664),(231060694),(231060724),(231060754),(231060784),(231060814),
  (231060844),(231060874),(231060904),(231060934),(231060964),(231060994),
  (231061024),(231061054),(231061084),(231061144),(231061174),(231061204),
  (231061234),(231061294),(231061354),(231061384),(231061414),(231061474),
  (231061564),(231061594),(231061624),(231061684),(231061714),(231061774),
  (231061804),(231061894),(231061984),(231062074),(231062134),(231062224),
  (231062254),(231062314),(231062374),(231062434),(231062494),(231062554),
  (231062584),(231062614),(231062644),(231062704),(231062734),(231062794),
  (231062854),(231062884),(231062944),(231063004),(231063034),(231063064),
  (231063124),(231063154),(231063184),(231063214),(231063274),(231063334),
  (231063394),(231063424),(231063454),(231063514),(231063574),(231063664);
 
CREATE TABLE t4 (
  carrier char(2) NOT NULL default '' PRIMARY KEY,
  id int(11) default NULL,
  INDEX id(id)
);
INSERT INTO t4 VALUES
  ('99',6),('SK',456),('UA',486),('AI',1081),('OS',1111),('VS',1510);

CREATE TABLE t5 (
  carrier_id int default NULL,
  INDEX carrier_id(carrier_id)
);
INSERT INTO t5 VALUES
  (6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),
  (6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),
  (6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),
  (6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),
  (6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),
  (6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(6),(456),(456),(456),
  (456),(456),(456),(456),(456),(456),(456),(456),(456),(456),(456),(456),
  (456),(486),(1081),(1111),(1111),(1111),(1111),(1510);

SELECT COUNT(*) 
  FROM ((t2 JOIN t1 ON t2.package_id = t1.id) 
        JOIN t3 ON t3.package_id = t1.id)
       LEFT JOIN 
       (t5 JOIN t4 ON t5.carrier_id = t4.id)
       ON t4.carrier = t1.carrier;

DROP TABLE t1,t2,t3,t4,t5;
[11 Feb 2010 10:36] Sveta Smirnova
Thank you for the report.

Verified as described.
[11 Feb 2010 13:57] Guilhem Bichot
goes away with
optimizer_switch=loosescan=off
or
optimizer_switch=semijoin=off
[15 Mar 2010 10:05] Jørgen Løland
Simplified test case:

CREATE TABLE t1 (
  carrier int,
  id int PRIMARY KEY
);
INSERT INTO t1 VALUES (1,11),(1,12),(2,13);

CREATE TABLE t2 (
  scan_date int,
  package_id int
);
INSERT INTO t2 VALUES (2008,21),(2008,22);

CREATE TABLE t3 (
  carrier int PRIMARY KEY,
  id int
);
INSERT INTO t3 VALUES (1,31);

CREATE TABLE t4 (
  carrier_id int,
  INDEX carrier_id(carrier_id)
);
INSERT INTO t4 VALUES (31),(32);

--echo
SELECT *
  FROM (t2 JOIN t1 )
       LEFT JOIN 
       (t3 JOIN t4 ON t3.id = t4.carrier_id)
       ON t3.carrier = t1.carrier;

DROP TABLE t1,t2,t3,t4;
[15 Mar 2010 10:25] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/103207

3822 Jorgen Loland	2010-03-15
      Bug#51084: "Batched key access crashes for SELECT with derived 
                  table and LEFT JOIN"
      
      JOIN::optimize() may decide to use a join cache to join a table,
      but may have to revise this decision later, e.g. if an 
      incompatible access method is chosen. If it is decided that a
      join cache cannot be employed after all, set_join_cache_denial()
      is called to destroy the cache object. However, if linked join
      buffers are in use, a join cache for another table may link to 
      the cache being removed through the next_cache pointer. The 
      crash happened when the destroyed object was tried accessed 
      through this link.
      
      The fix is to remove the next_cache pointer from the previous 
      cache if the cache being removed has a previous cache.
     @ mysql-test/r/join_cache.result
        Add test for BUG#51084
     @ mysql-test/t/join_cache.test
        Add test for BUG#51084
     @ sql/sql_select.cc
        When removing join cache from a table, the next_cache pointer from the previous cache needs to be reset so that it does not point to a destroyed object.
[17 Mar 2010 12:25] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/103574

3829 Jorgen Loland	2010-03-17
      Bug#51084: "Batched key access crashes for SELECT with derived 
                  table and LEFT JOIN"
      
      JOIN::optimize() may decide to use a join cache to join a table,
      but may have to revise this decision later, e.g. if an 
      incompatible access method is chosen. If it is decided that a
      join cache cannot be employed after all, set_join_cache_denial()
      is called to destroy the cache object. However, if linked join
      buffers are in use, a join cache for another table may link to 
      the cache being removed through the next_cache pointer. The 
      crash happened when the destroyed object was tried accessed 
      through this link.
      
      The fix is to remove the next_cache pointer from the previous 
      cache if the cache being removed has a previous cache.
     @ mysql-test/r/join_cache.result
        Add test for BUG#51084
     @ mysql-test/t/join_cache.test
        Add test for BUG#51084
     @ sql/sql_select.cc
        When removing join cache from a table, the next_cache pointer from the previous cache needs to be reset so that it does not point to a destroyed object.
[17 Mar 2010 12:39] Jørgen Løland
Pushed to 6.0-codebase-bugfixing
[24 Mar 2010 8:14] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100324081249-yfwol7qtcek6dh7w) (version source revid:alik@sun.com-20100324081113-kc7x1iytnplww91u) (merge vers: 6.0.14-alpha) (pib:16)
[12 Apr 2010 22:09] Paul DuBois
Noted in 6.0.14 changelog.
[10 May 2010 20:24] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/107893

3163 oystein.grovlen@sun.com	2010-05-10
      Bug#51084: "Batched key access crashes for SELECT with derived 
                  table and LEFT JOIN"
      
      (Backporting of revid:jorgen.loland@sun.com-20100317122515-qrhfyp5xhqy07wr9)
      
      JOIN::optimize() may decide to use a join cache to join a table,
      but may have to revise this decision later, e.g. if an 
      incompatible access method is chosen. If it is decided that a
      join cache cannot be employed after all, set_join_cache_denial()
      is called to destroy the cache object. However, if linked join
      buffers are in use, a join cache for another table may link to 
      the cache being removed through the next_cache pointer. The 
      crash happened when the destroyed object was tried accessed 
      through this link.
      
      The fix is to remove the next_cache pointer from the previous 
      cache if the cache being removed has a previous cache.
     @ mysql-test/r/join_cache.result
        Add test for BUG#51084
     @ mysql-test/t/join_cache.test
        Add test for BUG#51084
     @ sql/sql_select.cc
        When removing join cache from a table, the next_cache pointer from the previous cache needs to be reset so that it does not point to a destroyed object.
[10 May 2010 20:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/107894

3165 oystein.grovlen@sun.com	2010-05-10
      Bug#51084: "Batched key access crashes for SELECT with derived 
                  table and LEFT JOIN"
      
      (Backporting of revid:jorgen.loland@sun.com-20100317122515-qrhfyp5xhqy07wr9)
      
      JOIN::optimize() may decide to use a join cache to join a table,
      but may have to revise this decision later, e.g. if an 
      incompatible access method is chosen. If it is decided that a
      join cache cannot be employed after all, set_join_cache_denial()
      is called to destroy the cache object. However, if linked join
      buffers are in use, a join cache for another table may link to 
      the cache being removed through the next_cache pointer. The 
      crash happened when the destroyed object was tried accessed 
      through this link.
      
      The fix is to remove the next_cache pointer from the previous 
      cache if the cache being removed has a previous cache.
     @ mysql-test/r/join_cache.result
        Add test for BUG#51084
     @ mysql-test/t/join_cache.test
        Add test for BUG#51084
     @ sql/sql_select.cc
        When removing join cache from a table, the next_cache pointer from the previous cache needs to be reset so that it does not point to a destroyed object.
[16 Aug 2010 6:40] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100816062819-bluwgdq8q4xysmlg) (version source revid:alik@sun.com-20100816062612-enatdwnv809iw3s9) (pib:20)
[13 Nov 2010 16:15] Bugs System
Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (merge vers: 5.6.99-m4) (pib:21)
[23 Nov 2010 3:16] Paul DuBois
Bug does not appear in any released 5.6.x version. No 5.6.1 changelog entry needed.