Bug #40675 MySQL 5.1 crash with index merge algorithm and Merge tables
Submitted: 12 Nov 2008 20:17 Modified: 14 Mar 2009 1:35
Reporter: Didier Neisen Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Merge storage engine Severity:S2 (Serious)
Version:5.1.29, 6.0 OS:Linux
Assigned to: V Venkateswaran CPU Architecture:Any
Tags: regression
Triage: Triaged: D1 (Critical)

[12 Nov 2008 20:17] Didier Neisen
Description:
A query on a MyISAM Merge table could crash the server if the index merge algorithm is used.

MySQL 5.0 (5.0.67) is not affected by this bug.

How to repeat:
CREATE TABLE `t1` (
  `a` int(10) unsigned NOT NULL,
  `b` int(10) unsigned NOT NULL,
  KEY `a` (`a`),
  KEY `b` (`b`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

CREATE TABLE `t2` (
  `a` int(10) unsigned NOT NULL,
  `b` int(10) unsigned NOT NULL,
  KEY `a` (`a`),
  KEY `b` (`b`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

CREATE TABLE `t0` (
  `a` int(10) unsigned NOT NULL,
  `b` int(10) unsigned NOT NULL,
  KEY `a` (`a`),
  KEY `b` (`b`)
) ENGINE=MRG_MyISAM UNION=(`t1`,`t2`);

INSERT INTO t1 VALUES (1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1);
INSERT INTO t1 SELECT * FROM t1;
INSERT INTO t1 SELECT * FROM t1;
...
INSERT INTO t1 SELECT * FROM t1;
Query OK, 20480 rows affected (1.28 sec)
Records: 20480  Duplicates: 0  Warnings: 0

SELECT COUNT(*) FROM t1;
+----------+
| COUNT(*) |
+----------+
|    40960 |
+----------+
1 row in set (0.00 sec)

EXPLAIN SELECT t0.a, t0.b FROM t0 WHERE a=2 and b=2\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t0
         type: index_merge
possible_keys: a,b
          key: a,b
      key_len: 4,4
          ref: NULL
         rows: 1
        Extra: Using intersect(a,b); Using where; Using index
1 row in set (0.62 sec)

mysql> SELECT t0.a, t0.b FROM t0 WHERE a=2 and b=2;
ERROR 2013 (HY000): Lost connection to MySQL server during query

The same query without the index merge algorithm :

EXPLAIN SELECT t0.a, t0.b FROM t0 force index (a) WHERE a=2 and b=2\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t0
         type: ref
possible_keys: a
          key: a
      key_len: 4
          ref: const
         rows: 2
        Extra: Using where
1 row in set (0.00 sec)

mysql> SELECT t0.a, t0.b FROM t0 force index (a) WHERE a=2 and b=2;
Empty set (0.00 sec)
[12 Nov 2008 21:25] Miguel Solorzano
Thank you for the bug report. Repeatable with bzr source server and not repeatable with 6.0 and older Windows release 5.1.22.

miguel@hegel:~/dbs$ 5.1/libexec/mysqld
081112 19:14:12  InnoDB: Started; log sequence number 0 46409
081112 19:14:12 [Note] Event Scheduler: Loaded 0 events
081112 19:14:12 [Note] 5.1/libexec/mysqld: ready for connections.
Version: '5.1.31-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
mysqld: ha_myisammrg.cc:902: virtual int ha_myisammrg::external_lock(THD*, int): Assertion `this->file->children_attached' failed.

<cut>

5.1/libexec/mysqld(my_print_stacktrace+0x32)[0xb09ec2]
5.1/libexec/mysqld(handle_segfault+0x29b)[0x6b083a]
/lib/libpthread.so.0[0x7fd9fb3700f0]
/lib/libc.so.6(gsignal+0x35)[0x7fd9fa52ffd5]
/lib/libc.so.6(abort+0x183)[0x7fd9fa531b43]
/lib/libc.so.6(__assert_fail+0xe9)[0x7fd9fa528d49]
5.1/libexec/mysqld(_ZN12ha_myisammrg13external_lockEP3THDi+0x3f)[0x9c7ad9]
5.1/libexec/mysqld(_ZN7handler16ha_external_lockEP3THDi+0x8a)[0x7f226c]
5.1/libexec/mysqld(_ZN18QUICK_RANGE_SELECT20init_ror_merged_scanEb+0x1e1)[0x7e648f]
5.1/libexec/mysqld(_ZN26QUICK_ROR_INTERSECT_SELECT20init_ror_merged_scanEb+0x139)[0x7cfd6f]
5.1/libexec/mysqld(_ZN26QUICK_ROR_INTERSECT_SELECT5resetEv+0x6b)[0x7cee01]
5.1/libexec/mysqld[0x732a77]
5.1/libexec/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x133)[0x731e7b]
5.1/libexec/mysqld[0x742b1e]
5.1/libexec/mysqld(_ZN4JOIN4execEv+0x243d)[0x754e53]
5.1/libexec/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x339)[0x74fbaa]
5.1/libexec/mysqld(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x1f7)[0x75518f]
5.1/libexec/mysqld[0x6c0ad2]
5.1/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x87e)[0x6c29a4]
5.1/libexec/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x282)[0x6cb1c3]
5.1/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xbee)[0x6cbf45]
5.1/libexec/mysqld(_Z10do_commandP3THD+0x24d)[0x6cd38a]
5.1/libexec/mysqld(handle_one_connection+0x11c)[0x6ba19e]
/lib/libpthread.so.0[0x7fd9fb3683ea]
/lib/libc.so.6(clone+0x6d)[0x7fd9fa5e3c6d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x2bf6848 = SELECT t0.a, t0.b FROM t0 WHERE a=2 and b=2
thd->thread_id=1
thd->killed=NOT_KILLED
[9 Dec 2008 16:53] Miguel Solorzano
Bug: http://bugs.mysql.com/bug.php?id=41333 marked as duplicate of this one.
[6 Feb 2009 12:48] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/65475

2750 V Narayanan	2009-02-06
      Bug #40675  MySQL 5.1 crash with index merge algorithm and Merge tables
      
      A Query in the MyISAM merge table was crashing if the index merge algorithm
      was being used
      
      Index Merge optimization requires the reading of multiple indexes at the
      same time. Reading multiple indexes at once with current SE API means
      that we need to have handler instance for each to-be-read index. This is
      done by creating clones of the handlers instances. The clone internally
      does a open of the handler.
      
      The open for a MERGE engine is handled in the following phases
      
      1) open parent table
      2) generate list of underlying
         table
      3) attach underlying tables
      
      But the current implementation does only the first phase (i.e.)
      open parent table.
      
      The current patch fixes this at the MERGE engine level, by handling the
      clone operation within the MERGE engine rather than in the storage engine
      API.
[10 Feb 2009 9:13] Ingo Strüwing
Approved with suggestions. Possible memory leak. Please see email for details.
[10 Feb 2009 20:35] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/65803

2750 V Narayanan	2009-02-11
      Bug #40675  MySQL 5.1 crash with index merge algorithm and Merge tables
      
      A Query in the MyISAM merge table was crashing 
      if the index merge algorithm was being used
      
      Index Merge optimization requires the reading of 
      multiple indexes at the same time. Reading multiple 
      indexes at once with current SE API means that we 
      need to have handler instance for each to-be-read 
      index. This is done by creating clones of the handlers 
      instances. The clone internally does a open of the handler.
      
      The open for a MERGE engine is handled in the following 
      phases
      
      1) open parent table
      2) generate list of underlying
         table
      3) attach underlying tables
      
      But the current implementation does only the first 
      phase (i.e.) open parent table.
      
      The current patch fixes this at the MERGE engine level, 
      by handling the clone operation within the MERGE engine 
      rather than in the storage engine API. It opens and 
      attaches the MyISAM tables on the MyISAM storage engine 
      interface directly within the MERGE engine. The new MyISAM 
      table instances, as well as the MERGE clone itself, are not 
      visible in the table cache. This is not a problem because 
      all locking is handled by the original MERGE table from which
      this is cloned of.
[11 Feb 2009 12:19] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/65895

2750 V Narayanan	2009-02-11
      Bug#40675  MySQL 5.1 crash with index merge algorithm and Merge tables
            
      A Query in the MyISAM merge table was crashing 
      if the index merge algorithm was being used
      
      Index Merge optimization requires the reading of 
      multiple indexes at the same time. Reading multiple 
      indexes at once with current SE API means that we 
      need to have handler instance for each to-be-read 
      index. This is done by creating clones of the handlers 
      instances. The clone internally does a open of the handler.
      
      The open for a MERGE engine is handled in the following 
      phases
      
      1) open parent table
      2) generate list of underlying
         table
      3) attach underlying tables
      
      But the current implementation does only the first 
      phase (i.e.) open parent table.
      
      The current patch fixes this at the MERGE engine level, 
      by handling the clone operation within the MERGE engine 
      rather than in the storage engine API. It opens and 
      attaches the MyISAM tables on the MyISAM storage engine 
      interface directly within the MERGE engine. The new MyISAM 
      table instances, as well as the MERGE clone itself, are not 
      visible in the table cache. This is not a problem because
      all locking is handled by the original MERGE table from which
      this is cloned of.
[12 Feb 2009 11:12] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/66021

2801 V Narayanan	2009-02-12
      Bug#40675  MySQL 5.1 crash with index merge algorithm and Merge tables
                  
      A Query in the MyISAM merge table was crashing 
      if the index merge algorithm was being used
      
      Index Merge optimization requires the reading of 
      multiple indexes at the same time. Reading multiple 
      indexes at once with current SE API means that we 
      need to have handler instance for each to-be-read 
      index. This is done by creating clones of the handlers 
      instances. The clone internally does a open of the handler.
      
      The open for a MERGE engine is handled in the following 
      phases
      
      1) open parent table
      2) generate list of underlying
         table
      3) attach underlying tables
      
      But the current implementation does only the first 
      phase (i.e.) open parent table.
      
      The current patch fixes this at the MERGE engine level, 
      by handling the clone operation within the MERGE engine 
      rather than in the storage engine API. It opens and 
      attaches the MyISAM tables on the MyISAM storage engine 
      interface directly within the MERGE engine. The new MyISAM 
      table instances, as well as the MERGE clone itself, are not 
      visible in the table cache. This is not a problem because
      all locking is handled by the original MERGE table from which
      this is cloned of.
[19 Feb 2009 13:03] Bugs System
Pushed into 6.0.10-alpha (revid:sergey.glukhov@sun.com-20090218125737-5y5b2xo3duo1wlvo) (version source revid:azundris@mysql.com-20090213182753-tsqikd41gkwowrqq) (merge vers: 6.0.10-alpha) (pib:6)
[4 Mar 2009 15:55] Tony Bedford
An entry was added to the 6.0.10 changelog:

MySQL 5.1 crashed with index merge algorithm and merge tables.

A query in the MyISAM merge table caused a crash if the index merge algorithm was being used.
[13 Mar 2009 16:38] Valeriy Kravchuk
This should be fixed in 5.1.x also!
[13 Mar 2009 19:07] Bugs System
Pushed into 5.1.33 (revid:joro@sun.com-20090313111355-7bsi1hgkvrg8pdds) (version source revid:vvaintroub@mysql.com-20090218093153-sjzxf01i4ezte0xp) (merge vers: 5.1.33) (pib:6)
[14 Mar 2009 1:35] Paul Dubois
Noted in 5.1.33 changelog.
[9 May 2009 16:46] Bugs System
Pushed into 5.1.34-ndb-6.2.18 (revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (version source revid:jonas@mysql.com-20090508100057-30ote4xggi4nq14v) (merge vers: 5.1.33-ndb-6.2.18) (pib:6)
[9 May 2009 17:42] Bugs System
Pushed into 5.1.34-ndb-6.3.25 (revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (version source revid:jonas@mysql.com-20090508175813-s6yele2z3oh6o99z) (merge vers: 5.1.33-ndb-6.3.25) (pib:6)
[9 May 2009 18:40] Bugs System
Pushed into 5.1.34-ndb-7.0.6 (revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (version source revid:jonas@mysql.com-20090509073226-09bljakh9eppogec) (merge vers: 5.1.33-ndb-7.0.6) (pib:6)
[21 Jan 2012 4:10] Shane Bester
bug #64074 is a duplicate of this.