Bug #37756 enabling fulltext indexes with myisam_repair_threads > 1 causes crash
Submitted: 1 Jul 2008 6:04 Modified: 13 Feb 2009 20:13
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: MyISAM storage engine Severity:S1 (Critical)
Version:5.1, 6.0 OS:Any
Assigned to: Ramil Kalimullin CPU Architecture:Any
Tags: myisam_repair_threads

[1 Jul 2008 6:04] Shane Bester
Description:
Having multiple indexes on a table and myisam_repair_threads>1 then a separate thread will be used for each index.  This alter table .. enable keys crashes for fulltext indexes.

Testcase outputs this:

<cut>
Query OK, 371293 rows affected (1.16 sec)
Records: 371293  Duplicates: 0  Warnings: 0

mysql> alter table t1 enable keys;
ERROR 2013 (HY000): Lost connection to MySQL server during query

backtrace:
mysqld.exe!ft_add_word
mysqld.exe!ft_parse_internal
mysqld.exe!ft_default_parser_parse
mysqld.exe!ft_parse
mysqld.exe!_mi_ft_parse
mysqld.exe!_mi_ft_parserecord
mysqld.exe!sort_ft_key_read
mysqld.exe!thr_find_all_keys
mysqld.exe!pthread_start
mysqld.exe!_callthreadstart
mysqld.exe!_threadstart

How to repeat:
run server with --myisam_repair_threads=2 then run this sql.
use 64-bit debug version if release or 32-bit doesn't didn't crash at will.
if nothing crashes, run under valgrind.

drop view if exists `v1`;
create view `v1` as select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all
select 1 as `1` union all select 1 as `1`;
drop table if exists t1;
create table t1(a longtext,
fulltext(a),
fulltext(a),
fulltext(a),
fulltext(a),
fulltext(a))engine=myisam;
alter table t1 disable keys;
set @`v_a` := -2147483648;

replace into t1
select
(@`v_a` := (@`v_a` + 4) mod 4294967296) as `a`
from
`v1` `v0`,
`v1` `v1`,
`v1` `v2`,
`v1` `v3`,
`v1` `v4`;
alter table t1 enable keys;#crash
[1 Jul 2008 6:19] MySQL Verification Team
the crash is here:

TREE_ELEMENT *tree_insert(TREE *tree, void *key, uint key_size, 
             void* custom_arg)

parent= tree->parents;
*parent = &tree->root; element= tree->root; <---crash

Unhandled exception at 0x0086a94a in mysqld.exe: 0xC0000005: Access violation writing location 0x00829bdf.  So it looks like some wild memory pointer.
[1 Jul 2008 7:20] MySQL Verification Team
valgrind errors on 5.1.26
-------------------------
Thread 14:
Use of uninitialised value of size 4
   at 0x8446ECB: ft_add_word (ft_parser.c:279)
   by 0x8447252: ft_parse_internal (ft_parser.c:299)
   by 0x8447BE7: ft_default_parser_parse (ft_static.c:630)
   by 0x8446E0B: ft_parse (ft_parser.c:323)
   by 0x847CC2B: _mi_ft_parse (ft_update.c:111)
   by 0x847CD79: _mi_ft_parserecord (ft_update.c:127)
   by 0x8472D9B: sort_ft_key_read (mi_check.c:3167)
   by 0x847ED85: thr_find_all_keys (sort.c:414)
   by 0x4893DA: start_thread (in /lib/libpthread-2.5.so)
   by 0x3D606D: clone (in /lib/libc-2.5.so)

Use of uninitialised value of size 4
   at 0x84A20A2: tree_insert (tree.c:206)
   by 0x8446EEB: ft_add_word (ft_parser.c:279)
   by 0x8447252: ft_parse_internal (ft_parser.c:299)
   by 0x8447BE7: ft_default_parser_parse (ft_static.c:630)
   by 0x8446E0B: ft_parse (ft_parser.c:323)
   by 0x847CC2B: _mi_ft_parse (ft_update.c:111)
   by 0x847CD79: _mi_ft_parserecord (ft_update.c:127)
   by 0x8472D9B: sort_ft_key_read (mi_check.c:3167)
   by 0x847ED85: thr_find_all_keys (sort.c:414)
   by 0x4893DA: start_thread (in /lib/libpthread-2.5.so)
   by 0x3D606D: clone (in /lib/libc-2.5.so)
[1 Jul 2008 9:54] Susanne Ebrecht
Verified as described by using 5.1 bzr-tree.

Here my test:

set myisam_repair_threads=2;

drop view if exists `v1`;

create view `v1` as select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all 
select 1 as `1` union all select 1 as `1` union all
select 1 as `1` union all select 1 as `1`;

drop table if exists t1;

create table t1(a longtext,
fulltext(a),
fulltext(a),
fulltext(a),
fulltext(a),
fulltext(a))engine=myisam;

alter table t1 disable keys;
set @`v_a` := -2147483648;

replace into t1
select
(@`v_a` := (@`v_a` + 4) mod 4294967296) as `a`
from
`v1` `v0`,
`v1` `v1`,
`v1` `v2`,
`v1` `v3`,
`v1` `v4`;
Query OK, 371293 rows affected (1.93 sec)
Records: 371293  Duplicates: 0  Warnings: 0

alter table t1 enable keys;
ERROR 2013 (HY000): Lost connection to MySQL server during query
[1 Jul 2008 9:55] Susanne Ebrecht
Tested on 64bit Linux.
[1 Jul 2008 9:57] Susanne Ebrecht
It also crashes by using MySQL 6.0 bzr tree.
[1 Jul 2008 10:10] MySQL Verification Team
even old version 5.1.15-beta-debug crashed here for me.
[1 Jul 2008 10:22] Susanne Ebrecht
12:09 < sbester> susanne: wait. 5.1.15-beta-debug crashes !
12:09 < sbester> so, it is not a regression
[28 Jul 2008 6:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50578

2680 Ramil Kalimullin	2008-07-28
      Fix for bug#37756: enabling fulltext indexes with myisam_repair_threads > 1 causes crash
      
      Problem: fulltext doesn't support parallel idexes repair.
      
      Fix: support it.
[31 Jul 2008 4:51] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50756

2681 Ramil Kalimullin	2008-07-31
      Fix for bug #37756: enabling fulltext indexes with 
        myisam_repair_threads > 1 causes crash
      
      Problem: parallel repair (myisam_repair_threads > 1) of a myisam
      table with two or more fulltext keys that use the same parser may
      lead to a server crash. ALTER TABLE ENABLE KEYS is affected as well.
      
      Fix: properly initialize fulltext structures for parallel repair.
      
      Note: 1. there's no deterministic test case.
      2. now we call parser->init() for each each fulltext key
      (not for each fulltext parser used).
[23 Sep 2008 11:37] MySQL Verification Team
bug #39599 was marked as a duplicate of this
[5 Dec 2008 9:44] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/60701

2727 Ramil Kalimullin	2008-12-05
      Fix for bug #37756: enabling fulltext indexes with
        myisam_repair_threads > 1 causes crash
      
      Problem: parallel repair (myisam_repair_threads > 1) of a myisam
      table with two or more fulltext keys that use the same parser may
      lead to a server crash. ALTER TABLE ENABLE KEYS is affected as well.
      
      Fix: properly initialize fulltext structures for parallel repair.
      
      Note: 1. there's no deterministic test case.
      2. now we call parser->init() for each fulltext key
      (not for each fulltext parser used).
[26 Jan 2009 6:35] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/64009

2747 Ramil Kalimullin	2009-01-26
      Fix for bug #37756: enabling fulltext indexes with
        myisam_repair_threads > 1 causes crash
      
      Problem: parallel repair (myisam_repair_threads > 1) of a myisam
      table with two or more fulltext keys that use the same parser may
      lead to a server crash. ALTER TABLE ENABLE KEYS is affected as well.
      
      Fix: properly initialize fulltext structures for parallel repair.
      
      Note: 1. there's no deterministic test case.
      2. now we call parser->init() for each fulltext key
      (not for each fulltext parser used).
[3 Feb 2009 9:40] Bugs System
Pushed into 5.1.32 (revid:joro@sun.com-20090203090549-gos3v4320vimrzg6) (version source revid:ramil@mysql.com-20090126063515-ode8uwydi4oap8gw) (merge vers: 5.1.32) (pib:6)
[4 Feb 2009 11:15] Bugs System
Pushed into 6.0.10-alpha (revid:kostja@sun.com-20090204104420-mw1i2u9lum4bxjo6) (version source revid:ramil@mysql.com-20090126073817-eag34ejn5jkiuoy4) (merge vers: 6.0.10-alpha) (pib:6)
[13 Feb 2009 20:13] Paul DuBois
Noted in 5.1.32, 6.0.10 changelogs.

Setting myisam_repair_threads greater than 1 caused a server crash
for table repair or alteration operations for MyISAM tables with
multiple FULLTEXT indexes.
[17 Feb 2009 14:53] Bugs System
Pushed into 5.1.32-ndb-6.3.23 (revid:tomas.ulin@sun.com-20090217131017-6u8qz1edkjfiobef) (version source revid:tomas.ulin@sun.com-20090203133556-9rclp06ol19bmzs4) (merge vers: 5.1.32-ndb-6.3.22) (pib:6)
[17 Feb 2009 16:41] Bugs System
Pushed into 5.1.32-ndb-6.4.3 (revid:tomas.ulin@sun.com-20090217134419-5ha6xg4dpedrbmau) (version source revid:tomas.ulin@sun.com-20090203133556-9rclp06ol19bmzs4) (merge vers: 5.1.32-ndb-6.3.22) (pib:6)
[17 Feb 2009 18:17] Bugs System
Pushed into 5.1.32-ndb-6.2.17 (revid:tomas.ulin@sun.com-20090217134216-5699eq74ws4oxa0j) (version source revid:tomas.ulin@sun.com-20090202111723-1zzwax187rtls913) (merge vers: 5.1.32-ndb-6.2.17) (pib:6)