Bug #29464 load data infile into table with big5 chinese fulltext index hangs 100% cpu
Submitted: 1 Jul 2007 1:51 Modified: 23 Jul 2007 20:20
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: FULLTEXT search Severity:S2 (Serious)
Version:5.1.21-BK, 6.0.1-BK OS:Linux (suse 9.3)
Assigned to: Sergey Vojtovich CPU Architecture:Any

[1 Jul 2007 1:51] Shane Bester
Description:
completes in <5 seconds on 5.0.44.

5.1.21 and 6.0.1 seems to hang:

mysql> show processlist;
+----+------+------+------+---------+------+-------------------+------------------------
| Id | User | Host | db   | Command | Time | State             | Info
+----+------+------+------+---------+------+-------------------+------------------------
|  3 | root |      | test | Query   |  471 | Repair by sorting | load data local infile
|  5 | root |      | a    | Query   |    0 | NULL              | show processlist
+----+------+------+------+---------+------+-------------------+------------------------

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                              
 4712 sbester   16   0 69336  34m 5392 S 99.3  1.7  10:50.90 mysqld       

How to repeat:
drop table if exists tbl_0;
create table tbl_0 (a varchar(2) character set big5 collate big5_chinese_ci,b varchar(2) character set big5 collate big5_chinese_ci,fulltext key(a,b),key(b,a),key(b),key(a))engine=myisam;
load data local infile '/tmp/dump.txt' replace into table tbl_0 (@h,@i) set a=UNHEX(@h),b=UNHEX(@h);
check table tbl_0 extended;

Suggested fix:
.
[1 Jul 2007 1:52] MySQL Verification Team
dump.txt used in the testcase.

Attachment: dump.txt (text/plain), 105.25 KiB.

[1 Jul 2007 2:02] MySQL Verification Team
Maybe irrelevant but I caught it in debugger. In ft_simple_get_word() there's "uint mwc, length, mbl;" but somewhere mbl is getting a return value of -2 which turns into 4294967294, hence a >0 check presumably misbehaves.
[5 Jul 2007 11:13] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/30365

ChangeSet@1.2528, 2007-07-05 15:17:06+05:00, svoj@mysql.com +3 -0
  BUG#29464 - load data infile into table with big5 chinese fulltext index
              hangs 100% cpu
  
  Fulltext parser may fall into infinite loop when it gets illegal
  multibyte sequence (or a sequence that doesn't have mapping to unicode).
  
  Affects 5.1 only.
[12 Jul 2007 18:38] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/30828

ChangeSet@1.2545, 2007-07-13 03:29:25+05:00, svoj@mysql.com +4 -0
  BUG#29464 - load data infile into table with big5 chinese fulltext index
              hangs 100% cpu
  
  Moved a test case for BUG#29464 into fulltext3.test, since it requires big5
  character set.
[17 Jul 2007 15:31] Bugs System
Pushed into 5.1.21-beta
[23 Jul 2007 20:20] Paul Dubois
Noted in 5.1.21 changelog.

The full-text parser could enter an infinite loop if it encountered
an illegal multi-byte sequence or a sequence that has no mapping to
Unicode.
[30 Jun 2008 18:05] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:09] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:12] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:15] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:18] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:20] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:22] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:24] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:26] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:27] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:28] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:30] Bugs System
Pushed into 5.1.25-maria-alpha
[30 Jun 2008 18:31] Bugs System
Pushed into 5.1.25-maria-alpha