Bug #31159 fulltext search on ucs2 column crashes server
Submitted: 23 Sep 2007 17:01 Modified: 19 Nov 2007 4:06
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: FULLTEXT search Severity:S1 (Critical)
Version:5.0.48, 5.0.50-debug OS:Any
Assigned to: Sergey Vojtovich CPU Architecture:Any
Tags: crash, fulltext, ucs2

[23 Sep 2007 17:01] Shane Bester
Description:
Linux '5.0.48-enterprise-gpl-debug

0x81901c5 handle_segfault + 427
0x83b44c1 ft_get_word + 75
0x83b5dba _ftb_parse_query + 699
0x83b63cf ft_init_boolean_search + 388
0x83b4d0a ft_init_search + 42
0x82504c7 ha_myisam::ft_init_ext(unsigned int, unsigned int, String*) + 43
0x813a407 Item_func_match::init_search(bool) + 805
0x81d005b init_ftfuncs(THD*, st_select_lex*, bool) + 115
0x81d8468 JOIN::optimize() + 4722
0x81daf3f _Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select + 417
0x81d6537 handle_select(THD*, st_lex*, select_result*, unsigned long) + 251
0x81a687f mysql_execute_command(THD*) + 1721
0x81acfe2 mysql_parse(THD*, char const*, unsigned int, char const**) + 334
0x81a5075 dispatch_command(enum_server_command, THD*, char*, unsigned int) + 1663
0x81a49ea do_command(THD*) + 442
0x81a3ca4 handle_one_connection + 812
0x40045aa7 _end + 933199639
0x40176c2e _end + 934449310

How to repeat:
repeated on 5.0.48-enterprise on XP and on linux. 5.1.23BK on linux didn't crash.

drop table if exists t1;
create table `t1` (`col000` char (19) charset ucs2 collate ucs2_esperanto_ci not null ,key(`col000` ),`col003` float (21,19) not null ,key(`col003` ))engine=myisam;
insert into t1 values (),(),();
select * from t1 where match(`col000`) against ((select `col003` from t1 limit 1 ) in boolean mode);
[23 Sep 2007 17:01] Shane Bester
very recent 5.0BK should be tested. Hence, I left 'open' status.
[23 Sep 2007 17:10] Valeriy Kravchuk
Verified just as described with latest 5.0.50-BK on Linux:

openxs@linux:~/dbs/5.0> bin/mysql -uroot test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.0.50-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> drop table if exists t1;
create table `t1` (`col000` char (19) charset ucs2 collate ucs2_esperanto_ci not null
,key(`col000` ),`col003` float (21,19) not null ,key(`col003` ))engine=myisam;
insert into t1 values (),(),();
select * from t1 where match(`col000`) against ((select `col003` from t1 limit 1 ) in
boolean mode);Query OK, 0 rows affected (0.01 sec)

mysql> create table `t1` (`col000` char (19) charset ucs2 collate ucs2_esperanto_ci not null
    -> ,key(`col000` ),`col003` float (21,19) not null ,key(`col003` ))engine=myisam;
Query OK, 0 rows affected (0.07 sec)

mysql> insert into t1 values (),(),();
Query OK, 3 rows affected, 2 warnings (0.04 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> select * from t1 where match(`col000`) against ((select `col003` from t1 limit 1 ) in
    -> boolean mode);
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>
Number of processes running now: 0
070917 14:02:11  mysqld restarted

Resolved stack trace is:

openxs@linux:~/dbs/5.0> bin/resolve_stack_dump -s /tmp/mysqld5.sym -n 31159.stack
0x81a31b0 handle_segfault + 428
0x8458287 ft_get_word + 81
0x8459c12 _ftb_parse_query + 695
0x845a278 ft_init_boolean_search + 392
0x8458b0f ft_init_search + 39
0x8267d6b _ZN9ha_myisam11ft_init_extEjjP6String + 43
0x8148260 _ZN15Item_func_match11init_searchEb + 724
0x81e7f50 _Z12init_ftfuncsP3THDP13st_select_lexb + 120
0x81f04ca _ZN4JOIN8optimizeEv + 4774
0x81f3023 _Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select + 411
0x81ee4b7 _Z13handle_selectP3THDP6st_lexP13select_resultm + 251
0x81b8431 _Z21mysql_execute_commandP3THD + 1749
0x81bf676 _Z11mysql_parseP3THDPKcjPS2_ + 334
0x81b6ba7 _Z16dispatch_command19enum_server_commandP3THDPcj + 1687
0x81b6503 _Z10do_commandP3THD + 453
0x81b578b handle_one_connection + 799
0x40050aa7 _end + 932711367
0x40247c2e _end + 934772046
[23 Sep 2007 17:33] Shane Bester
at least the following ucs2 collations caused a crash:
ucs2_unicode_ci     
ucs2_icelandic_ci   
ucs2_latvian_ci     
ucs2_romanian_ci    
ucs2_slovenian_ci   
ucs2_polish_ci      
ucs2_estonian_ci    
ucs2_spanish_ci     
ucs2_swedish_ci     
ucs2_turkish_ci     
ucs2_czech_ci       
ucs2_danish_ci      
ucs2_lithuanian_ci  
ucs2_slovak_ci      
ucs2_spanish2_ci    
ucs2_roman_ci       
ucs2_persian_ci     
ucs2_esperanto_ci   
ucs2_hungarian_ci
[18 Oct 2007 12:20] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/35836

ChangeSet@1.2686, 2007-10-18 17:09:55+05:00, svoj@mysql.com +5 -0
  BUG#31159 - fulltext search on ucs2 column crashes server
  
  ucs2 doesn't provide required by fulltext ctype array. Crash
  happens because fulltext attempts to use unitialized ctype
  array.
  
  Fixed by converting ucs2 fields to compatible utf8 analogue.
[19 Oct 2007 10:27] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/35900

ChangeSet@1.2686, 2007-10-19 14:24:59+05:00, svoj@mysql.com +5 -0
  BUG#31159 - fulltext search on ucs2 column crashes server
  
  ucs2 doesn't provide required by fulltext ctype array. Crash
  happens because fulltext attempts to use unitialized ctype
  array.
  
  Fixed by converting ucs2 fields to compatible utf8 analogue.
[24 Oct 2007 12:06] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/36252

ChangeSet@1.2686, 2007-10-24 16:09:30+05:00, svoj@mysql.com +5 -0
  BUG#31159 - fulltext search on ucs2 column crashes server
  
  ucs2 doesn't provide required by fulltext ctype array. Crash
  happens because fulltext attempts to use unitialized ctype
  array.
  
  Fixed by converting ucs2 fields to compatible utf8 analogue.
[25 Oct 2007 12:12] Sergey Vojtovich
Fixed in 4.1 and 5.0.
5.1 and up are unaffected.
[14 Nov 2007 9:40] Bugs System
Pushed into 6.0.4-alpha
[14 Nov 2007 9:44] Bugs System
Pushed into 5.1.23-rc
[14 Nov 2007 9:49] Bugs System
Pushed into 5.0.52
[14 Nov 2007 9:55] Bugs System
Pushed into 4.1.24
[19 Nov 2007 4:06] Paul Dubois
Noted in 4.1.24, 5.0.52 changelogs.

Full-text searches on ucs2 columns caused a server crash. (FULLTEXT
indexes on ucs2 columns cannot be used, but it should be possible to
perform IN BOOLEAN MODE searches on ucs2 columns without a crash.)