Bug #51866 crash with repair by sort and fulltext keys
Submitted: 9 Mar 2010 12:02 Modified: 18 Jun 2010 2:09
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: MyISAM storage engine Severity:S1 (Critical)
Version:5.1.44,5.5.2-m2 OS:Any
Assigned to: Sergey Vojtovich CPU Architecture:Any
Tags: myisam_sort_buffer_size, regression

[9 Mar 2010 12:02] Shane Bester
Description:
Version: '5.1.44-enterprise-gpl-advanced-debug'  socket: ''  port: 3306  MySQL Enterprise Server - Advanced Edition Debug
Assertion failed: maxcount > 0, file .\sort.c, line 917

mysqld-debug.exe!my_sigabrt_handler()[mysqld.cc:2047]
mysqld-debug.exe!raise()[winsig.c:597]
mysqld-debug.exe!abort()[abort.c:78]
mysqld-debug.exe!_wassert()[assert.c:212]
mysqld-debug.exe!merge_buffers()[sort.c:917]
mysqld-debug.exe!merge_index()[sort.c:1049]
mysqld-debug.exe!_create_index_by_sort()[sort.c:215]
mysqld-debug.exe!mi_repair_by_sort()[mi_check.c:2434]
mysqld-debug.exe!ha_myisam::repair()[ha_myisam.cc:1148]
mysqld-debug.exe!ha_myisam::repair()[ha_myisam.cc:1031]
mysqld-debug.exe!handler::ha_repair()[handler.cc:3154]
mysqld-debug.exe!mysql_admin_table()[sql_table.cc:4780]
mysqld-debug.exe!mysql_repair_table()[sql_table.cc:5046]
mysqld-debug.exe!mysql_execute_command()[sql_parse.cc:2983]
mysqld-debug.exe!mysql_parse()[sql_parse.cc:5975]
mysqld-debug.exe!dispatch_command()[sql_parse.cc:1235]
mysqld-debug.exe!do_command()[sql_parse.cc:874]
mysqld-debug.exe!handle_one_connection()[sql_connect.cc:1127]
mysqld-debug.exe!pthread_start()[my_winthread.c:85]
mysqld-debug.exe!_callthreadstart()[thread.c:295]
mysqld-debug.exe!_threadstart()[thread.c:277]
kernel32.dll!BaseThreadStart()

affects both release and debug binaries.

How to repeat:
drop table if exists t5;
create table `t5` (`col2` tinytext not null, fulltext (`col2`)) engine=myisam;
insert into t5 values ('aaaaaa');
set session myisam_sort_buffer_size=1;
repair table t5;
[9 Mar 2010 12:06] MySQL Verification Team
more details

Attachment: bug51866_5.1.44_valgrind_output.txt (text/plain), 21.17 KiB.

[9 Mar 2010 12:20] MySQL Verification Team
Thank you for the bug report. Verified as described.
[17 Mar 2010 9:42] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/103548

3404 Sergey Vojtovich	2010-03-17
      BUG#51866 - crash with repair by sort and fulltext keys
      
      Repairing MyISAM table with fulltext indexes and low
      myisam_sort_buffer_size may crash the server.
      
      Estimation of number of index entries was done incorrectly,
      causing further assertion failure or server crash.
     @ mysql-test/r/fulltext.result
        A test case for BUG#51866.
     @ mysql-test/t/fulltext.test
        A test case for BUG#51866.
     @ storage/myisam/mi_check.c
        When estimating number of index entries for external
        fulltext parser, take into account that key_length may
        be bigger than myisam_sort_buffer_size. Reuse logic
        from _create_index_by_sort(): force MIN_SORT_BUFFER to
        be min value for myisam_sort_buffer_size.
        
        Another problem is that ftkey_nr has no other meaning
        than serial number of fulltext index starting with 1.
        We can't say if this key using built-in or external
        parser basing on it's value. In other words we always
        entered if-branch for external parser. At this point,
        the only way to check if we use default parser is to
        compare keyinfo::parser with &ft_default_parser.
     @ storage/myisam/sort.c
        Get rid of MIN_SORT_MEMORY, use MIN_SORT_BUFFER instead
        (defined in myisamdef.h, has the same value and purpose).
[20 Mar 2010 17:35] MySQL Verification Team
Fix for this bug is very trivial and in my humble opinion current minimal setting is not good.

This is a fix:

=== modified file 'sql/mysqld.cc'
--- sql/mysqld.cc       2010-02-05 12:55:20 +0000
+++ sql/mysqld.cc       2010-03-20 17:31:14 +0000
@@ -6843,7 +6843,7 @@
    "The buffer that is allocated when sorting the index when doing a REPAIR or when creating indexes with CREATE INDEX or ALTER TABLE.",
    (uchar**) &global_system_variables.myisam_sort_buff_size,
    (uchar**) &max_system_variables.myisam_sort_buff_size, 0,
-   GET_ULONG, REQUIRED_ARG, 8192*1024, 4, ~0L, 0, 1, 0},
+   GET_ULONG, REQUIRED_ARG, 8192*1024, 4096, ~0L, 0, 1, 0},
   {"myisam_use_mmap", OPT_MYISAM_USE_MMAP,
    "Use memory mapping for reading and writing MyISAM tables.",
    (uchar**) &opt_myisam_use_mmap,
[24 Mar 2010 16:18] Ingo Strüwing
Approved with a comment. Please see email.
[25 Mar 2010 12:09] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/104309

3404 Sergey Vojtovich	2010-03-25
      BUG#51866 - crash with repair by sort and fulltext keys
      
      Repairing MyISAM table with fulltext indexes and low
      myisam_sort_buffer_size may crash the server.
      
      Estimation of number of index entries was done incorrectly,
      causing further assertion failure or server crash.
      
      Docs note: min value for myisam_sort_buffer_size has been
      changed from 4 to 4096.
     @ mysql-test/r/fulltext.result
        A test case for BUG#51866.
     @ mysql-test/r/myisam.result
        Min value for myisam_sort_buffer_size is 4096.
     @ mysql-test/r/variables.result
        Min value for myisam_sort_buffer_size is 4096.
     @ mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_32.result
        Min value for myisam_sort_buffer_size is 4096.
     @ mysql-test/t/fulltext.test
        A test case for BUG#51866.
     @ sql/mysqld.cc
        Min value for myisam_sort_buffer_size is 4096.
     @ storage/myisam/mi_check.c
        When estimating number of index entries for external
        fulltext parser, take into account that key_length may
        be bigger than myisam_sort_buffer_size. Reuse logic
        from _create_index_by_sort(): force MIN_SORT_BUFFER to
        be min value for myisam_sort_buffer_size.
        
        Another problem is that ftkey_nr has no other meaning
        than serial number of fulltext index starting with 1.
        We can't say if this key using built-in or external
        parser basing on it's value. In other words we always
        entered if-branch for external parser. At this point,
        the only way to check if we use default parser is to
        compare keyinfo::parser with &ft_default_parser.
     @ storage/myisam/sort.c
        Get rid of MIN_SORT_MEMORY, use MIN_SORT_BUFFER instead
        (defined in myisamdef.h, has the same value and purpose).
[25 Mar 2010 16:35] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/104364

3424 Ramil Kalimullin	2010-03-25
      An addition to fix for BUG#51866 - crash with repair 
      by sort and fulltext keys.
      
      Min value for myisam_sort_buffer_size is 4096.
     @ mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_64.result
        An addition to fix for BUG#51866 - crash with repair 
        by sort and fulltext keys.
        
        Min value for myisam_sort_buffer_size is 4096.
[6 Apr 2010 7:58] Bugs System
Pushed into 5.1.46 (revid:sergey.glukhov@sun.com-20100405111026-7kz1p8qlzglqgfmu) (version source revid:ramil@mysql.com-20100325163507-arlto2w20tfqoo6y) (merge vers: 5.1.46) (pib:16)
[15 Apr 2010 16:18] Paul Dubois
Noted in 5.1.46 changelog.

If myisam_sort_buffer_size was set to a small value, table repair for
MyISAM tables with FULLTEXT indexes could crash the server.

Setting report to Need Merge pending push to Celosia.
[28 May 2010 5:59] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100524190136-egaq7e8zgkwb9aqi) (version source revid:alik@sun.com-20100422150750-vp0n37kp9ywq5ghf) (pib:16)
[28 May 2010 6:28] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100524190941-nuudpx60if25wsvx) (version source revid:alik@sun.com-20100422150658-fkhgnwwkyugtxrmu) (merge vers: 6.0.14-alpha) (pib:16)
[28 May 2010 6:56] Bugs System
Pushed into 5.5.5-m3 (revid:alik@sun.com-20100524185725-c8k5q7v60i5nix3t) (version source revid:alexey.kopytov@sun.com-20100402151743-xowc2u930h729jsy) (merge vers: 5.5.4-m3) (pib:16)
[30 May 2010 0:23] Paul Dubois
Noted in 5.5.5, 6.0.14 changelogs.
[17 Jun 2010 12:02] Bugs System
Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[17 Jun 2010 12:44] Bugs System
Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100609211156-tsac5qhw951miwtt) (merge vers: 5.1.46-ndb-6.2.19) (pib:16)
[17 Jun 2010 13:29] Bugs System
Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)