Bug #35993 severe memory corruption and crash with multibyte conversion
Submitted: 11 Apr 2008 12:26 Modified: 13 May 2008 20:04
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: DML Severity:S1 (Critical)
Version:4.1.22, 5.1.24, 5.0.58 OS:Any
Assigned to: Gleb Shchepa CPU Architecture:Any
Tags: BIG5, convert, gbk

[11 Apr 2008 12:26] Shane Bester
Description:
this crash took many different forms

1) memory corruption (visible in innodb ibuf crashes)
2) out of memory
3) corrupted tmpfile messages
4) the crash below:

mysqld.exe!my_strnxfrm_gbk
mysqld.exe!Field_blob::sort_string
mysqld.exe!make_sortkey
mysqld.exe!find_all_keys
mysqld.exe!filesort
mysqld.exe!create_sort_index
mysqld.exe!JOIN::exec
mysqld.exe!mysql_select
mysqld.exe!mysql_derived_filling
mysqld.exe!mysql_handle_derived
mysqld.exe!open_and_lock_tables_derived
mysqld.exe!execute_sqlcom_select
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_callthreadstart
mysqld.exe!_threadstart
kernel32.dll!BaseThreadStart

How to repeat:
run the script a few times if no crash happens.  or increase the repeat value of 200000 to something larger.

drop table if exists t2;
create table t2(a int)engine=innodb;
insert into t2 values ();
drop table if exists t1;
create table t1(a mediumtext COLLATE ucs2_estonian_ci) engine=innodb;
insert into t1 values(repeat(0x1125,200000));
select convert(t1.`a` using gbk)
from t2,t1 group by 1 limit 1 into @nullll;
[11 Apr 2008 12:30] MySQL Verification Team
On XP 32-bit:

 	mysqld.exe!_my_strnncoll_gbk()  + 0x140	C
>	mysqld.exe!Field_blob::sort_string(unsigned char * to=0x0194ce19, unsigned int length=1024)  Line 7948 + 0xe	C++
 	mysqld.exe!make_sortkey(st_sort_param * param=0x0302ec24, unsigned char * to=0x0194ce18, unsigned char * ref_pos=0x019050e8)  Line 731	C++
 	mysqld.exe!find_all_keys(st_sort_param * param=0x0302ec24, SQL_SELECT * select=0x00000000, unsigned char * * sort_keys=0x00000400, st_io_cache * buffpek_pointers=0x00030d40, st_io_cache * tempfile=0x0302ed4c, st_io_cache * indexfile=0x00000000)  Line 571 + 0x17	C++
 	mysqld.exe!filesort(THD * thd=0x019151c0, st_table * table=0x03050060, st_sort_field * sortorder=0x0194b520, unsigned int s_length=1, SQL_SELECT * select=0x00000000, unsigned __int64 max_rows=1, bool sort_positions=false, unsigned __int64 * examined_rows=0x0302ee24)  Line 245 + 0x27	C++
 	mysqld.exe!create_sort_index(THD * thd=0x019151c0, JOIN * join=0x00000001, st_order * order=0x0193e5a0, unsigned __int64 filesort_limit=1, unsigned __int64 select_limit=1, bool is_order_by=false)  Line 13350 + 0x29	C++
 	mysqld.exe!JOIN::exec()  Line 2127 + 0x57	C++
 	mysqld.exe!mysql_select(THD * thd=0x019151c0, Item * * * rref_pointer_array=0x01916594, TABLE_LIST * tables=0x0193e148, unsigned int wild_num=0, List<Item> & fields={...}, Item * conds=0x00000000, unsigned int og_num=1, st_order * order=0x00000000, st_order * group=0x0193e5a0, Item * having=0x00000000, st_order * proc_param=0x00000000, unsigned __int64 select_options=2147764736, select_result * result=0x0193e630, st_select_lex_unit * unit=0x01916220, st_select_lex * select_lex=0x01916498)  Line 2358	C++
 	mysqld.exe!handle_select(THD * thd=0x019151c0, st_lex * lex=0x019161c0, select_result * result=0x0193e630, unsigned long setup_tables_done_option=0)  Line 269 + 0x79	C++
 	mysqld.exe!execute_sqlcom_select(THD * thd=0x01951000, TABLE_LIST * all_tables=0x00000400)  Line 4752 + 0xa	C++
 	mysqld.exe!mysql_execute_command(THD * thd=)  Line 3180	C++
 	mysqld.exe!mysql_parse(THD * thd=0x019151c0, const char * inBuf=0x0193dea8, unsigned int length=75, const char * * found_semicolon=0x0302f9e4)  Line 5634	C++
 	mysqld.exe!dispatch_command(enum_server_command command=COM_QUERY, THD * thd=0x019151c0, char * packet=0x01935e79, unsigned int packet_length=75)  Line 1123	C++
 	mysqld.exe!do_command(THD * thd=0x00000003)  Line 781 + 0xf	C++
 	mysqld.exe!handle_one_connection(void * arg=0x019151c0)  Line 1115 + 0x6	C++
 	mysqld.exe!_pthread_start()  + 0x3b	C
 	mysqld.exe!_threadstart(void * ptd=0x0192e940)  Line 196 + 0x6	C
 	kernel32.dll!7c80b683()
[11 Apr 2008 12:32] MySQL Verification Team
Thank you for the bug report.
[11 Apr 2008 15:47] Konstantin Osipov
Setting the right lead.
[22 Apr 2008 19:33] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45836

ChangeSet@1.2610, 2008-04-23 00:30:26+05:00, gshchepa@host.loc +4 -0
  Fixed bug #35993: memory corruption and crash with multibyte conversion.
  
  Grouping of long BLOB/TEXT values combined with the call to CONVERT
  function casting to GBK or BIG5 charsets crashes the server:
  
    SELECT CONVERT(blob_column USING big5) FROM t GROUP BY 1.
  
  
  MySQL server uses sorting (the filesort procedure) in the temporary
  table to evaluate the GROUP BY clause in case of lack of suitable index.
  That procedure takes into account only first @max_sort_length bytes
  (system variable, usually 1024) of TEXT/BLOB sorting key string.
  The my_strnxfrm_gbk and my_strnxfrm_big5 fill temporary keys
  with data of whole blob length instead of @max_sort_length bytes
  length. That buffer overrun has been fixed.
[22 Apr 2008 21:16] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45849

ChangeSet@1.2610, 2008-04-23 02:14:58+05:00, gshchepa@host.loc +4 -0
  Fixed bug #35993: memory corruption and crash with multibyte conversion.
  
  Grouping or ordering of long values in not indexed BLOB/TEXT columns
  with GBK or BIG5 charsets crashes the server.
  
  MySQL server uses sorting (the filesort procedure) in the temporary
  table to evaluate the GROUP BY clause in case of lack of suitable index.
  That procedure takes into account only first @max_sort_length bytes
  (system variable, usually 1024) of TEXT/BLOB sorting key string.
  The my_strnxfrm_gbk and my_strnxfrm_big5 fill temporary keys
  with data of whole blob length instead of @max_sort_length bytes
  length. That buffer overrun has been fixed.
[1 May 2008 6:16] Bugs System
Pushed into 5.1.25-rc
[1 May 2008 6:19] Bugs System
Pushed into 6.0.6-alpha
[6 May 2008 0:25] Bugs System
Pushed into 5.0.62
[13 May 2008 20:04] Paul DuBois
Noted in 5.0.62, 5.1.25, 6.0.6 changelogs.

Grouping or ordering of long values in unindexed BLOB or TEXT columns
with the gbk or big5 character set crashed the server.