Bug #35993 severe memory corruption and crash with multibyte conversion
Submitted: 11 Apr 2008 14:26 Modified: 13 May 2008 22:04
Reporter: Shane Bester
Status: Closed
Category:Server: DML Severity:S1 (Critical)
Version:4.1.22, 5.1.24, 5.0.58 OS:Any
Assigned to: Gleb Shchepa Target Version:5.0+
Tags: convert, gbk, BIG5
Triage: D1 (Critical)

[11 Apr 2008 14:26] Shane Bester
Description:
this crash took many different forms

1) memory corruption (visible in innodb ibuf crashes)
2) out of memory
3) corrupted tmpfile messages
4) the crash below:

mysqld.exe!my_strnxfrm_gbk
mysqld.exe!Field_blob::sort_string
mysqld.exe!make_sortkey
mysqld.exe!find_all_keys
mysqld.exe!filesort
mysqld.exe!create_sort_index
mysqld.exe!JOIN::exec
mysqld.exe!mysql_select
mysqld.exe!mysql_derived_filling
mysqld.exe!mysql_handle_derived
mysqld.exe!open_and_lock_tables_derived
mysqld.exe!execute_sqlcom_select
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_callthreadstart
mysqld.exe!_threadstart
kernel32.dll!BaseThreadStart

How to repeat:
run the script a few times if no crash happens.  or increase the repeat value of 200000
to something larger.

drop table if exists t2;
create table t2(a int)engine=innodb;
insert into t2 values ();
drop table if exists t1;
create table t1(a mediumtext COLLATE ucs2_estonian_ci) engine=innodb;
insert into t1 values(repeat(0x1125,200000));
select convert(t1.`a` using gbk)
from t2,t1 group by 1 limit 1 into @nullll;
[11 Apr 2008 14:30] Miguel Solorzano
On XP 32-bit:

 	mysqld.exe!_my_strnncoll_gbk()  + 0x140	C
>	mysqld.exe!Field_blob::sort_string(unsigned char * to=0x0194ce19, unsigned int
length=1024)  Line 7948 + 0xe	C++
 	mysqld.exe!make_sortkey(st_sort_param * param=0x0302ec24, unsigned char *
to=0x0194ce18, unsigned char * ref_pos=0x019050e8)  Line 731	C++
 	mysqld.exe!find_all_keys(st_sort_param * param=0x0302ec24, SQL_SELECT *
select=0x00000000, unsigned char * * sort_keys=0x00000400, st_io_cache *
buffpek_pointers=0x00030d40, st_io_cache * tempfile=0x0302ed4c, st_io_cache *
indexfile=0x00000000)  Line 571 + 0x17	C++
 	mysqld.exe!filesort(THD * thd=0x019151c0, st_table * table=0x03050060, st_sort_field *
sortorder=0x0194b520, unsigned int s_length=1, SQL_SELECT * select=0x00000000, unsigned
__int64 max_rows=1, bool sort_positions=false, unsigned __int64 *
examined_rows=0x0302ee24)  Line 245 + 0x27	C++
 	mysqld.exe!create_sort_index(THD * thd=0x019151c0, JOIN * join=0x00000001, st_order *
order=0x0193e5a0, unsigned __int64 filesort_limit=1, unsigned __int64 select_limit=1,
bool is_order_by=false)  Line 13350 + 0x29	C++
 	mysqld.exe!JOIN::exec()  Line 2127 + 0x57	C++
 	mysqld.exe!mysql_select(THD * thd=0x019151c0, Item * * * rref_pointer_array=0x01916594,
TABLE_LIST * tables=0x0193e148, unsigned int wild_num=0, List<Item> & fields={...}, Item *
conds=0x00000000, unsigned int og_num=1, st_order * order=0x00000000, st_order *
group=0x0193e5a0, Item * having=0x00000000, st_order * proc_param=0x00000000, unsigned
__int64 select_options=2147764736, select_result * result=0x0193e630, st_select_lex_unit
* unit=0x01916220, st_select_lex * select_lex=0x01916498)  Line 2358	C++
 	mysqld.exe!handle_select(THD * thd=0x019151c0, st_lex * lex=0x019161c0, select_result *
result=0x0193e630, unsigned long setup_tables_done_option=0)  Line 269 + 0x79	C++
 	mysqld.exe!execute_sqlcom_select(THD * thd=0x01951000, TABLE_LIST *
all_tables=0x00000400)  Line 4752 + 0xa	C++
 	mysqld.exe!mysql_execute_command(THD * thd=)  Line 3180	C++
 	mysqld.exe!mysql_parse(THD * thd=0x019151c0, const char * inBuf=0x0193dea8, unsigned
int length=75, const char * * found_semicolon=0x0302f9e4)  Line 5634	C++
 	mysqld.exe!dispatch_command(enum_server_command command=COM_QUERY, THD *
thd=0x019151c0, char * packet=0x01935e79, unsigned int packet_length=75)  Line 1123	C++
 	mysqld.exe!do_command(THD * thd=0x00000003)  Line 781 + 0xf	C++
 	mysqld.exe!handle_one_connection(void * arg=0x019151c0)  Line 1115 + 0x6	C++
 	mysqld.exe!_pthread_start()  + 0x3b	C
 	mysqld.exe!_threadstart(void * ptd=0x0192e940)  Line 196 + 0x6	C
 	kernel32.dll!7c80b683()
[11 Apr 2008 14:32] Miguel Solorzano
Thank you for the bug report.
[11 Apr 2008 17:47] Konstantin Osipov
Setting the right lead.
[22 Apr 2008 21:33] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45836

ChangeSet@1.2610, 2008-04-23 00:30:26+05:00, gshchepa@host.loc +4 -0
  Fixed bug #35993: memory corruption and crash with multibyte conversion.
  
  Grouping of long BLOB/TEXT values combined with the call to CONVERT
  function casting to GBK or BIG5 charsets crashes the server:
  
    SELECT CONVERT(blob_column USING big5) FROM t GROUP BY 1.
  
  
  MySQL server uses sorting (the filesort procedure) in the temporary
  table to evaluate the GROUP BY clause in case of lack of suitable index.
  That procedure takes into account only first @max_sort_length bytes
  (system variable, usually 1024) of TEXT/BLOB sorting key string.
  The my_strnxfrm_gbk and my_strnxfrm_big5 fill temporary keys
  with data of whole blob length instead of @max_sort_length bytes
  length. That buffer overrun has been fixed.
[22 Apr 2008 23:16] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45849

ChangeSet@1.2610, 2008-04-23 02:14:58+05:00, gshchepa@host.loc +4 -0
  Fixed bug #35993: memory corruption and crash with multibyte conversion.
  
  Grouping or ordering of long values in not indexed BLOB/TEXT columns
  with GBK or BIG5 charsets crashes the server.
  
  MySQL server uses sorting (the filesort procedure) in the temporary
  table to evaluate the GROUP BY clause in case of lack of suitable index.
  That procedure takes into account only first @max_sort_length bytes
  (system variable, usually 1024) of TEXT/BLOB sorting key string.
  The my_strnxfrm_gbk and my_strnxfrm_big5 fill temporary keys
  with data of whole blob length instead of @max_sort_length bytes
  length. That buffer overrun has been fixed.
[1 May 2008 8:16] Bugs System
Pushed into 5.1.25-rc
[1 May 2008 8:19] Bugs System
Pushed into 6.0.6-alpha
[6 May 2008 2:25] Bugs System
Pushed into 5.0.62
[13 May 2008 22:04] Paul DuBois
Noted in 5.0.62, 5.1.25, 6.0.6 changelogs.

Grouping or ordering of long values in unindexed BLOB or TEXT columns
with the gbk or big5 character set crashed the server.