Bug #36023 crash/huge memory alloc with max and case when if statement
Submitted: 13 Apr 2008 10:27 Modified: 13 May 2008 20:15
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: DML Severity:S1 (Critical)
Version:5.0.58, 5.1.24 OS:Any
Assigned to: Evgeny Potemkin CPU Architecture:Any

[13 Apr 2008 10:27] Shane Bester
Description:
stack trace on 32-bit binary:

mysqld-nt.exe!hp_close
mysqld-nt.exe!heap_close
mysqld-nt.exe!ha_heap::close
mysqld-nt.exe!free_tmp_table
mysqld-nt.exe!create_tmp_table
mysqld-nt.exe!JOIN::optimize
mysqld-nt.exe!mysql_select
mysqld-nt.exe!handle_select
mysqld-nt.exe!mysql_execute_command
mysqld-nt.exe!mysql_parse
mysqld-nt.exe!dispatch_command
mysqld-nt.exe!do_command
mysqld-nt.exe!handle_one_connection
mysqld-nt.exe!pthread_start
mysqld-nt.exe!_threadstart

64-bit binary goes and allocates 4GB of ram for the query below, so watch out.

How to repeat:
drop table if exists t1;
create table t1(`c` decimal(9,2));
insert into t1 values (300),(120),(201.11),(220);
select max(case rand() when 'a' then b.c else null end) from t1 a,t1 b group by b.c;
[13 Apr 2008 10:52] MySQL Verification Team
there are two bugs here:

1) this query shouldn't require 4GB of ram.
2) failure to gracefully handle the OOM situation.
[13 Apr 2008 15:29] Valeriy Kravchuk
Thank you for a bug report. On 32-bit 5.1.24 I've got a crash with the following stack trace:

 	mysqld.exe!ha_heap::drop_table(const char * name=0x02319f98)  Line 559 + 0x6 bytes	C++
 	mysqld.exe!handler::ha_drop_table(const char * name=0x02319f98)  Line 3199 + 0xf bytes	C++
 	mysqld.exe!free_tmp_table(THD * thd=0x04d36098, st_table * entry=0x02319460)  Line 10517	C++
 	mysqld.exe!create_tmp_table(THD * thd=0x04d36098, TMP_TABLE_PARAM * param=0x023656a0, List<Item> & fields={...}, st_order * group=0x02364598, bool distinct=false, bool save_sum_fields=false, unsigned __int64 select_options=2147764736, unsigned __int64 rows_limit=18446744073709551615, char * table_alias=0x0079701c)  Line 10241 + 0x19 bytes	C++
 	mysqld.exe!JOIN::optimize()  Line 1432 + 0x55 bytes	C++
 	mysqld.exe!mysql_select(THD * thd=0x04d36098, Item * * * rref_pointer_array=0x04d3746c, TABLE_LIST * tables=0x023640f8, unsigned int wild_num=0, List<Item> & fields={...}, Item * conds=0x00000000, unsigned int og_num=1, st_order * order=0x00000000, st_order * group=0x02364598, Item * having=0x00000000, st_order * proc_param=0x00000000, unsigned __int64 select_options=2147764736, select_result * result=0x02364628, st_select_lex_unit * unit=0x04d370f8, st_select_lex * select_lex=0x04d37370)  Line 2342 + 0x7 bytes	C++
 	mysqld.exe!handle_select(THD * thd=0x04d36098, st_lex * lex=0x04d37098, select_result * result=0x02364628, unsigned long setup_tables_done_option=0)  Line 269 + 0x79 bytes	C++
 	mysqld.exe!execute_sqlcom_select(THD * thd=0x00000000, TABLE_LIST * all_tables=0x023640f8)  Line 4752 + 0xa bytes	C++
 	mysqld.exe!mysql_execute_command(THD * thd=0x04d36098)  Line 2052 + 0xc bytes	C++
 	mysqld.exe!mysql_parse(THD * thd=0x04d36098, const char * inBuf=0x02363b80, unsigned int length=83, const char * * found_semicolon=0x0542f9e8)  Line 5634	C++
 	mysqld.exe!dispatch_command(enum_server_command command=COM_QUERY, THD * thd=0x04d36098, char * packet=0x0235fb61, unsigned int packet_length=83)  Line 1123	C++
 	mysqld.exe!do_command(THD * thd=0x00000003)  Line 781 + 0xf bytes	C++
 	mysqld.exe!handle_one_connection(void * arg=0x04d36098)  Line 1115 + 0x6 bytes	C++
 	mysqld.exe!pthread_start(void * param=0x02350c10)  Line 85 + 0x3 bytes	C
>	mysqld.exe!_threadstart(void * ptd=0x04d38810)  Line 196 + 0x6 bytes	C

On 32-bit 5.0.58:

 	mysqld-nt.exe!hp_close(st_heap_info * info=0x00000000)  Line 44 + 0x4 bytes	C
 	mysqld-nt.exe!heap_close(st_heap_info * info=0x00000000)  Line 28 + 0xa bytes	C
 	mysqld-nt.exe!ha_heap::close()  Line 113 + 0xc bytes	C++
 	mysqld-nt.exe!free_tmp_table(THD * thd=0x01ac31e0, st_table * entry=0x01ab7490)  Line 10172	C++
 	mysqld-nt.exe!create_tmp_table(THD * thd=0x01ac31e0, TMP_TABLE_PARAM * param=0x01ab6b58, List<Item> & fields={...}, st_order * group=0x01ab5e00, int distinct=0, int save_sum_fields=0, unsigned __int64 select_options=2156153344, unsigned __int64 rows_limit=18446744073709551615, char * table_alias=0x0074301c)  Line 9892 + 0x19 bytes	C++
 	mysqld-nt.exe!JOIN::optimize()  Line 1409 + 0x57 bytes	C++
 	mysqld-nt.exe!mysql_select(THD * thd=0x01ac31e0, Item * * * rref_pointer_array=0x01ac4260, TABLE_LIST * tables=0x01ab5960, unsigned int wild_num=0, List<Item> & fields={...}, Item * conds=0x00000000, unsigned int og_num=1, st_order * order=0x00000000, st_order * group=0x01ab5e00, Item * having=0x00000000, st_order * proc_param=0x00000000, unsigned __int64 select_options=2156153344, select_result * result=0x01ab5ea0, st_select_lex_unit * unit=0x01ac3ea0, st_select_lex * select_lex=0x01ac4130)  Line 2282 + 0x7 bytes	C++
 	mysqld-nt.exe!handle_select(THD * thd=0x01ac31e0, st_lex * lex=0x01ac3e40, select_result * result=0x01ab5ea0, unsigned long setup_tables_done_option=0)  Line 257 + 0x79 bytes	C++
 	mysqld-nt.exe!mysql_execute_command(THD * thd=0x01ac31e0)  Line 2720 + 0xa bytes	C++
 	mysqld-nt.exe!mysql_parse(THD * thd=0x01ac31e0, const char * inBuf=0x01ab5478, unsigned int length=83, const char * * found_semicolon=0x053afb64)  Line 6159	C++
 	mysqld-nt.exe!dispatch_command(enum_server_command command=COM_QUERY, THD * thd=0x01ac31e0, char * packet=0x044e7401, unsigned int packet_length=84)  Line 1876	C++
 	mysqld-nt.exe!do_command(THD * thd=0x00000000)  Line 1580 + 0xe bytes	C++
 	mysqld-nt.exe!handle_one_connection(void * arg=0x01ac31e0)  Line 1186 + 0x9 bytes	C++
 	mysqld-nt.exe!pthread_start(void * param=0x01ab9938)  Line 85 + 0x3 bytes	C
>	mysqld-nt.exe!_threadstart(void * ptd=0x01ac4eb0)  Line 196 + 0x6 bytes	C
[23 Apr 2008 20:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45920

ChangeSet@1.2610, 2008-04-24 00:51:59+04:00, evgen@moonbone.local +3 -0
  Bug#36023: Incorrect handling of zero length caused an assertion to fail.
  
  When a zero length is provided to the my_decimal_length_to_precision
  function along with unsigned_flag set to false it returns a negative value.
  For queries that employs temporary tables may cause failed assertion or
  excessive memory consumption while temporary table creation.
  
  Now the my_decimal_length_to_precision and the my_decimal_precision_to_length
  functions take unsigned_flag into account only if the length/precision
  argument is non-zero.
[1 May 2008 6:16] Bugs System
Pushed into 5.1.25-rc
[1 May 2008 6:19] Bugs System
Pushed into 6.0.6-alpha
[6 May 2008 0:25] Bugs System
Pushed into 5.0.62
[13 May 2008 20:15] Paul DuBois
Noted in 5.0.62, 5.1.25, 6.0.6 changelogs.

An error in calculation of the precision of zero-length items (such
as NULL) caused a server crash for queries that employed temporary
tables.
[6 Aug 2008 14:16] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51012

ChangeSet@1.2613, 2008-04-25 00:39:37+04:00, evgen@moonbone.local +3 -0
  Bug#36023: Incorrect handling of zero length caused an assertion to fail.
  
  When a zero length is provided to the my_decimal_length_to_precision
  function along with unsigned_flag set to false it returns a negative value.
  For queries that employs temporary tables may cause failed assertion or
  excessive memory consumption while temporary table creation.
  
  Now the my_decimal_length_to_precision and the my_decimal_precision_to_length
  functions take unsigned_flag into account only if the length/precision
  argument is non-zero.