MySQL Bugs: #60034: valgrind errors then assertion in make_sortkey when grouping by make

Bug #60034	valgrind errors then assertion in make_sortkey when grouping by make_set
Submitted:	9 Feb 2011 6:06	Modified:	24 Dec 2012 9:44
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	MySQL Server: Optimizer	Severity:	S1 (Critical)
Version:	4.1.25,5.0.92,5.1.56,5.5.10,5.6.2	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	assertion, make_set, valgrind

Description:
valgrind warnings with either an assertion failure or duplicate group_key message;

Version: '5.6.2-m5-valgrind-max-debug'  socket: 'sock'  port: 3306  Source distribution
1 errors in context 1 of 461:
Conditional jump or move depends on uninitialised value(s)
at: my_strnncollsp_simple (ctype-simple.c:171)
by: hp_key_cmp (hp_hash.c:656)
by: hp_search (hp_hash.c:118)
by: heap_rkey (hp_rkey.c:61)
by: ha_heap::index_read_map (ha_heap.cc:295)
by: handler::ha_index_read_map (handler.cc:2251)
by: end_update (sql_select.cc:18827)
by: evaluate_join_record (sql_select.cc:17775)
by: sub_select (sql_select.cc:17469)
by: do_select (sql_select.cc:16997)
by: JOIN::exec (sql_select.cc:2917)
by: mysql_select (sql_select.cc:3584)
by: handle_select (sql_select.cc:323)
by: execute_sqlcom_select (sql_parse.cc:4512)
by: mysql_execute_command (sql_parse.cc:2095)
by: mysql_parse (sql_parse.cc:5549)
by: dispatch_command (sql_parse.cc:1078)
by: do_command (sql_parse.cc:815)
by: do_handle_one_connection (sql_connect.cc:748)
by: handle_one_connection (sql_connect.cc:684)
by: startthread (pthread_create.c:301)

 Uninitialised value was created by a heap allocation
at: malloc (vg_replace_malloc.c:195)
by: my_malloc (my_malloc.c:38)
by: alloc_root (my_alloc.c:166)
by: multi_alloc_root (my_alloc.c:280)
by: create_tmp_table (sql_select.cc:15400)
by: JOIN::optimize (sql_select.cc:2526)
by: mysql_select (sql_select.cc:3570)
by: handle_select (sql_select.cc:323)
by: execute_sqlcom_select (sql_parse.cc:4512)
by: mysql_execute_command (sql_parse.cc:2095)
by: mysql_parse (sql_parse.cc:5549)
by: dispatch_command (sql_parse.cc:1078)
by: do_command (sql_parse.cc:815)
by: do_handle_one_connection (sql_connect.cc:748)
by: handle_one_connection (sql_connect.cc:684)
by: start_thread (pthread_create.c:301)

If you see the assertion, it will look like this:
Version: '5.6.2-m5-valgrind-max-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
mysqld: ./sql/filesort.cc:878: void make_sortkey(Sort_param*, uchar*, uchar*): Assertion `0' failed.
110209  5:17:37 - mysqld got signal 6 ;

How to repeat:
#start mysqld in valgrind then:

drop table if exists `g59`;
create table `g59`(`a` blob)engine=myisam;
insert into `g59` values (),();
select count(*) from `g59`  
group by @`a`:=make_set(`a`,1) desc;

some details....from 5.6.2

Attachment: bug60034_valgrind_outputs.txt (text/plain), 19.68 KiB.

Verified on 32-but Ubuntu 10.04:

==1786== Thread 18:
==1786== Conditional jump or move depends on uninitialised value(s)
==1786==    at 0x8823A8B: my_strnncollsp_utf8 (ctype-utf8.c:5585)
==1786==    by 0x877D126: hp_key_cmp (hp_hash.c:656)
==1786==    by 0x877C0D5: hp_search (hp_hash.c:118)
==1786==    by 0x877F48B: heap_rkey (hp_rkey.c:61)
==1786==    by 0x8779DF5: ha_heap::index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function) (ha_heap.cc:295)
==1786==    by 0x819EEA8: handler::ha_index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function) (handler.cc:2251)
==1786==    by 0x8355ED2: end_update(JOIN*, st_join_table*, bool) (sql_select.cc:18827)
==1786==    by 0x83534C2: evaluate_join_record(JOIN*, st_join_table*, int) (sql_select.cc:17775)
==1786==    by 0x8352B94: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:17469)
==1786==    by 0x8351FF6: do_select(JOIN*, List<Item>*, TABLE*, Procedure*) (sql_select.cc:16997)
==1786==    by 0x83314EA: JOIN::exec() (sql_select.cc:2917)
==1786==    by 0x83336C8: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3584)
==1786== 
==1786== 
==1786== 1 errors in context 2 of 393:
==1786== Conditional jump or move depends on uninitialised value(s)
==1786==    at 0x877D0AC: hp_key_cmp (hp_hash.c:651)
==1786==    by 0x877C0D5: hp_search (hp_hash.c:118)
==1786==    by 0x877F48B: heap_rkey (hp_rkey.c:61)
==1786==    by 0x8779DF5: ha_heap::index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function) (ha_heap.cc:295)
==1786==    by 0x819EEA8: handler::ha_index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function) (handler.cc:2251)
==1786==    by 0x8355ED2: end_update(JOIN*, st_join_table*, bool) (sql_select.cc:18827)
==1786==    by 0x83534C2: evaluate_join_record(JOIN*, st_join_table*, int) (sql_select.cc:17775)
==1786==    by 0x8352B94: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:17469)
==1786==    by 0x8351FF6: do_select(JOIN*, List<Item>*, TABLE*, Procedure*) (sql_select.cc:16997)
==1786==    by 0x83314EA: JOIN::exec() (sql_select.cc:2917)
==1786==    by 0x83336C8: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3584)
==1786==    by 0x832AA97: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:323)

...

Fixed as a duplicate of an internally filed bug.

This was fixed only in 5.6 tree (5.6.5)