MySQL Bugs: #58705: optimizer let engine depend on uninitialized values created by opt_sum

Bug #58705	optimizer let engine depend on uninitialized values created by opt_sum_query
Submitted:	3 Dec 2010 16:14	Modified:	3 May 2011 1:07
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Optimizer	Severity:	S1 (Critical)
Version:	5.1.54, 5.1.55, 5.5.9, 5.6.1	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
5.5.9 valgrind output for engine=myisam:

Conditional jump or move depends on uninitialised value(s)
at: ha_key_cmp (my_handler.c:308)
by: _mi_bin_search (mi_search.c:200)
by: _mi_search (mi_search.c:89)
by: mi_rkey (mi_rkey.c:94)
by: ha_myisam::index_read_map (ha_myisam.cc:1558)
<cut: see attachment for full output>

5.5.9 valgrind output for engine=innodb:

Conditional jump or move depends on uninitialised value(s)
at: cmp_dtuple_rec_with_match (rem0cmp.c:582)
by: page_cur_search_with_match (page0cur.c:437)
by: btr_cur_search_to_nth_level (btr0cur.c:699)
by: btr_pcur_open_with_no_init_func (btr0pcur.ic:542)
<cut: see attachment for full output>

How to repeat:
run mysqld in valgrind, then:

drop table if exists `t1`;
create table `t1`(`a` int not null,key (`a`)) engine=innodb;
insert into `t1` values (0),(1);
select 1 from `t1` where `a` < some
 (select `a` from `t1` where `a` <=>
   (select `a` from `t1`)
 );

5.5.59 full valgrind + system info output

Attachment: bug58705_full_output.txt (text/plain), 33.09 KiB.

This is what I see with 5.1.54-valgrind on 32-bit Ubuntu 10.04:

==2032== Conditional jump or move depends on uninitialised value(s)
==2032==    at 0x825D610: key_cmp_if_same(st_table*, unsigned char const*, unsigned int, unsigned int) (key.cc:306)
==2032==    by 0x83B6D96: reckey_in_range(bool, st_table_ref*, Field*, Item*, unsigned int, unsigned int) (opt_sum.cc:923)
==2032==    by 0x83B5B80: opt_sum_query(TABLE_LIST*, List<Item>&, Item*) (opt_sum.cc:385)
==2032==    by 0x82F3398: JOIN::optimize() (sql_select.cc:959)
==2032==    by 0x823150E: subselect_single_select_engine::exec() (item_subselect.cc:1901)
==2032==    by 0x822C0D4: Item_subselect::exec() (item_subselect.cc:280)
==2032==    by 0x822CFE2: Item_singlerow_subselect::val_int() (item_subselect.cc:593)
==2032==    by 0x81F2C5F: Arg_comparator::compare_int_signed() (item_cmpfunc.cc:1462)
==2032==    by 0x81D684D: Arg_comparator::compare() (item_cmpfunc.h:81)
==2032==    by 0x81F40B0: Item_func_lt::val_int() (item_cmpfunc.cc:1999)
==2032==    by 0x81F044E: Item_func_nop_all::val_int() (item_cmpfunc.cc:357)
==2032==    by 0x830E86A: evaluate_join_record(JOIN*, st_join_table*, int) (sql_select.cc:11469)

I also see this:

==2032== Conditional jump or move depends on uninitialised value(s)
==2032==    at 0x852542D: cmp_dtuple_rec_with_match (rem0cmp.c:584)
==2032==    by 0x851C51B: page_cur_search_with_match (page0cur.c:414)
==2032==    by 0x84D0C67: btr_cur_search_to_nth_level (btr0cur.c:516)
==2032==    by 0x855EC61: btr_pcur_open_with_no_init (btr0pcur.ic:535)
==2032==    by 0x8539933: row_search_for_mysql (row0sel.c:3673)
==2032==    by 0x84C89C3: ha_innobase::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function) (ha_innodb.cc:4742)
==2032==    by 0x83C68A9: handler::index_read_map(unsigned char*, unsigned char const*, unsigned long, ha_rkey_function) (handler.h:1404)
==2032==    by 0x83B53EC: get_index_max_value(st_table*, st_table_ref*, unsigned int) (opt_sum.cc:206)
==2032==    by 0x83B5AEF: opt_sum_query(TABLE_LIST*, List<Item>&, Item*) (opt_sum.cc:382)
==2032==    by 0x82F3398: JOIN::optimize() (sql_select.cc:959)
==2032==    by 0x823150E: subselect_single_select_engine::exec() (item_subselect.cc:1901)
==2032==    by 0x822C0D4: Item_subselect::exec() (item_subselect.cc:280)
==2032== 
...

Could this be related to Bug#58628?

Bug#58628 failed to detect errors returned from store_key::copy() while the constant part of a REF key was build. This could later cause a partial uninitialized REF key to be used when accessing rows from the storage engines.

Ole John, 

Could you please check if your patch for Bug#58628 also fixes this bug? If so, please close this one as duplicate.

Thanks,
Manyi

Check with patch for bug#58628 - and it did not fix this problem.

Unassign myself from this bug.

i'm seeing crashes (signal 11) on 5.6.2 due to this bug...

segfaults 5.6.2 when run in valgrind:

drop table if exists t1;
create table `t1`(`a` varbinary(1)not null,key(a))engine=innodb;
insert into `t1` values (''),('');
select 1 from `t1`
where 1 <= some 
  (select `a` from `t1` where `a`  <=> 
    (select `a` from `t1`)
  );

this bug causes segfaults in 5.5 and 5.6...

Attachment: bug58705_5.5.10_crash.txt (text/plain), 13.90 KiB.

Noted in 5.1.58, 5.5.13, 5.6.3 changelogs.

Valgrind warnings caused by comparing index values to an
uninitialized field were corrected. 

CHANGESET - http://lists.mysql.com/commits/135426