| Bug #52538 | Valgrind bug: Item_in_subselect::init_left_expr_cache() (item_subselect.cc:1930) | ||
|---|---|---|---|
| Submitted: | 1 Apr 2010 19:21 | Modified: | 22 Nov 2010 1:34 |
| Reporter: | Patrick Crews | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Optimizer | Severity: | S3 (Non-critical) |
| Version: | 6.0-codebase-bugfixing | OS: | Any |
| Assigned to: | Tor Didriksen | CPU Architecture: | Any |
| Tags: | uninitialized value, valgrind | ||
[1 Apr 2010 19:21]
Patrick Crews
[1 Apr 2010 22:10]
Sveta Smirnova
Thank you for the report. I can not repeat described behavior. Did I understand correctly bug should be repeatable with optimizer_join_cache_level 0-5? Which version of valgrind do you use? How did you bild the server?
[2 Apr 2010 5:40]
Sveta Smirnova
Thank you for the feedback. Verified as described.
[8 Apr 2010 11:54]
Manyi Lu
Omer, I think you have forgot to add the tag. Thanks, Manyi
[8 Apr 2010 20:24]
Patrick Crews
Not present in 5.1-main nor 5.5 trees.
[19 Apr 2010 12:40]
Tor Didriksen
At head of 6.0-codebase-bugfixing I'm getting a segfault: ==9669== Invalid write of size 8 ==9669== at 0x63430D: DTCollation::set(DTCollation&) (item.h:86) ==9669== by 0x61876A: change_cond_ref_to_const(THD*, I_List<COND_CMP>*, Item*, Item*, Item*, Item*) (sql_select.cc:12433) ==9669== by 0x6185D4: change_cond_ref_to_const(THD*, I_List<COND_CMP>*, Item*, Item*, Item*, Item*) (sql_select.cc:12414) ==9669== by 0x618FD9: propagate_cond_constants(THD*, I_List<COND_CMP>*, Item*, Item*) (sql_select.cc:12564) ==9669== by 0x618C64: propagate_cond_constants(THD*, I_List<COND_CMP>*, Item*, Item*) (sql_select.cc:12524) ==9669== by 0x61B26F: optimize_cond(JOIN*, Item*, List<TABLE_LIST>*, bool, Item::cond_result*) (sql_select.cc:13775) ==9669== by 0x5FCF49: JOIN::optimize() (sql_select.cc:1549) ==9669== by 0x7DB490: subselect_single_select_engine::exec() (item_subselect.cc:2253) ==9669== by 0x7D5702: Item_subselect::exec() (item_subselect.cc:296) ==9669== by 0x7D587F: Item_in_subselect::exec() (item_subselect.cc:346) ==9669== by 0x7D7623: Item_in_subselect::val_bool() (item_subselect.cc:942) ==9669== by 0x54F7D2: Item::val_bool_result() (item.h:855) ==9669== by 0x785035: Item_in_optimizer::val_int() (item_cmpfunc.cc:1837) ==9669== by 0x7A74B4: eval_const_cond(Item*) (item_func.cc:78) ==9669== by 0x61B818: internal_remove_eq_conds(THD*, Item*, Item::cond_result*) (sql_select.cc:13912) ==9669== by 0x61B3D3: internal_remove_eq_conds(THD*, Item*, Item::cond_result*) (sql_select.cc:13822) ==9669== Address 0x58 is not stack'd, malloc'd or (recently) free'd
[22 Apr 2010 13:28]
Tor Didriksen
Reverting back a few weeks, in order to reproduce the valgrind warning. ./mtr --valgrind-mysqld --valgrind-option=--track-origins=yes bug52538 yields this: ==23682== Conditional jump or move depends on uninitialised value(s) ==23682== at 0x7D9722: Item_in_subselect::init_left_expr_cache() (item_subselect.cc:1930) ==23682== by 0x7D4836: Item_in_subselect::exec() (item_subselect.cc:317) ==23682== by 0x7D6663: Item_in_subselect::val_bool() (item_subselect.cc:929) ==23682== by 0x54F0F0: Item::val_bool_result() (item.h:846) ==23682== by 0x7849A7: Item_in_optimizer::val_int() (item_cmpfunc.cc:1825) ==23682== by 0x67D85F: SQL_SELECT::skip_record() (opt_range.h:763) ==23682== by 0x74292B: find_all_keys(st_sort_param*, SQL_SELECT*, unsigned char**, st_io_cache*, st_io_cache*, st_io_cache*) (filesort.cc:624) ==23682== by 0x7417B1: filesort(THD*, TABLE*, st_sort_field*, unsigned int, SQL_SELECT*, unsigned long long, bool, unsigned long long*) (filesort.cc:248) ==23682== by 0x62942D: create_sort_index(THD*, JOIN*, st_order*, unsigned long long, unsigned long long, bool) (sql_select.cc:19491) ==23682== by 0x6016F8: JOIN::exec() (sql_select.cc:2926) ==23682== by 0x6020BB: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3180) ==23682== by 0x5F93D2: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:304) ==23682== by 0x5D61FD: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:5009) ==23682== by 0x5CE187: mysql_execute_command(THD*) (sql_parse.cc:2272) ==23682== by 0x5D8315: mysql_parse(THD*, char const*, unsigned int, char const**) (sql_parse.cc:6037) ==23682== by 0x5CB9A6: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1091) ==23682== Uninitialised value was created by a heap allocation ==23682== at 0x4A0515D: malloc (vg_replace_malloc.c:195) ==23682== by 0x98EAB2: my_malloc (my_malloc.c:37) ==23682== by 0x98450B: alloc_root (my_alloc.c:201) ==23682== by 0x54CF44: Query_arena::alloc(unsigned long) (sql_class.h:599) ==23682== by 0x7DBEBB: subselect_hash_sj_engine::init_permanent(List<Item>*) (item_subselect.cc:3107) ==23682== by 0x7D9494: Item_in_subselect::setup_engine() (item_subselect.cc:1843) ==23682== by 0x60383E: JOIN::setup_subquery_materialization() (sql_select.cc:3773) ==23682== by 0x5FDE52: JOIN::optimize() (sql_select.cc:2005) ==23682== by 0x602025: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3166) ==23682== by 0x5F93D2: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:304) ==23682== by 0x5D61FD: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:5009) ==23682== by 0x5CE187: mysql_execute_command(THD*) (sql_parse.cc:2272) ==23682== by 0x5D8315: mysql_parse(THD*, char const*, unsigned int, char const**) (sql_parse.cc:6037) ==23682== by 0x5CB9A6: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1091) ==23682== by 0x5CADBD: do_command(THD*) (sql_parse.cc:775) ==23682== by 0x6BBB5F: do_handle_one_connection(THD*) (sql_connect.cc:1173)
[22 Apr 2010 14:14]
Tor Didriksen
The previous valgrind stack was a red herring (??)
The warning disappears if I do this:
bool
JOIN::make_simple_join(JOIN *parent, TABLE *tmp_table)
{
DBUG_ENTER("JOIN::make_simple_join");
/*
Reuse TABLE * and JOIN_TAB if already allocated by a previous call
to this function through JOIN::exec (may happen for sub-queries).
*/
if (!parent->join_tab_reexec)
{
if (!(parent->join_tab_reexec= (JOIN_TAB*) thd->alloc(sizeof(JOIN_TAB))))
DBUG_RETURN(TRUE); /* purecov: inspected */
parent->join_tab_reexec->next_select= NULL;
}
join_tab= parent->join_tab_reexec;
[26 Apr 2010 13:37]
Tor Didriksen
The original valgrind warning is caused by Item_in_subselect::exec() It first calls init_left_expr_cache(), and then Item_subselect::exec() Item_subselect::exec() will optimize() and exec() the query, and the next_select function pointer will not be initialized until we actually call do_select()
[30 Apr 2010 13:57]
Tor Didriksen
The original query can be simplified to
SELECT table2 . col_varchar_key AS field2 ,
table2 . col_int_nokey AS field1
FROM ( C AS table1 LEFT OUTER JOIN C AS table2
ON (table2 . col_varchar_key = table1 . col_varchar_key ) )
WHERE table1.pk = 6
HAVING ( 8, 2 ) IN
( SELECT SUBQUERY2_t2 . col_int_nokey AS SUBQUERY2_field1 ,
SUBQUERY2_t2 . col_int_nokey AS SUBQUERY2_field2
FROM ( C AS SUBQUERY2_t1 JOIN C AS SUBQUERY2_t2
ON (SUBQUERY2_t2 . col_varchar_key = SUBQUERY2_t1 . col_varchar_key ) ) )
ORDER BY field2
;
The warning goes away with the introduction of
sergey.glukhov@sun.com-20100319092029-m2t1vgnovw5es2o6
This is his fix for:
Bug#48916 Server incorrectly processing HAVING clauses with an ORDER BY clause
The warning disappears, because we discover an 'impossible having' clause.
Removing the 'impossibleness' of the having clause, we get back the warning:
SELECT table2 . col_varchar_key AS field2 ,
table2 . col_int_nokey AS field1
FROM ( C AS table1 LEFT OUTER JOIN C AS table2
ON (table2 . col_varchar_key = table1 . col_varchar_key ) )
WHERE table1.pk = 6
HAVING ( 8, field1 ) IN
( SELECT SUBQUERY2_t2 . col_int_nokey AS SUBQUERY2_field1 ,
SUBQUERY2_t2 . col_int_nokey AS SUBQUERY2_field2
FROM ( C AS SUBQUERY2_t1 JOIN C AS SUBQUERY2_t2
ON (SUBQUERY2_t2 . col_varchar_key = SUBQUERY2_t1 . col_varchar_key ) ) )
ORDER BY field2
;
[8 Jun 2010 6:10]
Tor Didriksen
The root cause is lack of initialization of JOIN_TAB (and sub-) objects.
Two proposed solutions
- use tab= (JOIN_TAB*) thd->calloc(sizeof(JOIN_TAB))
rather than thd->alloc(sizeof(JOIN_TAB))
this will bzero() everything
- initialize properly, with constructors, by inheriting from Sql_alloc,
and using placement new: tab= new (thd->mem_root) JOIN_TAB;
I wrote a small unit test to compare the two alternatives:
const int num_allocations= 1000 * 1000 * 100;
const size_t table_count= 4;
// Tests allocation with thd->calloc()
TEST(Join_tab_test, use_calloc)
{
MEM_ROOT mem_root;
for (int i= 0; i < num_allocations; ++i)
{
init_sql_alloc(&mem_root, (table_count+1)*sizeof(JOIN_TAB), 0);
JOIN_TAB *tab;
if (tab= (JOIN_TAB*) alloc_root(&mem_root, sizeof(JOIN_TAB)))
bzero(tab, table_count * sizeof(*tab));
for (int i= 0; i < num_allocations; ++i)
{
tab[i].keys.init();
tab[i].const_keys.init();
tab[i].checked_keys.init();
tab[i].needed_reg.init();
}
free_root(&mem_root, MYF(0));
}
}
// Tests allocation with placement new.
TEST(Join_tab_test, use_ctor)
{
MEM_ROOT mem_root;
for (int i= 0; i < num_allocations; ++i)
{
init_sql_alloc(&mem_root, (table_count+1)*sizeof(JOIN_TAB), 0);
JOIN_TAB *tab= new (&mem_root) JOIN_TAB[table_count];
for (int i= 0; i < num_allocations; ++i)
{}
free_root(&mem_root, MYF(0));
}
}
./join-tab-t --disable-tap-output
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from Join_tab_test
[ RUN ] Join_tab_test.use_calloc
[ OK ] Join_tab_test.use_calloc (16296 ms)
[ RUN ] Join_tab_test.use_ctor
[ OK ] Join_tab_test.use_ctor (23403 ms)
[----------] 2 tests from Join_tab_test (39699 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (39699 ms total)
[ PASSED ] 2 tests.
[10 Jun 2010 14:12]
Tor Didriksen
- we prepare/optimize/execute a JOIN object with two JOIN_TABs - do_select sets join_tab[1]->next_select to end_write - the result of the execution is stored in a temporary table - then we convert this JOIN object in make_simple_join into a JOIN with only one JOIN_TAB, created as join_tab_reexec this JOIN_TAB is not properly initialized. - Item_in_subselect::exec calls init_left_expr_cache - init_left_expr_cache cache inspects join_tab[0]->next_select and reads uninitialized data.
[11 Jun 2010 13:38]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/110843 3192 Tor Didriksen 2010-06-11 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use thd->calloc() rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_select.cc Initialize allocated JOIN_TAB.
[11 Jun 2010 13:38]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/110845 3192 Tor Didriksen 2010-06-11 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use new (thd->mem_root) rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_bitmap.h Initialize Bitmap in constructor. @ sql/sql_select.cc Initialize allocated JOIN_TAB. Remove some assignments, no longer necessary since object is properly constructed. @ sql/sql_select.h Let st_join_table inherit new allocators from Sql_alloc. Implement conctructor for st_join_table
[11 Jun 2010 13:39]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/110847 3193 Tor Didriksen 2010-06-11 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use thd->calloc() rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_select.cc Initialize allocated JOIN_TAB.
[11 Jun 2010 13:39]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/110848 3193 Tor Didriksen 2010-06-11 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use new (thd->mem_root) rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_bitmap.h Initialize Bitmap in constructor. @ sql/sql_select.cc Initialize allocated JOIN_TAB. Remove some assignments, no longer necessary since object is properly constructed. @ sql/sql_select.h Let st_join_table inherit new allocators from Sql_alloc. Implement conctructor for st_join_table
[14 Jun 2010 12:38]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/111018 3197 Tor Didriksen 2010-06-14 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use new (thd->mem_root) rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_bitmap.h Initialize Bitmap in constructor. @ sql/sql_select.cc Initialize allocated JOIN_TAB. Remove some assignments, no longer necessary since object is properly constructed. @ sql/sql_select.h Let st_join_table inherit new allocators from Sql_alloc. Implement conctructor for st_join_table
[14 Jun 2010 12:38]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/111019 3197 Tor Didriksen 2010-06-14 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use thd->calloc() rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_select.cc Initialize allocated JOIN_TAB.
[15 Jun 2010 9:11]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/111099 3197 Tor Didriksen 2010-06-15 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use new (thd->mem_root) rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_bitmap.h Initialize Bitmap in constructor. @ sql/sql_select.cc Initialize allocated JOIN_TAB. Remove some assignments, no longer necessary since object is properly constructed. Use placement new for SJ_TMP_TABLE objects. Use object assignment rather than memcpy for join_tab_save. @ sql/sql_select.h Let st_join_table inherit new allocators from Sql_alloc. Implement conctructor for st_join_table
[15 Jun 2010 12:33]
Guilhem Bichot
approved with last comments sent by mail
[15 Jun 2010 13:58]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/111160 3197 Tor Didriksen 2010-06-15 Bug #52538 Valgrind bug: Item_in_subselect::init_left_expr_cache() The JOIN_TAB created in make_simple_join() was not properly initialized. Use new (thd->mem_root) rather than thd->alloc() @ mysql-test/r/subselect_mat.result Add test case. @ mysql-test/t/subselect_mat.test Add test case. @ sql/item_subselect.cc Do not ignore return value of init_left_expr_cache() Initialize allocated JOIN_TAB. Don't do function calls in DBUG_RETURN(). @ sql/item_subselect.h Introduce new member need_expr_cache. @ sql/sql_bitmap.h Initialize Bitmap in constructor. @ sql/sql_select.cc Initialize allocated JOIN_TAB. Remove some assignments, no longer necessary since object is properly constructed. Use placement new for SJ_TMP_TABLE objects. Use object assignment rather than memcpy for join_tab_save. @ sql/sql_select.h Let st_join_table inherit new allocators from Sql_alloc. Implement conctructor for st_join_table
[16 Jun 2010 8:31]
Tor Didriksen
Pushed to next-mr-opt-backporting
[16 Jun 2010 11:43]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/111251 3198 Tor Didriksen 2010-06-16 Followup to Bug #52538 Valgrind bug Instances of Bitmap<64> are now initialized when constructed, so there is no need to do clear_all() @ sql/field.cc Use default constructor for member variables of type key_map. @ sql/mysqld.cc key_map_full can be initialized at construct time. @ sql/mysqld.h key_map_full is const @ sql/opt_range.h Use non-default constructor to initialize key_map tmp; @ sql/partition_info.h Remove un-necessary clear_all calls. @ sql/sql_bitmap.h Add a named constant for all bits. @ sql/sql_select.cc No need to init(0) variables of type key_map.
[17 Jun 2010 9:31]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/111380 3199 Tor Didriksen 2010-06-17 Followup to Bug #52538 Valgrind bug Instances of Bitmap<64> are now initialized when constructed, so there is no need to do clear_all() @ sql/field.cc Use default constructor for member variables of type key_map. @ sql/opt_range.cc Default constructor for Bitmap<64> initializes objects, so no need for clear_all() @ sql/opt_range.h Use non-default constructor to initialize key_map tmp; @ sql/partition_info.h Remove un-necessary clear_all calls. @ sql/sql_bitmap.h Add a named constant for all bits. @ sql/sql_select.cc No need to init(0) variables of type key_map. Move declaration of usable_keys, and initialize it directly. @ sql/table.cc No need for the initialization loop of key_map arrays.
[21 Jun 2010 10:34]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/111673 3208 Tor Didriksen 2010-06-21 Followup to Bug #52538 Valgrind bug Instances of Bitmap<64> are now initialized when constructed, so there is no need to do clear_all() @ sql/opt_range.cc No need to do used_key_parts_map.clear_all() @ sql/sql_base.cc Remove unused function. @ sql/sql_base.h Remove unused function.
[16 Aug 2010 6:37]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100816062819-bluwgdq8q4xysmlg) (version source revid:alik@sun.com-20100816062612-enatdwnv809iw3s9) (pib:20)
[13 Nov 2010 16:13]
Bugs System
Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (merge vers: 5.6.99-m4) (pib:21)
[22 Nov 2010 1:34]
Paul DuBois
Bug is not in any released 5.6.x version. No changelog entry needed.
