| Bug #43360 | Server crash with a simple multi-table update (InnoDB table) | ||
|---|---|---|---|
| Submitted: | 4 Mar 2009 10:10 | Modified: | 22 Nov 2010 0:44 |
| Reporter: | Amit Saha | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Optimizer | Severity: | S3 (Non-critical) |
| Version: | 6.0-bzr | OS: | Linux (Ubuntu 8.04) |
| Assigned to: | Olav Sandstå | CPU Architecture: | Any |
| Tags: | index_condition_pushdown, optimizer_switch | ||
[4 Mar 2009 10:54]
Sveta Smirnova
Thank you for the report. Verified as described. Bug is only repeatable if storage engine used is InnoDB. Backtrace from my environment: #0 0x002ce402 in __kernel_vsyscall () #0 0x002ce402 in __kernel_vsyscall () #1 0x0046264f in pthread_kill () from /lib/libpthread.so.0 #2 0x08821c0b in my_write_core (sig=6) at stacktrace.c:309 #3 0x082be0f7 in handle_segfault (sig=6) at mysqld.cc:2690 #4 <signal handler called> #5 0x002ce402 in __kernel_vsyscall () #6 0x00314f90 in raise () from /lib/libc.so.6 #7 0x00316678 in abort () from /lib/libc.so.6 #8 0x086d564f in mem_area_free (ptr=0xadb01528, pool=0xaacad40) at mem/mem0pool.c:518 #9 0x086d48a2 in mem_heap_block_free (heap=0xadb01528, block=0xadb01528) at mem/mem0mem.c:524 #10 0x086d3ad5 in mem_heap_free_func (heap=0xadb01528, file_name=0x8a0b1ee "row/row0mysql.c", line=710) at ../../storage/innobase/include/mem0mem.ic:487 #11 0x086d3bbf in mem_free_func (ptr=0xadb01568, file_name=0x8a0b1ee "row/row0mysql.c", line=710) at ../../storage/innobase/include/mem0mem.ic:543 #12 0x086ed0c0 in row_prebuilt_free (prebuilt=0xadb0ae68) at row/row0mysql.c:710 #13 0x08690c2b in ha_innobase::close (this=0xb1dd188) at handler/ha_innodb.cc:2665 #14 0x0832e520 in closefrm (table=0xb1dcab0, free_share=true) at table.cc:2112 #15 0x0831d8d7 in intern_close_table (table=0xb1dcab0) at sql_base.cc:800 #16 0x0831d92b in free_cache_entry (table=0xb1dcab0) at sql_base.cc:822 #17 0x08327f9c in tdc_remove_table (thd=0xb2831e0, remove_type=TDC_RT_REMOVE_ALL, db=0xb200118 "test", table_name=0xb1ffef0 "t6") at sql_base.cc:7686 #18 0x0842529e in mysql_rm_table_part2 (thd=0xb2831e0, tables=0xb1fff18, if_exists=false, drop_temporary=false, drop_view=false, dont_log_query=false) at sql_table.cc:1622 #19 0x08425d26 in mysql_rm_table (thd=0xb2831e0, tables=0xb1fff18, if_exists=0 '\0', drop_temporary=0 '\0') at sql_table.cc:1512 #20 0x082d6649 in mysql_execute_command (thd=0xb2831e0) at sql_parse.cc:3344 #21 0x082dbac3 in mysql_parse (thd=0xb2831e0, inBuf=0xb1ffc50 "drop table t6,t7", length=16, found_semicolon=0xa7ec3f20) at sql_parse.cc:5763 #22 0x082dc505 in dispatch_command (command=COM_QUERY, thd=0xb2831e0, packet=0xb21aba9 "drop table t6,t7", packet_length=16) at sql_parse.cc:1009 #23 0x082dd869 in do_command (thd=0xb2831e0) at sql_parse.cc:691 #24 0x082cb657 in handle_one_connection (arg=0xb2831e0) at sql_connect.cc:1146 #25 0x0045fbd4 in start_thread () from /lib/libpthread.so.0 #26 0x003b74fe in clone () from /lib/libc.so.6
[5 Mar 2009 10:59]
Sveta Smirnova
test case
Attachment: bug43360.test (application/octet-stream, text), 854 bytes.
[5 Mar 2009 11:03]
Sveta Smirnova
Working test case provided.
[5 Mar 2009 13:45]
Heikki Tuuri
Memory corruption => Valgrind the code.
[5 Mar 2009 14:32]
Marko Mäkelä
I can repeat this with UNIV_DEBUG: #0 ut_dbg_assertion_failed (expr=0x8812a56 "n > 0", file=0x8812ab0 "ut/ut0ut.c", line=418) at ut/ut0dbg.c:43 #1 0x0858ef3a in ut_2_power_up (n=0) at ut/ut0ut.c:418 #2 0x0853cb4a in mem_area_free (ptr=0xb6ee9028, pool=0x982b5b8) at mem/mem0pool.c:509 #3 0x0853b871 in mem_heap_block_free (heap=0xb6ee9028, block=0xb6ee9028) at mem/mem0mem.c:524 #4 0x0853a48f in mem_heap_free_func (heap=0xb6ee9028, file_name=0x880766a "row/row0mysql.c", line=710) at ../../storage/innobase/include/mem0mem.ic:487 #5 0x0853a579 in mem_free_func (ptr=0xb6ee9068, file_name=0x880766a "row/row0mysql.c", line=710) at ../../storage/innobase/include/mem0mem.ic:543 #6 0x0855c611 in row_prebuilt_free (prebuilt=0xb6ee7668) at row/row0mysql.c:710 #7 0x084df349 in ha_innobase::close (this=0x9a0c2e8) at handler/ha_innodb.cc:2665 #8 0x082d2454 in closefrm (table=0x9b48560, free_share=true) at table.cc:2112 #9 0x082c196d in intern_close_table (table=0x9b48560) at sql_base.cc:800 #10 0x082c19c1 in free_cache_entry (table=0x9b48560) at sql_base.cc:822 #11 0x082c1c00 in tdc_remove_table (thd=0x9a91630, remove_type=TDC_RT_REMOVE_ALL, db=0x9b37698 "test", table_name=0x9b37470 "t6") at sql_base.cc:7686 #12 0x083ba6c6 in mysql_rm_table_part2 (thd=0x9a91630, tables=0x9b37498, if_exists=false, drop_temporary=false, drop_view=false, dont_log_query=false) at sql_table.cc:1622 #13 0x083bb155 in mysql_rm_table (thd=0x9a91630, tables=0x9b37498, if_exists=0 '\0', drop_temporary=0 '\0') at sql_table.cc:1512 #14 0x082739ad in mysql_execute_command (thd=0x9a91630) at sql_parse.cc:3344 #15 0x082787b0 in mysql_parse (thd=0x9a91630, inBuf=0x9b37400 "drop table t6,t7", length=16, found_semicolon=0xb1c55e80) at sql_parse.cc:5763 UNIV_DEBUG_VALGRIND didn't help much further: InnoDB: Error: Memory area size 256, next area size 0 not a power of 2! InnoDB: Possibly a memory overrun of the buffer being freed here. InnoDB: Apparent memory corruption: mem dump ==1651== Thread 11: ==1651== Uninitialised byte(s) found during client check request ==1651== at 0x856BF03: ut_print_buf (ut0ut.c:374) ==1651== by 0x8528109: mem_analyze_corruption (mem0dbg.c:766) ==1651== by 0x8529CD1: mem_area_free (mem0pool.c:516) ==1651== by 0x8528CD7: mem_heap_block_free (mem0mem.c:524) This was with http://bazaar.launchpad.net/%7Emysql/mysql-server/mysql-6.0/ checked out an hour ago.
[6 Mar 2009 8:16]
Marko Mäkelä
I can't repeat this after replacing the storage/innobase tree of the bzr snapshot with a snapshot of our private InnoDB tree. I glanced the 1,983 lines of diff, and the differences seem to be mostly fixes supplied by us that haven't been applied to the MySQL tree yet. The only differences related to dynamic memory management were in error handling in dict0load.c and in the handling of savepoints. The test case should trigger neither. In other words, this bug would most likely be fixed by applying our 6.0 snapshot to the MySQL tree. Reassigning to Calvin.
[6 Mar 2009 8:28]
Marko Mäkelä
I ran one more test with UNIV_MEM_DEBUG, and the corruption wasn't noticed any earlier: #3 0x0853bb66 in mem_hash_remove (heap=0xb6e3dd28, file_name=0x88099fa "row/row0mysql.c", line=710) at mem/mem0dbg.c:371 #4 0x0853acac in mem_heap_free_func (heap=0xb6e3dd28, file_name=0x88099fa "row/row0mysql.c", line=710) at ../../storage/innobase/include/mem0mem.ic:473 #5 0x0853ae03 in mem_free_func (ptr=0xb6e3dd70, file_name=0x88099fa "row/row0mysql.c", line=710) at ../../storage/innobase/include/mem0mem.ic:543 #6 0x0855e139 in row_prebuilt_free (prebuilt=0xb6e3b670) at row/row0mysql.c:710 #7 0x084df679 in ha_innobase::close (this=0xa088fb8) at handler/ha_innodb.cc:2665 Some relevant log output before that: InnoDB: Error: block b6e3dd28 mem field b6e3dd68 len 104 InnoDB: header check field is a306d676 but trailer 0 Inconsistency in memory heap or buffer n:o 279 created in handler/ha_innodb.cc line 3323 and tried to free in row/row0mysql.c line 710. Hex dump of 400 bytes around memory heap first block start: len 400; hex 00000000addededeaddeadadaddedeadadaddededeaddeaddedeaddedeaddedededeadadaddeadaddeadadaddedededeadaddeadadadadaddededeadaddeaddedeadadadadaddeadaddeadadaddedeaddeadaddeadadaddeaddedeadadaddeaddeadadaddedeaddedeaddeaddedeaddeadadaddedededeaddeaddeaddedededeadadadadadadaddedeaddedeadadaddedeaddedeadaddeaddeaddeadaddededeaddedededeadadaddeadadadadaddededeadaddeaddedede00010000ffffffffffffffffdedededeb307952d6e6f64622e636300fb0c00000100000028dde3b628dde3b60000000000000000b80000000000000000000000b80000004000000000000000adaddead00000068a306d67600000000010000000000000002000000babababa0000000002000000fe000000bebababa0800000001000000010000000000000001000000000000000200000015000000bebebaba00000000010000000f000000010000000800000001000000010000000000000000000000000000000000000002000000adadadad00000000; asc -nodb.cc ( ( @ h v ; Dump of the mem heap: Memory heap: Block 0: len 104; hex 00000000010000000000000002000000babababa0000000002000000fe000000bebababa0800000001000000010000000000000001000000000000000200000015000000bebebaba00000000010000000f0000000100000008000000010000000100000000000000; asc ;InnoDB: Error: block b6e3dd28 mem field b6e3dd68 len 104 InnoDB: header check field is a306d676 but trailer 0 090306 10:23:09 InnoDB: Assertion failure in thread 2998257552 in file mem/mem0dbg.c line 371 Again, I wouldn't investigate this further, as we can't repeat this with our 6.0 tree.
[10 Mar 2009 7:16]
Calvin Sun
Well, it is no surprise that the root cause of the crash is again the index condition pushdown. The crash goes away if HA_DO_INDEX_COND_PUSHDOWN is not returned. In ha_innodb.h, change the code
from:
ulong index_flags(uint idx, uint part, bool all_parts) const
{
return (HA_READ_NEXT |
HA_READ_PREV |
HA_READ_ORDER |
HA_READ_RANGE |
HA_KEYREAD_ONLY |
((idx == primary_key)? 0 : HA_DO_INDEX_COND_PUSHDOWN));
to:
ulong index_flags(uint idx, uint part, bool all_parts) const
{
return (HA_READ_NEXT |
HA_READ_PREV |
HA_READ_ORDER |
HA_READ_RANGE |
HA_KEYREAD_ONLY);
I can no longer see the crash after the change, even with the current MySQL 6.0 tree.
[9 Jun 2009 14:54]
Guilhem Bichot
Still crashes with azalea using this testcase:
[5 Mar 11:59] Sveta Smirnova
(compile-pentium64-valgrind-max):
At line 17: query 'drop table t6,t7' failed: 2013: Lost connection to MySQL server during query
#2 0x00000000006eda97 in handle_segfault (sig=11) at mysqld.cc:2714
#3 <signal handler called>
#4 0x00000000009df31b in mem_analyze_corruption (ptr=0x7f1f4bf60430) at mem/mem0dbg.c:824
#5 0x00000000009e0e39 in mem_area_free (ptr=0x7f1f4bf60448, pool=0x153c640) at mem/mem0pool.c:516
#6 0x00000000009dfedf in mem_heap_block_free (heap=0x7f1f4bf60448, block=0x7f1f4bf60448) at mem/mem0mem.c:524
#7 0x00000000009def50 in mem_heap_free_func (heap=0x7f1f4bf60448, file_name=0xd9f1b8 "row/row0mysql.c", line=710)
at ../../storage/innobase/include/mem0mem.ic:487
#8 0x00000000009df03d in mem_free_func (ptr=0x7f1f4bf604b8, file_name=0xd9f1b8 "row/row0mysql.c", line=710)
at ../../storage/innobase/include/mem0mem.ic:543
#9 0x00000000009f9b39 in row_prebuilt_free (prebuilt=0x7f1f4bf5f8b8) at row/row0mysql.c:710
#10 0x0000000000993e8d in ha_innobase::close (this=0x20995f8) at handler/ha_innodb.cc:2681
#11 0x000000000076a0c3 in closefrm (table=0x2062830, free_share=true) at table.cc:2115
#12 0x0000000000756ffe in intern_close_table (table=0x2062830) at sql_base.cc:800
#13 0x0000000000757044 in free_cache_entry (table=0x2062830) at sql_base.cc:822
#14 0x0000000000757294 in tdc_remove_table (thd=0x1fd7f00, remove_type=TDC_RT_REMOVE_ALL, db=0x2189638 "test", table_name=0x209a9e8 "t6")
at sql_base.cc:7738
#15 0x00000000008779ef in mysql_rm_table_part2 (thd=0x1fd7f00, tables=0x20621f8, if_exists=false, drop_temporary=false, drop_view=false,
dont_log_query=false) at sql_table.cc:1626
#16 0x0000000000878657 in mysql_rm_table (thd=0x1fd7f00, tables=0x20621f8, if_exists=0 '\0', drop_temporary=0 '\0') at sql_table.cc:1515
#17 0x0000000000703449 in mysql_execute_command (thd=0x1fd7f00) at sql_parse.cc:3491
#18 0x0000000000708154 in mysql_parse (thd=0x1fd7f00, inBuf=0x2196bf8 "drop table t6,t7", length=16, found_semicolon=0x41f13b40)
at sql_parse.cc:5999
If I remove the HA_DO_INDEX_COND_PUSHDOWN flags from ha_innodb.h (replace it by 0 in the expression), no crash, and result is then identical to MySQL 5.1.
[9 Jun 2009 15:04]
Guilhem Bichot
And without changing code, but by adding set engine_condition_pushdown=off; to the testcase, bug goes away too. So I'm putting this bug in the list of BUG#45029
[25 Jun 2009 13:51]
Guilhem Bichot
de-assigning from me; I was not supposed to fix it if it's in the field of InnoDB+ICP.
[7 Oct 2009 12:14]
Olav Sandstå
This crash is no longer reproducible with the current 6.0 code base after the patch for Bug#45029 has been pushed. This patch ( http://lists.mysql.com/commits/76684 ) disables MRR and turns of engine/index condition pushdown for InnoDB. By reverting the disabling of engine/index condition pushdown for InnoDB in the file storage/innobase/handler/ha_innodb.h the server crash is still reproducible using the test case.
[8 Oct 2009 10:55]
Olav Sandstå
When this crash occurs, glibc reports the following memory violation: *** glibc detected *** /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld: double free or corruption (!prev): 0x09152b60 *** ======= Backtrace: ========= /lib/tls/i686/cmov/libc.so.6[0xb7dc2454] /lib/tls/i686/cmov/libc.so.6(cfree+0x96)[0xb7dc44b6] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(row_prebuilt_free+0x129)[0x8458629] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_ZN11ha_innobase5closeEv+0x5b)[0x83e2559] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z8closefrmP5TABLEb+0x72)[0x82566bb] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z18intern_close_tableP5TABLE+0x9f)[0x824a0d4] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld[0x824a196] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z16tdc_remove_tableP3THD26enum_tdc_remove_table_typePKcS3_+0x199)[0x824a350] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z20mysql_rm_table_part2P3THDP10TABLE_LISTbbbb+0x361)[0x8326249] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z14mysql_rm_tableP3THDP10TABLE_LISTcc+0xd9)[0x8326fac] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z21mysql_execute_commandP3THD+0x33e5)[0x82081ac] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x19c)[0x820da56] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x856)[0x820e9d7] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(_Z10do_commandP3THD+0x240)[0x820fe21] /home/olav/mysql/develop/opt-bug43360-icp/sql/mysqld(handle_one_connection+0x304)[0x81ff22e] /lib/tls/i686/cmov/libpthread.so.0[0xb802b50f] /lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0xb7e34a0e]
[8 Oct 2009 12:49]
Guilhem Bichot
updating tags per Olav's findings
[8 Oct 2009 19:45]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/86245 3649 Olav.Sandstaa@sun.com 2009-10-08 Bug#43360 - Server crash with a simple multi-table update Add a test case for this bug to the optimizer_unfixed_bugs suite. Note that this test passes for the current version of the 6.0 codebase. In order to reproduce this bug it is neccessary to re-enable InnoDB's support for index condition pushdown in the file storage/innobase/handler/ha_innodb.h. @ mysql-test/suite/optimizer_unfixed_bugs/r/bug43360.result Result file for test case for Bug#43360 - Server crash with a simple multi-table update @ mysql-test/suite/optimizer_unfixed_bugs/t/bug43360.test Test case for Bug#43360 - Server crash with a simple multi-table update
[9 Oct 2009 11:20]
Olav Sandstå
The previous patch containing the test case for the optimizer_unfixed_bugs suite is now pushed into mysql-6.0-codebase-bugfixing.
[11 Oct 2009 20:27]
Olav Sandstå
By enabling the UNIV_DEBUG define in InnoDB and adding an extra assert to the _checkchunk method in safemalloc.c to get it to crash on the first issue found by safemalloc the crash now occurs earlier with the following call stack when running the test case for this bug:
#14 <signal handler called>
#15 0xb7fe7430 in __kernel_vsyscall ()
#16 0xb7cf48a0 in raise () from /lib/tls/i686/cmov/libc.so.6
#17 0xb7cf6268 in abort () from /lib/tls/i686/cmov/libc.so.6
#18 0xb7ced72e in __assert_fail () from /lib/tls/i686/cmov/libc.so.6
#19 0x087a8b95 in _checkchunk (irem=0xb03eeb0, filename=0x8a09090 "array.c",
lineno=305) at safemalloc.c:492
#20 0x087a8c17 in _sanity (filename=0x8a09090 "array.c", lineno=305)
at safemalloc.c:529
#21 0x087a83a3 in _myfree (ptr=0xb009b90, filename=0x8a09090 "array.c",
lineno=305, myflags=16) at safemalloc.c:277
#22 0x087bba55 in delete_dynamic (array=0xb0094c0) at array.c:305
#23 0x0844af28 in ~QUICK_RANGE_SELECT (this=0xb009490) at opt_range.cc:1255
#24 0x0844cce5 in SQL_SELECT::cleanup (this=0xb111860) at opt_range.cc:1130
#25 0x0844cd53 in ~SQL_SELECT (this=0xb111860) at opt_range.cc:1144
#26 0x08397ce2 in st_join_table::cleanup (this=0xb00b6a8)
at sql_select.cc:10481
#27 0x08397ead in JOIN::cleanup (this=0xb1038c0, full=true)
at sql_select.cc:10641
#28 0x083980b4 in JOIN::join_free (this=0xb1038c0) at sql_select.cc:10564
#29 0x08398767 in do_select (join=0xb1038c0, fields=0xb2408578, table=0x0,
procedure=0x0) at sql_select.cc:15863
#30 0x083b0dcb in JOIN::exec (this=0xb1038c0) at sql_select.cc:2896
#31 0x083ab4c7 in mysql_select (thd=0xb05f3f0, rref_pointer_array=0xb060954,
tables=0xb009ce8, wild_num=0, fields=@0xb2408578, conds=0xb00a908,
og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0,
select_options=1342177408, result=0xb00aa20, unit=0xb060398,
select_lex=0xb060850) at sql_select.cc:3087
#32 0x083d0dad in mysql_multi_update (thd=0xb05f3f0, table_list=0xb009ce8,
fields=0xb0608e4, values=0xb060b18, conds=0xb00a908, options=0,
handle_duplicates=DUP_ERROR, ignore=false, unit=0xb060398,
select_lex=0xb060850, result=0xb2408b70) at sql_update.cc:1246
#33 0x0831031a in mysql_execute_command (thd=0xb05f3f0) at sql_parse.cc:3196
#34 0x083168b3 in mysql_parse (thd=0xb05f3f0,
inBuf=0xb009c00 "UPDATE t1,t2 SET t1.b=UPPER(t1.b) WHERE t1.b LIKE 'United%'", length=59, found_semicolon=0xb24099c8) at sql_parse.cc:5991
#35 0x08317489 in dispatch_command (command=COM_QUERY, thd=0xb05f3f0,
packet=0xb00fce9 "UPDATE t1,t2 SET t1.b=UPPER(t1.b) WHERE t1.b LIKE 'United%'", packet_length=59) at sql_parse.cc:1074
#36 0x08318a1e in do_command (thd=0xb05f3f0) at sql_parse.cc:756
#37 0x08304800 in handle_one_connection (arg=0xb05f3f0) at sql_connect.cc:1164
#38 0xb7fa050f in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#39 0xb7daaa0e in clone () from /lib/tls/i686/cmov/libc.so.6
[15 Oct 2009 12:27]
Olav Sandstå
The memory corruption seems to happen in the function build_template() in storage/innobase/handler/ha_innodb.cc where we write to beyond the end of the mysql_template array of the prebuild structure.
When this function get called the mysql_template array is allocated with the size specified by n_fields (if is not already allocated). In the test case of this bug n_fields gets the value 2.
Later in the same function we fill the mysql_template array in a loop:
/*
Ok, now build an array of mysql_row_templ_struct structures.
If index condition pushdown is used, the array is split into two
parts: first go index fields, then go table fields.
Note that in InnoDB, i is the column number. MySQL calls columns
'fields'.
*/
for (i = 0; i < n_fields; i++) {
templ = prebuilt->mysql_template + n_requested_fields;
When index condition pushdown is not enabled this loops n_fields times (which is the same as the size of the array). Not memory corruption occurs.
But if index condition pushdown is enabled we repeat the "looping" twice by resetting the loop variable (i) when the first "looping" is completed. With the test case of this bug report this means that we fill data into 4 mysql_template elements instead of the two allocated and thus writes beyond the end of the allocated memory for the array. See also the comment in the code fragment above about index condition push down.
[31 Oct 2009 8:19]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091031081410-qkxmjsdzjmj840aq) (version source revid:guilhem@mysql.com-20091010123356-qfouds7ucvkvbczt) (merge vers: 6.0.14-alpha) (pib:13)
[25 Nov 2009 13:29]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/91611 3730 Olav Sandstaa 2009-11-25 Fix for Bug#43360 Server crash with a simple multi-table update (InnoDB table) This crash was due to a memory corruption occuring in the function build_template (in ha_innodb.cc) when Index condition pushdown (ICP) is used. With ICP the prebuilt.mysql_template array needs to store information about both index and table fields but the allocated size of this array was equal to the number of table fields. Thus, with ICP we wrote beyond the end of the array. This patch fixes this problem by allocating two times the number of fields for this array (which is the number of times the for loop that fills the array is run when ICP is used). A regression test based on a simplified version of the bug's test case is included. This will be run both for InnoDB and MyISAM. Note that due ICP being currently disabled for InnoDB this test will only be run for debug builds of InnoDB. @ mysql-test/include/icp_tests.inc Storage independent version of a simplified version the test case for Bug#43360. @ mysql-test/r/innodb_icp.result Result file for ICP Bug#43360 when run against InnoDB. @ mysql-test/r/myisam_icp.result Result file for ICP Bug#43360 when run against MyISAM. @ mysql-test/suite/optimizer_unfixed_bugs/r/bug43360.result This test case has now been included in include/icp_tests.inc @ mysql-test/suite/optimizer_unfixed_bugs/t/bug43360.test This test case has now been included in include/icp_tests.inc @ mysql-test/t/innodb_icp.test Test case for ICP code for InnoDB. Note that currently this test will only run for debug builds due to ICP being disabled for InnoDB in release builds. @ mysql-test/t/myisam_icp.test Test case for ICP code in MyISAM @ storage/innobase/handler/ha_innodb.cc Increase the size of prebuilt.mysql_template to be able to contain both information about index and table fields by allocating two times the number of fields entries.
[10 Dec 2009 12:37]
Olav Sandstå
A summary of the response given to Jørgen's review email where he asked a question about the consequence on memory usage by doubling the size of the mysql_template array: ====================================================================== The mysql_template array is part of the handler object. So each connection will have one handler object/mysql_template array per table it has opened against InnoDB. This handler object will be re-used for all queries on this connection (if I understand correctly the handler object does not get closed between queries). They are not shared between connections. I have looked a bit on how much doubling the size of the mysql_template could contribute to the memory usage. Here are some of the results: * Each entry in the mysql_template array contains a row_prebuilt_struct. The size of the row_prebuilt_struct is 104 bytes. * With the existing implementation the size of the mysql_template array will be the same as the number of fields/columns in the table. Eg. with a table of 5 columns approximately 0.5 KB of memory will be allocated. With my initial propsed fix we double the size of the mysql_template array, e.g, with at table of 5 columns now the memory requirement becomes about 1 KB. * The mysql_template array is stored in the handler object. So for each table a session/client has open there exists a handler object and a mysql_template object for this table. Example: a session having e.g. five open tables (each with five columns) will increase the memory allocation from 2.5 KB to 5 KB for storing these. * Large example (and here I am not sure about the details): a server with 1000 clients where each client has accessed 10 tables: -memory per client/session: up from 5 KB to 10 KB -total: memory up from 5 MB to 10 MB (I am not sure if we have any code that is closing handler objects if we have too many open? - and reopens when needed. So this example might be wrong.) Some statistics from having run the mysql test suite (note that I do not claim this to a relevant workload, still interesting). This test is run with ICP enabled and MRR disabled): * There are in total 2859 allocations of the mysql_template * 2853 of these (99.8%) of the allocations are done for a non-ICP query (i.e. no need for a larger initial array) * 6 of these are for an ICP query * 24 times it is necessary to increase the size of the mysql_template due to it a later query on the same handler object using ICP. * The mysql_template array is used about 36000 times, so for every allocation of it it is re-used about 12 times. (if these numbers are correct although not representative, it seems like ICP is fairly little used when just enabling support for ICP). If we want to extend the patch for this bug to only allocate the double size mysql_template array in order to save memory when it is actuall needed this too will be fairly easy ( but more code than just do a 2X as the current patch does): -we need to extend the row_prebuilt_struct with a field for storing the current size of the mysql_template array. -for every call to build_template we need to check if we need to re-size the mysql_template array. -add support for dynamically grow the array when needed (de-allocate the old one and alloacate a new)
[14 Dec 2009 12:35]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/93902 3770 Olav Sandstaa 2009-12-14 Fix for Bug#43360 Server crash with a simple multi-table update (InnoDB table) This crash was due to a memory corruption occurring in the function build_template (in ha_innodb.cc) when Index condition pushdown (ICP) is used. With ICP the prebuilt.mysql_template array needs to store information about both index and table fields but the allocated size of this array was equal to the number of table fields. Thus, with ICP we wrote beyond the end of the array. This patch fixes this problem by determining the required size of the mysql_template array based on whether ICP is in use or not and then re-allocate the mysql_template array if the current size is too small. A regression test based on a simplified version of the bug's test case is included. This will be run both for InnoDB and MyISAM. Note that due ICP being currently disabled for InnoDB this test will only be run for debug builds of InnoDB. @ mysql-test/include/icp_tests.inc Storage independent version of a simplified version the test case for Bug#43360. @ mysql-test/r/innodb_icp.result Result file for ICP Bug#43360 when run against InnoDB. @ mysql-test/r/myisam_icp.result Result file for ICP Bug#43360 when run against MyISAM. @ mysql-test/suite/optimizer_unfixed_bugs/r/bug43360.result This test case has now been included in include/icp_tests.inc @ mysql-test/suite/optimizer_unfixed_bugs/t/bug43360.test This test case has now been included in include/icp_tests.inc @ mysql-test/t/innodb_icp.test Test case for ICP code for InnoDB. Note that currently this test will only run for debug builds due to ICP being disabled for InnoDB in release builds. @ mysql-test/t/myisam_icp.test Test case for ICP code in MyISAM @ storage/innobase/handler/ha_innodb.cc On every call to build_template compute the max size of the mysql_template array based on whether ICP is used or not. If this size is larger than the currently allocated size then re-allocate the mysql_template array. @ storage/innobase/include/row0mysql.h Add a member to the row_prebuilt_struct to store the size of the allocated mysql_template array. @ storage/innobase/row/row0mysql.c Initialize the new mysql_template_size member variable.
[17 Dec 2009 14:48]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/94770 3778 Olav Sandstaa 2009-12-17 Fix for Bug#43360 Server crash with a simple multi-table update (InnoDB table) This crash was due to a memory corruption occurring in the function build_template (in ha_innodb.cc) when Index condition pushdown (ICP) is used. With ICP the prebuilt.mysql_template array needs to store information about both index and table fields but the allocated size of this array was equal to the number of table fields. Thus, with ICP we wrote beyond the end of the array. This patch fixes this problem by determining the required size of the mysql_template array based on whether ICP is in use or not and then re-allocate the mysql_template array if the current size is too small. A regression test based on a simplified version of the bug's test case is included. This will be run both for InnoDB and MyISAM. Note that due to ICP being currently disabled for InnoDB this test will only be run for debug builds of InnoDB. @ mysql-test/include/icp_tests.inc Storage independent version of a simplified version the test case for Bug#43360. @ mysql-test/r/innodb_icp.result Result file for ICP Bug#43360 when run against InnoDB. @ mysql-test/r/myisam_icp.result Result file for ICP Bug#43360 when run against MyISAM. @ mysql-test/suite/optimizer_unfixed_bugs/r/bug43360.result This test case has now been included in include/icp_tests.inc @ mysql-test/suite/optimizer_unfixed_bugs/t/bug43360.test This test case has now been included in include/icp_tests.inc @ mysql-test/t/innodb_icp.test Test case for ICP code for InnoDB. Note that currently this test will only run for debug builds due to ICP being disabled for InnoDB in release builds. @ mysql-test/t/myisam_icp.test Test case for ICP code in MyISAM. @ storage/innobase/handler/ha_innodb.cc On every call to build_template compute the max size of the mysql_template array based on whether ICP is used or not. If this size is larger than the currently allocated size then re-allocate the mysql_template array. @ storage/innobase/include/row0mysql.h Add a member to the row_prebuilt_struct to store the size of the allocated mysql_template array. @ storage/innobase/row/row0mysql.c Initialize the new mysql_template_size member variable.
[26 Feb 2010 8:49]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/101556 2969 Olav Sandstaa 2010-02-26 Testcase for Bug#43360 Server crash with a simple multi-table update (InnoDB table). The code for fixing the actual bug is included in the patch for Bug#36981. @ mysql-test/include/icp_tests.inc Testcase for Bug#43360 Server crash with a simple multi-table update (InnoDB table). @ mysql-test/r/innodb_icp.result Result file for test for Bug#43360. @ mysql-test/r/myisam_icp.result Result file for test for Bug#43360. @ mysql-test/suite/optimizer_unfixed_bugs/r/bug43360.result Test is now included in include/icp_tests.inc. @ mysql-test/suite/optimizer_unfixed_bugs/t/bug43360.test Test is now included in include/icp_tests.inc.
[26 Feb 2010 9:42]
Olav Sandstå
Patch pushed to mysql-6.0-codebase-bugfixing with the revision-id: olav@sun.com-20100226084919-5jw199gfsmoolcer .
[6 Mar 2010 10:29]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100306102742-yw9zzgw9ac5r65m5) (version source revid:bar@mysql.com-20100305074327-h09o5lw290s04lcf) (merge vers: 6.0.14-alpha) (pib:16)
[14 Mar 2010 1:07]
Paul DuBois
Changes to test case. No changelog entry needed.
[7 May 2010 11:49]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/107735 3147 Olav Sandstaa 2010-05-07 Testcase for Bug#43360 Server crash with a simple multi-table update (InnoDB table). (Backporting of revid:olav@sun.com-20100226084919-5jw199gfsmoolcer) The code for fixing the actual bug is included in the patch for Bug#36981. @ mysql-test/include/icp_tests.inc Testcase for Bug#43360 Server crash with a simple multi-table update (InnoDB table). @ mysql-test/r/innodb_icp.result Result file for test for Bug#43360. @ mysql-test/r/myisam_icp.result Result file for test for Bug#43360.
[16 Aug 2010 6:41]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100816062819-bluwgdq8q4xysmlg) (version source revid:alik@sun.com-20100816062612-enatdwnv809iw3s9) (pib:20)
[13 Nov 2010 16:20]
Bugs System
Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (merge vers: 5.6.99-m4) (pib:21)
Description: The following test scirpt: <code> CREATE TABLE t6 (a char(2) not null primary key, b varchar(20) not null, key (b)); CREATE TABLE t7 (a char(2) not null primary key, b varchar(20) not nu on CHAR, VARCHARll, key (b)); INSERT INTO t6 values ('AB','MySQLAB'),('JA','Sun Microsystems'),('MS','Microsoft'),('IB','IBM- Inc.'),('GO','Google Inc.'); INSERT INTO t7 values ('AB','Sweden'),('JA','USA'),('MS','United States of Amercica'),('IB','North America'),('GO','South America'); update t6,t7 set t6.a=LCASE(t6.a); select * from t6; update t6,t7 set t6.a=UCASE(t6.a) where t6.a='AB'; select * from t6; update t6,t7 set t6.b=UPPER(t6.b) where t6.b LIKE 'United%'; select * from t7; update t6,t7 set t6.b=UPPER(t6.b),t7.b=LOWER(t7.b) where LENGTH(t6.b) between 3 and 5 and t7.a=LOWER(t6.a); select * from t6; select * from t7; drop table t6,t7; </code> causes a server crash with InnoDB against a 'compile-pentium-debug-max' build of MySQL 6.0.9-alpha. Maria, Falcon, PBXT do not report a server crash How to repeat: Build MySQL 6.0.9 with 'BUILD/compile-pentium-debug-max' and run the above test via MTR.