| Bug #38130 | Falcon assertion in IndexNode::expandKey offset + length <= MAX_PHYSICAL_KEY_LEN | ||
|---|---|---|---|
| Submitted: | 15 Jul 2008 10:35 | Modified: | 15 May 2009 16:17 |
| Reporter: | Philip Stoev | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Falcon storage engine | Severity: | S1 (Critical) |
| Version: | 6.0-falcon | OS: | Any |
| Assigned to: | Lars-Erik Bjørk | CPU Architecture: | Any |
| Tags: | F_LIMIT | ||
| Triage: | Triaged: D1 (Critical) | ||
[15 Jul 2008 14:05]
Philip Stoev
Grammar file for bug 38130
Attachment: bug38130.yy (application/octet-stream, text), 1016 bytes.
[15 Jul 2008 14:08]
Philip Stoev
I just uploaded a gramma file for this bug. To reproduce, please clone the mysql-test-extra-6.0 tree and execute $ cd mysql-test-extra-6.0/mysql-test/gentest $ perl runall.pl \ --basedir=/path/to/mysql-6.0-falcon \ --grammar=/location/of/bug38130.yy \ --engine=falcon \ --mysqld=--falcon_lock_wait_timeout=1 \ --queries=100000 On a dual-core virtualized 1.4ghz it takes 12 min at 10 threads to crash, 141460 queries, 2602 rollbacks, 2558 commits.
[3 Jan 2009 11:37]
Philip Stoev
Still present in 6.0-falcon-team. To reproduce within 5 min, execute: $ perl runall.pl \ --mysqld=--falcon-page-size=4K \ --vardir=/tmp/vardir \ --rows=10000 \ --threads=4 \ --mask=58 \ --queries=1000000 \ --duration=300 \ --basedir=/build/bzr/6.0-falcon-team \ --engine=Falcon \ --grammar=conf/combinations.yy \ --gendata=conf/combinations.zz \ --reporter=Backtrace \ --mysqld=--loose-falcon-lock-wait-timeout=1 \ --mysqld=--log-output=none
[27 Feb 2009 9:39]
Lars-Erik Bjørk
This bug is closely related to bug#42341 The difference seems to be that the garbage, read behind the END_BUCKET node, is not caught in the first consistency check, but in a later check. This crash also disappears when applying the change suggested in the report for bug#42341
[2 Mar 2009 8:17]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/67956 3043 lars-erik.bjork@sun.com 2009-03-02 This is a patch for bug#42341 Falcon assertion (key - (UCHAR*) indexNode < 14) in IndexNode::parseNode and bug#38130 Falcon assertion in IndexNode::expandKey offset + length <= MAX_PHYSICAL_KEY_LENGTH These crashes happen because we are trying to use the data behind the last node in the bucket, when we are walking the index. The reason for this is that the node with the special record number -1 (which indicates END_BUCKET) is the only node in the page. WalkIndex::getNextNode has the following piece of code: int32 WalkIndex::getNextNode(void) { for (;; first = true) { if (first) { first = false; recordNumber = node.getNumber(); if (recordNumber >= 0) return recordNumber; else if (recordNumber == END_LEVEL) return -1; } node.getNext(endNodes); We fail to check if recordNumber == END_BUCKET. In the case of bug#42341, we try to parse some garbage data in IndexNode::parseNode and assert on a consistency check. In the case of bug#38130, we slip through this consistency check, but assert on a second check in IndexNode::expandKey Changing the if from else if (recordNumber == END_LEVEL) to else if (recordNumber == END_LEVEL || recordNumber == END_BUCKET) prevents both crashes. modified file 'storage/falcon/WalkIndex.cpp' ----------------------------------------------- Changed the if to prevent reading behind the END_BUCKET node.
[2 Mar 2009 14:39]
Kevin Lewis
Patch approved
[2 Apr 2009 17:38]
Bugs System
Pushed into 6.0.11-alpha (revid:hky@sun.com-20090402144811-yc5kp8g0rjnhz7vy) (version source revid:christopher.powers@sun.com-20090304040340-b4zoglfws0iswqm1) (merge vers: 6.0.11-alpha) (pib:6)
[15 May 2009 16:17]
MC Brown
Internal/test fix. No changelog entry required.
Description: While running a workload containing INSERT/DELETE/UPDATE/SELECT and LIMIT, Falcon asserted as follows: #0 0x00110416 in __kernel_vsyscall () #1 0x00581c78 in pthread_kill () from /lib/libpthread.so.0 #2 0x087abf37 in my_write_core (sig=6) at stacktrace.c:307 #3 0x082a9947 in handle_segfault (sig=6) at mysqld.cc:2638 #4 <signal handler called> #5 0x00110416 in __kernel_vsyscall () #6 0x00584fe0 in raise () from /lib/libpthread.so.0 #7 0x0856bd20 in Error::debugBreak () at Error.cpp:94 #8 0x0856bd9d in Error::error (string=0x894ec98 "assertion (%s) failed at line %d in file %s\n") at Error.cpp:71 #9 0x0856be47 in Error::assertionFailed (text=0x8950794 "offset + length <= MAX_PHYSICAL_KEY_LENGTH", fileName=0x8950768 "IndexNode.h", line=148) at Error.cpp:78 #10 0x0857ff4f in IndexNode::expandKey (this=0xb7570b60, indexKey=0xb7570b78) at IndexNode.h:148 #11 0x085fb8af in WalkIndex::getNextNode (this=0xb756b688) at WalkIndex.cpp:101 #12 0x085fb92b in WalkIndex::getNext (this=0xb756b688, lockForUpdate=false) at WalkIndex.cpp:65 #13 0x0850b386 in StorageDatabase::nextIndexed (this=0xb71ca130, storageTable=0xb501fb80, indexWalker=0xb756b688, lockForUpdate=false) at StorageDatabase.cpp:481 #14 0x08511ce8 in StorageTable::nextIndexed (this=0xb501fb80, recordNumber=0, lockForUpdate=false) at StorageTable.cpp:180 #15 0x08504b16 in StorageInterface::index_next (this=0xa6703298, buf=0xa6703460 "") at ha_falcon.cpp:1601 #16 0x08503743 in StorageInterface::read_range_first (this=0xa6703298, start_key=0xa670332c, end_key=0x0, eq_range_arg=false, sorted=true) at ha_falcon.cpp:1569 #17 0x083cb10e in handler::multi_range_read_next (this=0xa6703298, range_info=0xa7faec90) at handler.cc:4262 #18 0x083aba03 in QUICK_RANGE_SELECT::get_next (this=0xa940d30) at opt_range.cc:8518 #19 0x083c53ab in rr_quick (info=0xa8cbd6c) at records.cc:298 #20 0x0831ca57 in join_init_read_record (tab=0xa8cbd28) at sql_select.cc:14481 #21 0x08320699 in sub_select (join=0xa8ca938, join_tab=0xa8cbd28, end_of_records=false) at sql_select.cc:13636 #22 0x0832d37f in do_select (join=0xa8ca938, fields=0xaacf498, table=0x0, procedure=0x0) at sql_select.cc:13387 #23 0x0833fa62 in JOIN::exec (this=0xa8ca938) at sql_select.cc:2811 #24 0x0833af1b in mysql_select (thd=0xaace000, rref_pointer_array=0xaacf508, tables=0xa9fd6a0, wild_num=0, fields=@0xaacf498, conds=0xa9fe528, og_num=1, order=0xa9fe6c0, group=0x0, having=0x0, proc_param=0x0, select_options=2149337600, result=0xa9fe758, unit=0xaacf16c, select_lex=0xaacf404) at sql_select.cc:3001 #25 0x0833fd72 in handle_select (thd=0xaace000, lex=0xaacf110, result=0xa9fe758, setup_tables_done_option=0) at sql_select.cc:302 #26 0x082b8778 in execute_sqlcom_select (thd=0xaace000, all_tables=0xa9fd6a0) at sql_parse.cc:4847 #27 0x082b9f90 in mysql_execute_command (thd=0xaace000) at sql_parse.cc:2015 #28 0x082c2fc6 in mysql_parse (thd=0xaace000, inBuf=0xa9fd238 "SELECT X . int_key FROM D AS X LEFT JOIN A AS Y ON ( X . int_key = Y . int_key ) WHERE X . int_key > 180 ORDER BY X . int_key LIMIT 1", length=133, found_semicolon=0xa7fb0270) at sql_parse.cc:5811 #29 0x082c3a3a in dispatch_command (command=COM_QUERY, thd=0xaace000, packet=0xaa98081 "", packet_length=133) at sql_parse.cc:1051 #30 0x082c4d04 in do_command (thd=0xaace000) at sql_parse.cc:724 #31 0x082b29d3 in handle_one_connection (arg=0xaace000) at sql_connect.cc:1153 #32 0x0057d32f in start_thread () from /lib/libpthread.so.0 #33 0x0049a27e in clone () from /lib/libc.so.6 The assertion is here: 146 inline void IndexNode::expandKey(IndexKey *indexKey) 147 { 148 ASSERT (offset + length <= MAX_PHYSICAL_KEY_LENGTH); <<<< HERE 149 150 if (length) 151 memcpy (indexKey->key + offset, key, length); 152 (gdb) print offset $3 = 0 (gdb) print length $4 = 15874 (gdb) print MAX_PHYSICAL_KEY_LENGTH $5 = 10809 How to repeat: If this happens again, a test case will be provided.