| Bug #43765 | falcon_bug_22173a crashes in IndexPage::findNodeInLeaf | ||
|---|---|---|---|
| Submitted: | 20 Mar 2009 9:51 | Modified: | 15 May 2009 15:51 |
| Reporter: | Hakan Küçükyılmaz | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Falcon storage engine | Severity: | S1 (Critical) |
| Version: | 6.0-falcon-team | OS: | Linux |
| Assigned to: | Vladislav Vaintroub | CPU Architecture: | Any |
| Tags: | F_INDEX, pushbuild, test failure | ||
[20 Mar 2009 9:52]
Hakan Küçükyılmaz
Full stack trace
Attachment: stack.trace (application/octet-stream, text), 28.48 KiB.
[20 Mar 2009 15:30]
Hakan Küçükyılmaz
Verified by Pushbuild.
[23 Mar 2009 12:28]
Kevin Lewis
See also the duplicate Bug#43800
[23 Mar 2009 22:14]
John Embretsen
For what it is worth, last weekend I ran this test 3000 times on two different Solaris hosts (with no/little background load and plenty of memory) without any failures.
[1 Apr 2009 21:30]
Kevin Lewis
See also Duplicate Bug#43954
[9 Apr 2009 7:07]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/71722 3110 Vladislav Vaintroub 2009-04-09 Bug #43765 falcon_bug_22173a crashes in IndexPage::findNodeInLeaf Falcon crashes with assertion (level==0)in high-concurrent index updates Reason is that recent code refactoring in IndexPage code introduced a regression. When index is small and consists of a single root page only, findInsertionLeaf(), would check lock the root page with shared lock, check if the root page level is 0 (page is a leaf), unlock the page, lock it again excusively and return this page. The problem is that during the gap while page was not locked, it could have beens spli. In this case returning root page is wrong, as it is not a "insertion leaf", and generally not leaf page at all. The fix is not to return from the function prematurely, when reacquiring lock in the root and root page changes the level from 0 to greater number.
[9 Apr 2009 13:46]
Kevin Lewis
Code looks correct now OK to push.
[15 Apr 2009 17:00]
Bugs System
Pushed into 6.0.11-alpha (revid:hky@sun.com-20090415164923-9arx29ye5pzxd4zf) (version source revid:vvaintroub@mysql.com-20090409070645-katx7bwydr9y6ftt) (merge vers: 6.0.11-alpha) (pib:6)
[15 May 2009 15:51]
MC Brown
An entry has been added to the 6.0.11 changelog: When performing a high number of concurrent index updates on a Falcon table, mysqld could crash due to an assertion.
Description: falcon_bug_22173a fails 1 out of 100 times How to repeat: Run falcon_bug_22173a several times. I tried it running 112 times falcon.falcon_bug_22173a [ fail ] Test ended at 2009-03-20 10:09:07 CURRENT_TEST: falcon.falcon_bug_22173a mysqltest: At line 56: query 'call p1()' failed: 2013: Lost connection to MySQL server during query The result from queries just before the failure was: < snip > declare continue handler for 1020 begin end; declare continue handler for 1213 begin end; declare continue handler for 1015 begin end; while v1 < 2500 do /* SELECT 'insert', v1; */ INSERT INTO t1 VALUES (v1); SET v2 = rand() * 10000; UPDATE t1 SET a = v2 WHERE a = v1; /* SELECT ' update', v1; */ SET v1 = v1 + 1; if v1 mod 50 = 0 then /* SELECT ' delete'; */ /* DELETE FROM t1; */ TRUNCATE t1; end if; end while; end// # Sent call p1() to the server but do not pull the results call p1(); # Switch to connection conn1 More results from queries before failure can be found in /home/hakan/work/mysql/mysql-6.0-falcon-team/mysql-test/var/log/falcon_bug_22173a.log - saving '/home/hakan/work/mysql/mysql-6.0-falcon-team/mysql-test/var/log/falcon.falcon_bug_22173a/' to '/home/hakan/work/mysql/mysql-6.0-falcon-team/mysql-test/var/log/falcon.falcon_bug_22173a/' - found 'core.26578' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace Core generated by '/home/hakan/work/mysql/mysql-6.0-falcon-team/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- warning: Can't read pathname for load map: Input/output error. Core was generated by `/home/hakan/work/mysql/mysql-6.0-falcon-team/sql/mysqld --defaults-group-suffix'. Program terminated with signal 6, Aborted. [New process 26590] [New process 32572] [New process 32571] [New process 26594] [New process 26593] [New process 26592] [New process 26591] [New process 26578] [New process 26589] [New process 26588] [New process 26587] [New process 26586] [New process 26585] [New process 26584] [New process 26583] [New process 26582] [New process 26581] [New process 26580] #0 0x00007fb5e72e5b16 in pthread_kill () from /lib/libpthread.so.0 #0 0x00007fb5e72e5b16 in pthread_kill () from /lib/libpthread.so.0 #1 0x0000000000683a4a in handle_segfault (sig=6) at mysqld.cc:2689 #2 <signal handler called> #3 0x00007fb5e72e866b in raise () from /lib/libpthread.so.0 #4 0x00000000008e2af8 in Error::error (string=<value optimized out>) at Error.cpp:94 #5 0x0000000000969fe4 in IndexPage::findNodeInLeaf (this=0x7fb5e54f1000, indexKey=0x7fb5dabf2bc0, foundKey=0x0) at IndexPage.cpp:199 #6 0x00000000008f110a in IndexRootPage::addIndexEntry (dbb=0x7fb5e594c790, indexId=1, key=0x7fb5dabf2bc0, recordNumber=408, transId=0) at IndexRootPage.cpp:138 #7 0x00000000008f1c80 in IndexRootPage::indexMerge (dbb=0x7fb5e594c790, indexId=1, logRecord=0x7fb5dabf5d88, transId=0) at IndexRootPage.cpp:887 #8 0x000000000094a193 in SRLUpdateIndex::commit (this=0x7fb5dabf5d88) at SRLUpdateIndex.cpp:195 #9 0x00000000009318a1 in SerialLogTransaction::commit (this=0x4024e68) at SerialLogTransaction.cpp:92 #10 0x0000000000964d75 in Gopher::gopherThread (this=0x7fb5e558bb50) at Gopher.cpp:71 #11 0x00000000008a80f3 in Thread::thread (this=0x7fb5e5595668) at Thread.cpp:166 #12 0x00000000008a82c1 in Thread::thread (parameter=0x67d2) at Thread.cpp:145 #13 0x00007fb5e72e0faa in start_thread () from /lib/libpthread.so.0 #14 0x00007fb5e625b2bd in clone () from /lib/libc.so.6 #15 0x0000000000000000 in ?? ()