MySQL Bugs: #26330: falcon crashes in Table::fetchNext

Bug #26330	falcon crashes in Table::fetchNext
Submitted:	13 Feb 2007 16:11	Modified:	17 Sep 2007 17:02
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S3 (Non-critical)
Version:	5.2.3-falcon-alpha-debug	OS:	Linux (suse9.3 x86)
Assigned to:		CPU Architecture:	Any
Tags:	assertion, crash

Description:
when running testcase attached, the follow crash occurred:

the testcase simply does some insert/select/update to a table
with blob and text fields.

Version: '5.2.3-falcon-alpha-debug'  socket: '/tmp/mysql.sock'  port: 3306  yes
database open failed: can't open file "/home/sbester/server/5.2/mysql-5.2.3-falcon-alpha-linux-i686/data/test.fts": No such file or directory (2)
mysqld got signal 4;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388572
read_buffer_size=131072
max_used_connections=16
max_connections=151
threads_connected=16
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 336762 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x8e00880
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x4286069c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x81f5637 handle_segfault + 417
0xffffe410 _end + -142426112
0x83a9308 _ZN5Error10debugBreakEv + 18
0x83a927f _ZN5Error5errorEPKcz + 87
0x83a92ee _ZN5Error15assertionFailedEPKci + 32
0x837f103 _ZN5Table9fetchNextEi + 487
0x8376dca _ZN15StorageDatabase4nextEP12StorageTablei + 54
0x8379ceb _ZN12StorageTable4nextEi + 31
0x8370abb _ZN15NfsStorageTable8rnd_nextEPc + 125
0x82d0333 _Z13rr_sequentialP14st_read_record + 135
0x8261af7 _Z10sub_selectP4JOINP13st_join_tableb + 221
0x826162d _Z9do_selectP4JOINP4ListI4ItemEP8st_tableP9Procedure + 797
0x8250da6 _ZN4JOIN4execEv + 6980
0x825127f _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_sel + 637
0x824c4f9 _Z13handle_selectP3THDP6st_lexP13select_resultm + 329
0x8217115 _Z21execute_sqlcom_selectP3THDP13st_table_list + 759
0x820f9fe _Z21mysql_execute_commandP3THD + 1480
0x8218a91 _Z11mysql_parseP3THDPcj + 353
0x820df51 _Z16dispatch_command19enum_server_commandP3THDPcj + 2077
0x820d729 _Z10do_commandP3THD + 603
0x820c816 handle_one_connection + 900
0x40250aa7 _end + 933750423
0x401e6c2e _end + 933316638
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do 
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8ecbc10 = select c1 from t1 where c2='unifokidwzlnkdsjuawuiyxzpjgsqyterogbbbwmfdotictstosuebffniqorogtihjjtysjpshubsjgufkoyqgareapkkgcnvbwhkrxlcaalqnzajhmsappngfuixshpmbfydjjryrcuizxnevhkmulxywkvgpbzuwyyctorezriffrvewmgscalrypdojhwrnyglfaciqfgntuuywxzkbwsvcbsoiqszkeofikijjtotrqwa'
thd->thread_id=11
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

How to repeat:
build and run attached testcase.
crash occurred after about 10 - 15 minutes.  if you have problems,
increase/decrease NUMTHREADS and TESTTIME...

Suggested fix:
don't crash

see header of file for compile instructions + host,user,database,port

Attachment: testcase.c (text/plain), 7.03 KiB.

Verified as described.

Backtrace is:
Program received signal SIGILL, Illegal instruction.
[Switching to Thread 1127549872 (LWP 13112)]
0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0  0xffffe410 in __kernel_vsyscall ()
#1  0x40182541 in raise () from /lib/tls/libc.so.6
#2  0x08457822 in Error::debugBreak () at Error.cpp:93
#3  0x0845787f in Error::error (
    string=0x8764598 "assertion failed at line %d in file %s\n")
    at Error.cpp:70
#4  0x0845790f in Error::assertionFailed (fileName=0x875d358 "Table.cpp",
    line=481) at Error.cpp:77
#5  0x084207ab in Table::fetchNext (this=0x405247d8, start=11159)
    at Table.cpp:481
#6  0x08411653 in StorageDatabase::next (this=0x404fa5d4,
    storageTable=0x417bf034, recordNumber=11159) at StorageDatabase.cpp:243
#7  0x08415b33 in StorageTable::next (this=0x417bf034, recordNumber=11159)
    at StorageTable.cpp:110
#8  0x0840f444 in NfsStorageTable::rnd_next (this=0x8a0e6b0,
    buf=0x8a0e818 "\006\201 ") at ha_falcon.cpp:421
#9  0x0834f625 in rr_sequential (info=0x89ee908) at records.cc:362
#10 0x082cd0ce in sub_select (join=0x89ed2c8, join_tab=0x89ee8c8,
    end_of_records=false) at sql_select.cc:10543
#11 0x082cd429 in do_select (join=0x89ed2c8, fields=0x89e50c8, table=0x0,
    procedure=0x0) at sql_select.cc:10301
#12 0x082e0de0 in JOIN::exec (this=0x89ed2c8) at sql_select.cc:1909
#13 0x082e1137 in mysql_select (thd=0x89e4d58, rref_pointer_array=0x89e5158,
    tables=0x89ecd50, wild_num=0, fields=@0x89e50c8, conds=0x89ed170,
    og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0,
    select_options=2147764736, result=0x89ed2b8, unit=0x89e4e00,
    select_lex=0x89e503c) at sql_select.cc:2074
#14 0x082e1411 in handle_select (thd=0x89e4d58, lex=0x89e4d98,
    result=0x89ed2b8, setup_tables_done_option=0) at sql_select.cc:256
#15 0x08276a72 in execute_sqlcom_select (thd=0x89e4d58, all_tables=0x89ecd50)
    at sql_parse.cc:5344
#16 0x0827c483 in mysql_execute_command (thd=0x89e4d58) at sql_parse.cc:2699
#17 0x0828411c in mysql_parse (thd=0x89e4d58,
    inBuf=0x89ecb78 "select c1 from t1 where c2='oltszrfpbvpmqqcinjwvpzqvbtjalvlzhzsneusbiaovlanwhyhmobhjqpkxpygacapulgcjscflqtbtizfwmeolqxzfshkwlysyovnhvimylodqartwoepufwkxwwuw'", length=157) at sql_parse.cc:6162
#18 0x08284b27 in dispatch_command (command=COM_QUERY, thd=0x89e4d58,
    packet=0x8c629d1 "", packet_length=158) at sql_parse.cc:1857
#19 0x08285b8e in do_command (thd=0x89e4d58) at sql_parse.cc:1626
#20 0x0828603f in handle_one_connection (arg=0x89e4d58) at sql_parse.cc:1232
#21 0x40284297 in start_thread () from /lib/tls/libpthread.so.0
#22 0x4021937e in clone () from /lib/tls/libc.so.6
#23 0x43350bb0 in ?? ()

Regards, Hakan

The "test case" does random operations in multiple threads, making it virtually useless for either reproducing a bug or demonstrating that the bug has been fixed.

If you'd like a fix in the foreseeable future, please try to make a deterministic test case.

put back into analyzing and re-assigning to myself to make a better testcase.

Shane,

please consider to get a "semi" random test case for this bug. For instance you can look into my heavily modified version of BUG#26324.

Best regards,

Hakan

Does not crash anymore. Tested with latest Falcon code. Test case ran full 5600 seconds without crash or hang on my Linux laptop.

Best regards,

Hakan