Bug #33930 Falcon crash on rollback + crash on recovery
Submitted: 18 Jan 2008 18:55 Modified: 4 Feb 2008 20:46
Reporter: Philip Stoev Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S1 (Critical)
Version:6.0.4-p3 OS:Any
Assigned to: Jim Starkey CPU Architecture:Any

[18 Jan 2008 18:55] Philip Stoev
Description:
About an hour into the iuds2 test, Falcon crashed with the following stack trace:

#0  0x00002b6d6f2424c5 in pthread_kill () from /lib64/libpthread.so.0
#1  0x000000000064621c in handle_segfault ()
#2  <signal handler called>
#3  0x00002b6d6f244fed in raise () from /lib64/libpthread.so.0
#4  0x000000000080ec1c in Error::error ()
#5  0x000000000086c38f in SerialLogControl::nextRecord ()
#6  0x000000000086df79 in SerialLogTransaction::rollback ()
#7  0x0000000000892447 in Gopher::gopherThread ()
#8  0x00000000007e06f0 in Thread::thread ()
#9  0x00000000007e0589 in Thread::thread ()
#10 0x00002b6d6f23e193 in start_thread () from /lib64/libpthread.so.0
#11 0x00002b6d6fb9c45d in clone () from /lib64/libc.so.6
#12 0x0000000000000000 in ?? ()

I am sorry that this is a non-debug binary, I will try to reproduce with a debug one.

Recovery then proceeds as follows:

Recovering database /data1/6.0.4/systest_vardir-suse/master-data/falcon_master.fts ...
first recovery block is 198858
last recovery block is 371504
recovery read block is 200786
Bugcheck: page 17530 wrong page type, expected 2 got 7
080118 18:53:03 - mysqld got signal 4;

and crashes with the following stack trace:

#0  0x00002b76a8d144c5 in pthread_kill () from /lib64/libpthread.so.0
#1  0x000000000064621c in handle_segfault ()
#2  <signal handler called>
#3  0x00002b76a8d16fed in raise () from /lib64/libpthread.so.0
#4  0x000000000080ec1c in Error::error ()
#5  0x000000000088c0ea in Cache::fetchPage ()
#6  0x0000000000807969 in Dbb::handoffPage ()
#7  0x0000000000861432 in Section::getSectionPage ()
#8  0x000000000086395a in Section::getSectionPage ()
#9  0x000000000086354b in Section::redoRecordLocatorPage ()
#10 0x0000000000866c60 in SerialLog::recover ()
#11 0x00000000007fe3cc in Database::openDatabase ()
#12 0x00000000007fac78 in Connection::getDatabase ()
#13 0x00000000007f8334 in Connection::openDatabase ()
#14 0x00000000007ca911 in StorageDatabase::getOpenConnection ()
#15 0x00000000007cecb9 in StorageHandler::initialize ()
#16 0x00000000007ce5cd in StorageHandler::getStorageConnection ()
#17 0x00000000007c034c in StorageInterface::open ()
#18 0x0000000000712c2a in handler::ha_open ()
#19 0x000000000068bd91 in open_table_from_share ()
#20 0x0000000000688c8c in open_unireg_entry ()
#21 0x0000000000683212 in open_table ()
#22 0x00000000006845ae in open_tables ()
#23 0x000000000068498f in open_normal_and_derived_tables ()
#24 0x000000000073a519 in get_all_tables ()
#25 0x00000000007422c1 in get_schema_tables_result ()
#26 0x0000000000697e59 in JOIN::exec ()
#27 0x0000000000698105 in mysql_select ()
#28 0x000000000069352c in handle_select ()
#29 0x000000000065a2b9 in execute_sqlcom_select ()
#30 0x00000000006519b0 in mysql_execute_command ()
#31 0x0000000000657382 in mysql_parse ()
#32 0x000000000065054a in dispatch_command ()
#33 0x00000000006501bf in do_command ()
#34 0x000000000064e9ec in handle_one_connection ()
#35 0x00002b76a8d10193 in start_thread () from /lib64/libpthread.so.0
#36 0x00002b76a966e45d in clone () from /lib64/libc.so.6
#37 0x0000000000000000 in ?? ()

How to repeat:
A location to download the tablespace after the first crash but before recovery will be provided shortly.
[22 Jan 2008 21:10] Kevin Lewis
This looks like a damaged serial log in which a recovery would also not be possible.  Jim, if you agree, we can ask for this crash to be repeated with a debug build.
[4 Feb 2008 20:46] Kevin Lewis
Jim looked at this recovery problem and determined that the database was not right before the recovery started.  It could be a 'double recovery' issue which was fixed (33608).