MySQL Bugs: #36426: Assertion in SerialLogControl::nextRecord line 313 on live downgrade

Bug #36426	Assertion in SerialLogControl::nextRecord line 313 on live downgrade
Submitted:	30 Apr 2008 13:51	Modified:	5 Nov 2008 22:56
Reporter:	Philip Stoev	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S1 (Critical)
Version:	6.0.4	OS:	Any
Assigned to:	Vladislav Vaintroub	CPU Architecture:	Any
Tags:	F_MISCELLANY

Description:
When doing a live downgrade from 6.0.5 to 6.0.4, 6.0.4 crashes after the downgrade with the following stack trace. At least several situations where the crash will happen are known, so I would assume that most upgrades of that type would fail in real life.

Live downgrade means stop the new server and start the old server using the same vardir, without dumping and reloading the database.

#0  0x00110402 in __kernel_vsyscall ()
#1  0x00bdc617 in pthread_kill () from /lib/libpthread.so.0
#2  0x0839c9a8 in write_core (sig=32070) at stacktrace.c:240
#3  0x082609ff in handle_segfault (sig=4) at mysqld.cc:2313
#4  <signal handler called>
#5  0x00110402 in __kernel_vsyscall ()
#6  0x00bdf891 in raise () from /lib/libpthread.so.0
#7  0x0845c04b in Error::debugBreak () at Error.cpp:92
#8  0x0845c09a in Error::error (string=0x87c0ff4 "assertion failed at line %d in file %s\n") at Error.cpp:69
#9  0x0845c0f2 in Error::assertionFailed (fileName=0x87c92c8 "SerialLogControl.cpp", line=313) at Error.cpp:76
#10 0x0849f707 in SerialLogControl::nextRecord (this=0xb70cd068) at SerialLogControl.cpp:313
#11 0x0849c3cf in SerialLog::recover (this=0xb7535d38) at SerialLog.cpp:321
#12 0x08451473 in Database::openDatabase (this=0xb73e5628, filename=0xb70cd65c "/build/updown_vardir/master/master-data/falcon_master.fts")
    at Database.cpp:726
#13 0x08449a58 in Connection::getDatabase (this=0x0, dbName=0xb71e44e4 "FALCON_MASTER",
    dbFileName=0xb70cd65c "/build/updown_vardir/master/master-data/falcon_master.fts", threads=0xb71e4540) at Connection.cpp:1653
#14 0x08449be3 in Connection::openDatabase (this=0xb71e4738, dbName=0xb71e44e4 "FALCON_MASTER", filename=0xb71e4514 "falcon_master.fts",
    account=0x8730785 "mysql", password=0x8730785 "mysql", address=0x0, parent=0xb71e4540) at Connection.cpp:931
#15 0x08423f77 in StorageDatabase::getOpenConnection (this=0xb71e43f0) at StorageDatabase.cpp:133
#16 0x08425f34 in StorageHandler::initialize (this=0xb73e5028) at StorageHandler.cpp:912
#17 0x084263a7 in StorageHandler::getStorageConnection (this=0xb73e5028, tableShare=0xb71e4128, mySqlThread=0x9f729b8, mySqlThdId=1, createFlag=OpenDatabase)
    at StorageHandler.cpp:644
#18 0x0841a827 in StorageInterface::open (this=0x9ff7f98, name=0x9f6cff0 "./trigdb/t2", mode=2, test_if_locked=2) at ha_falcon.cpp:446
#19 0x08340523 in handler::ha_open (this=0x9ff7f98, table_arg=0x9ff74f8, name=0x9f6cff0 "./trigdb/t2", mode=2, test_if_locked=2) at handler.cc:1587
#20 0x082b2d7e in open_table_from_share (thd=0x9f729b8, share=0x9f6cd78, alias=0x9ff24f8 "t2", db_stat=39, prgflag=44, ha_open_flags=0, outparam=0x9ff74f8,
    open_mode=OTM_OPEN) at table.cc:1950
#21 0x082a91f1 in open_unireg_entry (thd=0x9f729b8, entry=0x9ff74f8, table_list=0x9ff2500, alias=0x9ff24f8 "t2", cache_key=0xb70ce0d6 "trigdb",
    cache_key_length=10, mem_root=0xb70ce53c, flags=0) at sql_base.cc:3821
#22 0x082aae91 in open_table (thd=0x9f729b8, table_list=0x9ff2500, mem_root=0xb70ce53c, refresh=0xb70ce57b, flags=<value optimized out>) at sql_base.cc:2941
#23 0x082aba12 in open_tables (thd=0x9f729b8, start=0xb70ce5e4, counter=0xb70ce5c4, flags=0) at sql_base.cc:4440
#24 0x082abf13 in open_and_lock_tables_derived (thd=0x9f729b8, tables=0x9ff2500, derived=true) at sql_base.cc:4825
#25 0x082e29e0 in mysql_insert (thd=0x9f729b8, table_list=0x9ff2500, fields=@0x9f73cb0, values_list=@0x9f73cd4, update_fields=@0x9f73cc8,
    update_values=@0x9f73cbc, duplic=DUP_ERROR, ignore=false) at mysql_priv.h:1476
#26 0x0826feca in mysql_execute_command (thd=0x9f729b8) at sql_parse.cc:2658
#27 0x0827473c in mysql_parse (thd=0x9f729b8, inBuf=0x9ff2438 "INSERT INTO t2 VALUES (44,'2-new-forth')", length=40, found_semicolon=0xb70cf2f4)
    at sql_parse.cc:5410
#28 0x08274f71 in dispatch_command (command=COM_QUERY, thd=0x9f729b8, packet=0x9fe3599 "INSERT INTO t2 VALUES (44,'2-new-forth')", packet_length=40)
    at sql_parse.cc:921
#29 0x08275d79 in do_command (thd=0x9f729b8) at sql_parse.cc:697
#30 0x0826846d in handle_one_connection (arg=0x9f72a30) at sql_connect.cc:1146
#31 0x00bd750b in start_thread () from /lib/libpthread.so.0
#32 0x00b18b2e in clone () from /lib/libc.so.6

How to repeat:
It appears that a few queries are all that is required to reproduce the crash. If needed, a test case will be uploaded.

Suggested fix:
If this does not warrant further investigation and a fix, then it should at least be noted in the Release Notes. Hence this bug.

Docs, The history of Falcon this last two years has included many changes to the serial log and a few to the file structure.  We want to be sensitive to the ease in which one can move data back and forth between versions.  But upgrading is generally more important than downgrading.

The safe way to do either is always to export the data from on version, install, and import the data into the new installation.  This is especially true for downgrades.  There are many asserts in the code which assume things to be a certain way on disk.  The code cannot know when one of them will change in the future.  

We need to fix this bug by not allowing the engine to attempt to use a serial log that is newer than what the engine can support.  In the mean time, please document for the current release that Falcon does not support 'live downgrades', which is an older engine with newer tablespaces.  

Then please put this bug back into the Open state.

A note has been added to the limits section of the manual to highlight the issue and provide the mysqldump/mysql workaround.