Bug #42743 Falcon fails to recover;Test tablespace file is not open when doing a fetchPage
Submitted: 10 Feb 2009 23:02 Modified: 15 May 2009 17:01
Reporter: Kevin Lewis Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S3 (Non-critical)
Version:mysql-6.0-falcon-team OS:Any
Assigned to: Vladislav Vaintroub CPU Architecture:Any
Tags: F_RECOVERY

[10 Feb 2009 23:02] Kevin Lewis
Description:
I keep killing the Falcon engine with the debugger in the middle of various transactions.  A few times, I got an unrecoverable database.  I will attach an example.  The test database is very small.  It always fails in IO::pread() on this line;

HANDLE hFile = (HANDLE)_get_osfhandle(fileId);

because the fileId is -1.  

Here is the call stack;
 	mysqld.exe!my_sigabrt_handler(int sig=22)  Line 497	C
 	mysqld.exe!raise(int signum=22)  Line 590 + 0x7 bytes	C
 	mysqld.exe!__crtMessageWindowW(int nRptType=2, const wchar_t * szFile=0x00e54a08, const wchar_t * szLine=0x00136734, const wchar_t * szModule=0x00000000, const wchar_t * szUserMessage=0x00134734)  Line 430 + 0x7 bytes	C++
 	mysqld.exe!_VCrtDbgReportW(int nRptType=2, const wchar_t * szFile=0x00e54a08, int nLine=314, const wchar_t * szModule=0x00000000, const wchar_t * szFormat=0x00e54840, char * arglist=0x0013e7d4)  Line 670 + 0x28 bytes	C
 	mysqld.exe!_CrtDbgReportWV(int nRptType=2, const wchar_t * szFile=0x00e54a08, int nLine=314, const wchar_t * szModule=0x00000000, const wchar_t * szFormat=0x00e54840, char * arglist=0x0013e7d4)  Line 300 + 0x1d bytes	C++
 	mysqld.exe!_CrtDbgReportW(int nRptType=2, const wchar_t * szFile=0x00e54a08, int nLine=314, const wchar_t * szModule=0x00000000, const wchar_t * szFormat=0x00e54840, ...)  Line 317 + 0x1d bytes	C++
 	mysqld.exe!_get_osfhandle(int fh=-1)  Line 314 + 0x45 bytes	C
>	mysqld.exe!IO::pread(__int64 offset=16384, int length=16384, unsigned char * buffer=0x14218000)  Line 585 + 0xc bytes	C++
 	mysqld.exe!IO::readPage(Bdb * bdb=0x04784334)  Line 287 + 0x1e bytes	C++
 	mysqld.exe!Cache::fetchPage(Dbb * dbb=0x04069880, int pageNumber=1, PageType pageType=PAGE_inventory, LockType lockType=Shared)  Line 300	C++
 	mysqld.exe!Dbb::fetchPage(int pageNumber=1, PageType pageType=PAGE_inventory, LockType lockType=Shared)  Line 220	C++
 	mysqld.exe!PageInventoryPage::isPageInUse(Dbb * dbb=0x04069880, int pageNumber=3)  Line 188 + 0x10 bytes	C++
 	mysqld.exe!Cache::fetchPage(Dbb * dbb=0x04069880, int pageNumber=3, PageType pageType=PAGE_any, LockType lockType=Exclusive)  Line 236 + 0x1e bytes	C++
 	mysqld.exe!Dbb::fetchPage(int pageNumber=3, PageType pageType=PAGE_any, LockType lockType=Exclusive)  Line 220	C++
 	mysqld.exe!Section::getSectionPage(Dbb * dbb=0x04069880, int root=3, int sequence=0, LockType requestedLockType=Exclusive, unsigned int transId=0)  Line 205 + 0x12 bytes	C++
 	mysqld.exe!IndexRootPage::setIndexRoot(Dbb * dbb=0x04069880, int indexId=0, int pageNumber=0, unsigned int transId=0)  Line 909 + 0x15 bytes	C++
 	mysqld.exe!IndexRootPage::redoIndexDelete(Dbb * dbb=0x04069880, int indexId=0)  Line 923 + 0x11 bytes	C++
 	mysqld.exe!SRLDeleteIndex::redo()  Line 93 + 0x10 bytes	C++
 	mysqld.exe!SerialLog::recover()  Line 397 + 0x15 bytes	C++
 	mysqld.exe!Database::openDatabase(const char * filename=0x0013f460)  Line 753 + 0xe bytes	C++
 	mysqld.exe!Connection::getDatabase(const char * dbName=0x0415025c, const char * dbFileName=0x0013f460, Threads * threads=0x041502b8)  Line 1651	C++
 	mysqld.exe!Connection::openDatabase(const char * dbName=0x0415025c, const char * filename=0x0415028c, const char * account=0x00e1e824, const char * password=0x00e1e81c, const char * address=0x00000000, Threads * parent=0x041502b8)  Line 933 + 0x17 bytes	C++
 	mysqld.exe!StorageDatabase::getOpenConnection()  Line 137	C++
 	mysqld.exe!StorageHandler::initialize()  Line 987 + 0xe bytes	C++
 	mysqld.exe!StorageInterface::falcon_init(void * p=0x029a6eb0)  Line 257 + 0xb bytes	C++
 	mysqld.exe!ha_initialize_handlerton(st_plugin_int * plugin=0x017ab480)  Line 440 + 0x11 bytes	C++
 	mysqld.exe!plugin_initialize(st_plugin_int * plugin=0x017ab480)  Line 1006 + 0x17 bytes	C++
 	mysqld.exe!plugin_init(int * argc=0x010a944c, char * * argv=0x003b9bc0, int flags=0)  Line 1213 + 0x9 bytes	C++
 	mysqld.exe!init_server_components()  Line 4114 + 0x30 bytes	C++
 	mysqld.exe!win_main(int argc=2, char * * argv=0x003b3768)  Line 4574 + 0x5 bytes	C++
 	mysqld.exe!mysql_service(void * p=0x00000000)  Line 4737 + 0x12 bytes	C++
 	mysqld.exe!main(int argc=2, char * * argv=0x003b3768)  Line 4910 + 0x7 bytes	C++
 	mysqld.exe!__tmainCRTStartup()  Line 327 + 0x19 bytes	C
 	mysqld.exe!mainCRTStartup()  Line 196	C
 	kernel32.dll!7c817067() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]	

How to repeat:
Recover the attached database.

Suggested fix:
Make sure the tablespace is opened before trying to read from it
[10 Feb 2009 23:03] Kevin Lewis
Verified by attached database.
[10 Feb 2009 23:04] Kevin Lewis
Replace the master-data directory with this and start Falcon

Attachment: master-data.zip (application/zip, text), 106.96 KiB.

[10 Feb 2009 23:30] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/65827

3013 Vladislav Vaintroub	2009-02-11
      Bug#42743 : Falcon fails to recover;Test tablespace file is not open when doing a fetchPage
      
      The tablespace was not open when read was attempted.
      This is a regression from previous behavior, introduced during great tablespace recovery cleanup.
      (TableSpaceManager::bootstrap() does not open files).
      
      Fix reverts the change, and all tablespaces that can l be opened, will be opened during bootstrap.
[3 Apr 2009 9:06] Hakan Küçükyılmaz
Fix is in 6.0.11
[15 May 2009 17:01] MC Brown
A note has been added to the 6.0.11 changelog: 

Recovery of Falcon tables while there were active transaction during the crash may fail to recover completely