MySQL Bugs: #41837: Falcon recovery error: page 102/0 wrong page type, expected 7 got 0

Bug #41837	Falcon recovery error: page 102/0 wrong page type, expected 7 got 0
Submitted:	3 Jan 2009 12:34	Modified:	15 May 2009 15:59
Reporter:	Philip Stoev	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S1 (Critical)
Version:	6.0-falcon-team	OS:	Any
Assigned to:	Vladislav Vaintroub	CPU Architecture:	Any
Tags:	F_RECOVERY, Tablespace

Description:
Recovery from the attached tablespace, produced after a crash, is not possible. Falcon prints the following:

[Falcon] Error: page 102/0 wrong page type, expected 7 got 0

090103 14:32:00 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338364 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = (nil) thread_stack 0x40000
./mysqld(my_print_stacktrace+0x2e) [0xaba2ce]
./mysqld(handle_segfault+0x32a) [0x6563aa]
/lib64/libpthread.so.0 [0x315b00f0f0]
/lib64/libpthread.so.0(raise+0x2b) [0x315b00efab]
./mysqld(Error::error(char const*, ...)+0xe8) [0x8a6d88]
./mysqld(Cache::fetchPage(Dbb*, int, PageType, LockType)+0x10a) [0x883dea]
./mysqld(Dbb::handoffPage(Bdb*, int, PageType, LockType)+0x2b) [0x89c9bb]
./mysqld(Section::findNextRecord(int, int, Stream*)+0x1b0) [0x8e61b0]
./mysqld(Section::findNextRecord(int, int, Stream*)+0xd5) [0x8e60d5]
./mysqld(TableSpaceManager::bootstrap(int)+0x20b) [0x869cab]
./mysqld(Database::openDatabase(char const*)+0x421) [0x8974c1]
./mysqld(Connection::getDatabase(char const*, char const*, Threads*)+0xe3) [0x88b463]
./mysqld(Connection::openDatabase(char const*, char const*, char const*, char const*, char const*, Threads*)+0xa4) [0x88cab4]
./mysqld(StorageDatabase::getOpenConnection()+0x77) [0x850e27]
./mysqld(StorageHandler::initialize()+0x96) [0x853a06]
./mysqld(StorageInterface::falcon_init(void*)+0x186) [0x848c76]
./mysqld(ha_initialize_handlerton(st_plugin_int*)+0x3f) [0x7535af]
./mysqld [0x7daa02]
./mysqld(plugin_init(int*, char**, int)+0x824) [0x7dd974]
./mysqld [0x657f79]
./mysqld(main+0x198) [0x65b008]
/lib64/libc.so.6(__libc_start_main+0xe6) [0x315a41e546]
./mysqld [0x584379]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
[Falcon] Error: assertion (lockState == 0) failed at line 132 in file SyncObject.cpp

Aborted

How to repeat:
The tablespace will be uploaded shortly.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/66783

3026 Vladislav Vaintroub	2009-02-18
      Bug #42745 Exception: can't find table space during recovery
      Bug #41837 Falcon recovery error: page 102/0 wrong page type, 
      expected 7 got 0 
      
      
      Problem: Falcon internal table system.tablespaces can be inconsistent 
      when mysqld has crashed or was killed.
      
      This table (possibly inconsistent) was previously read on Falcon startup
      in TableSpaceManager::bootstrap(), to provide recovery with enough 
      information to associate tablespace IDs in log records with actual 
      tablespace files.
      
      But since the table is possibly outdated information, it was still possible
      to have lost tablespaceid without any correspond file. Recovery will stop 
      then.
      
      Another problem with TableSpace::bootstrap() is attempt to read behind the 
      end of falcon_master.fts. Typically it would be a page present referenced 
      in a section page, and referenced page was not yet flushed to disk at the 
      moment of crash(Bug#41837)
      
      Solution:
      Avoid reading system.tablespaces from disk whenever possible. Instead, 
      when a new tablespace is added or some tablespace is deleted, current state
      (listof all existing tablespaces) is written to serial log in 
      SRLTableSpaces record. If recovery finds SRLTableSpaces during the first 
      pass, last SRLTableSpaces record is used to reconstruct the before-crash 
      state.
      
      If recovery does not find SRLTableSpaces, it will still read system.table
      spaces with TableSpaceManager::bootstrap() as before. However reading from
      disk is safe, because tablespaces were not modified since the last
      checkpoint.
      added:
        storage/falcon/SRLTableSpaces.cpp
        storage/falcon/SRLTableSpaces.h
      modified:
        storage/falcon/CMakeLists.txt
        storage/falcon/Database.cpp
        storage/falcon/Makefile.am
        storage/falcon/SRLVersion.h
        storage/falcon/SerialLog.cpp
        storage/falcon/SerialLogControl.cpp
        storage/falcon/SerialLogControl.h
        storage/falcon/SerialLogRecord.h
        storage/falcon/TableSpaceManager.cpp
        storage/falcon/TableSpaceManager.h

Code reviewed by Kevin Lewis and Jim Starkey

Pushed into 6.0.11-alpha (revid:alik@sun.com-20090302140208-lfdejjbcyezlhhjt) (version source revid:vvaintroub@mysql.com-20090218174140-unfccjescdawur5g) (merge vers: 6.0.10-alpha) (pib:6)

An entry has been added to the 6.0.11 changelog: 

Recovery of Falcon tables could fail with an indicating that a wrong page type was identified in the Falcon serial log.