Bug #44114 Falcon recovery Error: page 5342/1 wrong page type, expected 5 got 511
Submitted: 6 Apr 2009 13:15 Modified: 15 May 2009 15:52
Reporter: Philip Stoev Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S1 (Critical)
Version:6.0-falcon-team OS:Any
Assigned to: Vladislav Vaintroub CPU Architecture:Any
Triage: Triaged: D1 (Critical)

[6 Apr 2009 13:15] Philip Stoev
Description:
Since all bugs for "wrong page type, expected 5" are now closed, and this error continues to happen, I am filing a new bug.

# 12:12:07 [Falcon] Error: page 5342/1 wrong page type, expected 5 got 511

mysqld(Error::debugBreak()+0xe) [0x9f1b24]
mysqld(Error::error(char const*, ...)+0x131) [0x9f1c57]
mysqld(Cache::fetchPage(Dbb*, int, PageType, LockType)+0x4cc) [0x9c0298]
mysqld(Dbb::fetchPage(int, PageType, LockType)+0x4c) [0x9e70d4]
mysqld(Dbb::handoffPage(Bdb*, int, PageType, LockType)+0x2c) [0x9e72fc]
mysqld(IndexRootPage::findInsertionLeaf(Dbb*, int, IndexKey*, int, unsigned int, bool*)+0x246) [0xa02$
mysqld(IndexRootPage::indexMerge(Dbb*, int, SRLUpdateIndex*, unsigned int)+0x16f) [0xa03323]
mysqld(SRLUpdateIndex::execute()+0x9f) [0xa75d5d]
mysqld(SRLUpdateIndex::redo()+0x15) [0xa75d77]
mysqld(SerialLog::recover()+0xd06) [0xa56f74]
mysqld(Database::openDatabase(char const*)+0x407) [0x9e068b]
mysqld(Connection::getDatabase(char const*, char const*, Threads*)+0x16a) [0x9cec84]
mysqld(Connection::openDatabase(char const*, char const*, char const*, char const*, char const*, Thre$
mysqld(StorageDatabase::getOpenConnection()+0x89) [0x985b7d]
mysqld(StorageHandler::initialize()+0xa3) [0x988c0f]
mysqld(StorageInterface::falcon_init(void*)+0x2ac) [0x97ef94]
mysqld(ha_initialize_handlerton(st_plugin_int*)+0xae) [0x820ea2]
mysqld [0x8d89e0]
mysqld(plugin_init(int*, char**, int)+0x673) [0x8dc1d9]
mysqld [0x6ca7f7]
mysqld(main+0x1f0) [0x6cb2f5]

How to repeat:
The tablespace will be uploaded shortly. It was produced with the following command:

$ perl runall.pl    --engine=Falcon   --reporters=Deadlock,ErrorLog,Backtrace,Recovery,Shutdown   --mysqld=--loose-falcon-lock-wait-timeout=1   --mysqld=--loose-innodb-lock-wait-timeout=1   --mysqld=--log-output=none   --mysqld=--skip-safemalloc   --mysqld=--transaction-isolation=READ-COMMITTED --mysqld=--falcon-page-size=32K  --rows=10000 --threads=4 --mysqld=--falcon-use-deferred-index-hash=1   --mask=57022   --queries=100000000   --duration=900   --basedir=/build/bzr/6.0-falcon-team  --gendata=conf/combinations.zz  --grammar=conf/combinations.yy  --vardir=/ide/april05/simple/current  

Note, --mysqld=--falcon-use-deferred-index-hash=1 which may or may not be important.
[6 Apr 2009 13:29] Philip Stoev
Tablespace before recovery

http://mysql-systemqa.s3.amazonaws.com/bug44114-2.zip
[6 Apr 2009 13:42] Philip Stoev
This failure comes in the following 4 forms:

[Falcon] Error: page 569/1 wrong page type, expected 5 got 2596
[Falcon] Error: page 1906/1 wrong page type, expected 5 got 7
[Falcon] Error: page 1104/1 wrong page type, expected 5 got 9
[Falcon] Error: page 21/1 wrong page type, expected 5 got 511

and is very widespread in PB2 with dozens of occurrences, including with default Falcon startup options.
[7 Apr 2009 17:44] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/71568

3105 Vladislav Vaintroub	2009-04-07
      Bug #44114 Falcon recovery Error: page 5342/1 wrong page type, expected 5 got 511
      
      This patch makes sure pages written are valid (at least they have valid page types). 
      Falcon will assert if garbage data is written to the tablespaces. This will make errors like "got page type 511" impossible. The actual error that
      causes page corruption  will be simpler to find.
[7 Apr 2009 18:28] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/71569

3105 Vladislav Vaintroub	2009-04-07
      Bug #44114 Falcon recovery Error: page 5342/1 wrong page type, expected 5 got 511
      
      This patch makes sure pages written are valid (at least they have valid page types and page numbers) 
      Falcon will assert if garbage data is written to the tablespaces. This will make errors like "got page type 511" impossible. The actual error that
      causes page corruption  will be simpler to find.
[7 Apr 2009 22:31] Ann Harrison
Vlad's patch will be definitive, but I suspect that the problem is
not that we're writing bad pages, but that we sometimes read a page
number that's no longer valid by reading beyond the maxLine on a
RecordLocator or Section page.
[15 Apr 2009 16:59] Bugs System
Pushed into 6.0.11-alpha (revid:hky@sun.com-20090415164923-9arx29ye5pzxd4zf) (version source revid:vvaintroub@mysql.com-20090407182323-qohwrke048hvp8hn) (merge vers: 6.0.11-alpha) (pib:6)
[15 May 2009 15:52] MC Brown
An entry has been added to the 6.0.11 changelog: 

Recovery of Falcon tables could fail with an indicating that a wrong page type was identified in the Falcon serial lo