MySQL Bugs: #40114: sporadic falcon crash in main suite on embedded in 6.0-rpl

Bug #40114	sporadic falcon crash in main suite on embedded in 6.0-rpl
Submitted:	17 Oct 2008 15:58	Modified:	11 Nov 2008 17:45
Reporter:	Sven Sandberg	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S2 (Serious)
Version:	6.0-rpl	OS:	Any
Assigned to:	Assigned Account	CPU Architecture:	Any
Tags:	6.0-rpl-green, crash, F_PAGE IO, pushbuild, sporadic, test failure

Description:
Pushbuild failure happened a few times during august 2008:

main.ctype_latin2_ch                     [ fail ]

CURRENT_TEST: main.ctype_latin2_ch
[Falcon] Error: page 112/0 wrong page type, expected 7 got 2

mysqltest got signal 6
read_command_buf at 0x12f0e20 = 
Attempting backtrace...
stack_bottom = (nil) thread_stack 0x40000
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(my_print_stacktrace+0x32)[0x650e78]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded[0x5f4d57]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded[0x5f4d83]
/lib64/libc.so.6[0x2ba3fcb56b20]
/lib64/libc.so.6(gsignal+0x35)[0x2ba3fcb56aa5]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN5Error10debugBreakEv+0xe)[0xa87e44]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN5Error5errorEPKcz+0x12f)[0xa87f75]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN5Cache9fetchPageEP3Dbbi8PageType8LockType+0x405)[0xb74f89]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN3Dbb9fetchPageEi8PageType8LockType+0x4c)[0xb972c6]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN7Section11storeRecordEP17RecordLocatorPageiP11RecordIndexP6Streamjb+0x124)[0xbee8d4]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN7Section12updateRecordEiP6Streamjb+0x481)[0xbf09c1]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN3Dbb12updateRecordEP7SectioniP6StreamP11Transactionb+0x5b)[0xb96fdd]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN3Dbb10updateBlobEP7SectioniP6StreamP11Transaction+0xb4)[0xb970d0]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN5Table9getBlobIdEP5ValueibP11Transaction+0x301)[0xaa9895]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN13EncodedRecord15encodeAsciiBlobEP5Value+0x36)[0xcd2c9e]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN17EncodedDataStream6encodeEiP5Value+0x11da)[0xba162a]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN6Record8finalizeEP11Transaction+0x136)[0xbd8b52]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN5Table6insertEP11TransactioniPPN3Nfs5FieldEPP5Value+0x2a5)[0xab0d39]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN8NReplace9doReplaceEPN3Nfs9StatementEP9ResultSet+0x22b)[0xbc8ce9]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN8NReplace13evalStatementEPN3Nfs9StatementE+0x94)[0xbc91f4]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN3Nfs9Statement5startEP5NNode+0xf2)[0xc20c20]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN17PreparedStatement13executeUpdateEv+0x90)[0xbd3818]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN10TableSpace4saveEv+0x13d)[0xab63b1]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN17TableSpaceManager10initializeEv+0x94)[0xab41e8]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN8Database12openDatabaseEPKc+0x7ab)[0xb91503]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN10Connection11getDatabaseEPKcS1_P7Threads+0x16a)[0xb82372]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN10Connection12openDatabaseEPKcS1_S1_S1_S1_P7Threads+0x19c)[0xb8392e]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN15StorageDatabase17getOpenConnectionEv+0x89)[0xa993d9]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN14StorageHandler10initializeEv+0x9a)[0xa9bde8]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_ZN16StorageInterface11falcon_initEPv+0x21f)[0xa911ff]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_Z24ha_initialize_handlertonP13st_plugin_int+0xa0)[0x967ac4]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded[0x6eb5c0]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(_Z11plugin_initPiPPci+0x60c)[0x6ec9d6]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded[0x61c7f5]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(init_embedded_server+0x12c)[0x61d15e]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(mysql_server_init+0x127)[0x60dc3b]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(main+0x32a)[0x6023fc]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x2ba3fcb44154]
/data0/pushbuild/pb/bzr_mysql-6.0-rpl/53/bzr_mysql-6.0-rpl-exp/libmysqld/examples/mysqltest_embedded(__gxx_personality_v0+0xb9)[0x5f2189]
[Falcon] Error: page 112/0 wrong page type, expected 7 got 2

 - saving '/dev/shm/var-embedded-131/1/log/main.ctype_latin2_ch/' to '/dev/shm/var-embedded-131/log/main.ctype_latin2_ch/'

Retrying test, attempt(2/3)...

 

The failure seems to have stopped now, but that could be random. There is a similar failure that still happens, reported in BUG#40109. I suggest that the person who fixes BUG#40109 checks this bug too.

How to repeat:
E.g., here: https://intranet.mysql.com/secure/pushbuild/showpush.pl?dir=bzr_mysql-6.0-rpl&order=53 sapsrv1/embedded
xref: http://tinyurl.com/6zlmb7

Forgot to mention that this happens all over the main suite. It's not specific to one test.

Hmmm, the failures in that xref actually look a bit different. Some of them contain the line:

[Falcon] Error: assertion (lockState == 0) failed at line 131 in file SyncObject.cpp

Others contain the lines:

[Falcon] Error: can't continue after fatal error
[Falcon] Error: read error on page 85 of "/data0/pushbuild/pb/bzr_mysql-6.0-rpl/43/mysql-6.0.7-alpha-pb43/mysql-test/falcon_master.fts": Success (0) 

And yet others contain the following line repeated before and after the stack trace:

[Falcon] Error: page 112/0 wrong page type, expected 7 got 2

Ann,  This might be associated with the PIP investigation you are currently doing.  The error here is "wrong page type, expected 7 got 2"

The failure with message

[Falcon] Error: assertion (lockState == 0) failed at line 131 in file SyncObject.cpp

is reported as BUG#40633. The failure with message

[Falcon] Error: read error on page 85 of "/data0/pushbuild/pb/bzr_mysql-6.0-rpl/43/mysql-6.0.7-alpha-pb43/mysql-test/falcon_master.fts": Success (0) 

is reported as BUG#40133. So please ignore these failures in the present bug.

Marking as a duplicate of Bug#39458 which is "Can't Repeat"

This main issue in this bug seems to be complaining about
Test output match (LIKE):
    %Falcon% Error: page % wrong page type, expected
This pushbuild failure occurred regularly up until 2008-10-03 07:39:00
but has not occurred since.
 
Bug#39458 is now 'Can't Repeat' according to this comment;
[7 Oct 20:27] Ann Harrison
After the changes to the Cache::flush method were reverted, 
this problem no longer occurs.