MySQL Bugs: #39661: Assert in SerialLogWindow.cpp during very long lasting insert transaction

Bug #39661	Assert in SerialLogWindow.cpp during very long lasting insert transaction
Submitted:	26 Sep 2008 11:30	Modified:	1 Sep 9:42
Reporter:	Olav Sandstaa
Status:	Duplicate
Category:	Server: Falcon	Severity:	S2 (Serious)
Version:	6.0.7	OS:	Any
Assigned to:	Olav Sandstaa	Target Version:	6.0-beta
Tags:	F_MEMORY
Triage:	Triaged: D1 (Critical)

Description:
In an attempt to reproduce bug #39575 I run into a different crash scenario that may or
may not be caused by the same problem as in bug #39575. The test client attempts to
insert 100 million records each of about 104 byte into Falcon as a single transaction.
During this operation the server hits the following assert:

[Falcon] Error: assertion (firstBlock()->blockNumber == firstBlockNumber) failed at line
296 in file SerialLogWindow.cpp

The error file claims that a core file is being written but I have been unable to locate
it. Possibly there was not room on my disk for writing a complete core of this process.

Just as in bug 39575 this seems to be related to that the size of one of the serial log
files reaches 4 GB:

-rw-rw----   1 olav  6291456 Sep 25 15:07 falcon_master.fl2
-rw-rw----   1 olav  4294967296 Sep 26 03:28 falcon_master.fl1

How to repeat:
Start Falcon with a fairly large record cache and page cache (in my case the record cache
was 10 GB and the page cache 8 GB). The insert 100 million records of about 100 byte each
into Falcon as a single transaction. This should make the serial log file grow beyond 4
GB.

Note that in bug 39575 there are some comments that this issue might be a Solaris/Sun
Studio problem. This far I have only tried to run this test on Solaris  and with Falcon
compiled using Sun Studio 12.

Suggested fix:
Falcon should not crash when the size of the serial log file grows to 4 GB.

Some additional observations about the crash:

1. The client exited with the following error message from the server:

 Sep 26, 2008 2:16:06 AM LoadData main
 SEVERE: null
 java.sql.SQLException: Got error 305 'record memory is exhausted' from Falcon

2. The server crash/assert occurred over one hour after the client had exited.

3. After the crash, Falcon is unable to recover:

080926 03:28:54 mysqld_safe mysqld restarted
New table space FALCON_USER, id 1, type 0, filename falcon_user.fts
New table space FALCON_TEMPORARY, id 2, type 0, filename falcon_temporary.fts
080926  3:29:52 [ERROR] Falcon: Recovery failed: corrupted serial log
080926  3:29:52 [ERROR] Plugin 'Falcon' init function returned error.
080926  3:29:52 [ERROR] Plugin 'Falcon' registration as a STORAGE ENGINE failed.
080926  3:29:52  InnoDB: Started; log sequence number 0 46409
080926  3:29:52 [ERROR] Unknown/unsupported table type: falcon
080926  3:29:52 [ERROR] Aborting

Load client used for reproducing the crash

Attachment: LoadData.java (text/x-java), 3.86 KiB.

Verified that the crash is reproducable using the included test client. Falcon is compiled
with Sun Studio 12 and running on Solaris 10 MU2 server. Falcon is configured with 10 GB
of record cache and 8 GB page cache.

I have not been able to reproduce this after the fix to bug#39575 was applied. Most likely
this bug was also caused by the same code error as bug#39575. Closing this as a duplicate
of that bug.