Bug #40691 Falcon: Chill/thaw operations do not lock SerialLog::syncWrite
Submitted: 13 Nov 2008 10:02 Modified: 13 Dec 2008 10:16
Reporter: Christopher Powers
Status: Closed
Category:Server: Falcon Severity:S1 (Critical)
Version:6.0 OS:Any
Assigned to: Christopher Powers Target Version:6.0-beta
Tags: F_CHILL THAW
Triage: Triaged: D1 (Critical)

[13 Nov 2008 10:02] Christopher Powers 
Description:
Deferred index thaw operations sometimes fail or crash because the virtual offset into
the serial log that points to the chilled record is incorrect.

How to repeat:
Branch or checkout the mysql-test-extra-6.0 bazaar tree and run:

$ cd mysql-test\gentest
$ perl runall.pl \
 --basedir=path\to\mysql-6.0-falcon-team \
 --reporters=Deadlock,ErrorLog,Backtrace \
 --mysqld=--loose-falcon-lock-wait-timeout=1 \
 --engine=Falcon \
 --grammar=conf/falcon_chill_thaw.yy \ 
 --queries=100000 \
 --mysqld=--falcon-index-chill-threshold=4K \ 
 --mysqld=--falcon-record-chill-threshold=4K \ 
 --duration=1200

Suggested fix:
During the falcon_chill_thaw_stress test, DeferredIndex::virtualOffset will sometimes
fall several serial log records short of the intended location.

This is because SRLUpdateIndex::append sets DeferredIndex::virtualOffset outside of the
SerialLog::syncWrite lock. By the time SerialLog::startRecord() is called, the virtual
offset may no longer be valid.

The solution is to set DeferredIndex::virtualOffset *after* SerialLog::syncWrite is
locked and *before* the first write into the new serial log record.

Similarly, SRLUpdateRecords::thaw() should also lock SerialLog::syncWrite before
attempting to restore a record from the serial log.
[13 Nov 2008 10:07] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58610

2909 Christopher Powers	2008-11-13
      Bug#40690, "Falcon: Deferred index thaw crashes in addNode()"
      Bug#40691, "Falcon: Chill/thaw operations do not lock SerialLog::syncWrite"
[14 Nov 2008 16:42] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58805

2913 Christopher Powers	2008-11-14
      Bug#40690, "Falcon: Deferred index thaw crashes in addNode()"
      Bug#40691, "Falcon: Chill/thaw operations do not lock SerialLog::syncWrite"
      
      Closed synchronization gaps for record+index chill/thaw
[20 Nov 2008 13:56] Bugs System 
Pushed into 6.0.9-alpha  (revid:cpowers@mysql.com-20081114153844-wgyg2mm8ihig75el)
(version source revid:cpowers@mysql.com-20081114153844-wgyg2mm8ihig75el) (pib:5)
[13 Dec 2008 10:16] MC Brown 
No documentation needed (internal change only)