Bug #40691 Falcon: Chill/thaw operations do not lock SerialLog::syncWrite
Submitted: 13 Nov 2008 9:02 Modified: 13 Dec 2008 9:16
Reporter: Christopher Powers Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S1 (Critical)
Version:6.0 OS:Any
Assigned to: Christopher Powers CPU Architecture:Any
Tags: F_CHILL THAW
Triage: Triaged: D1 (Critical)

[13 Nov 2008 9:02] Christopher Powers
Description:
Deferred index thaw operations sometimes fail or crash because the virtual offset into the serial log that points to the chilled record is incorrect.

How to repeat:
Branch or checkout the mysql-test-extra-6.0 bazaar tree and run:

$ cd mysql-test\gentest
$ perl runall.pl \
 --basedir=path\to\mysql-6.0-falcon-team \
 --reporters=Deadlock,ErrorLog,Backtrace \
 --mysqld=--loose-falcon-lock-wait-timeout=1 \
 --engine=Falcon \
 --grammar=conf/falcon_chill_thaw.yy \ 
 --queries=100000 \
 --mysqld=--falcon-index-chill-threshold=4K \ 
 --mysqld=--falcon-record-chill-threshold=4K \ 
 --duration=1200

Suggested fix:
During the falcon_chill_thaw_stress test, DeferredIndex::virtualOffset will sometimes fall several serial log records short of the intended location.

This is because SRLUpdateIndex::append sets DeferredIndex::virtualOffset outside of the SerialLog::syncWrite lock. By the time SerialLog::startRecord() is called, the virtual offset may no longer be valid.

The solution is to set DeferredIndex::virtualOffset *after* SerialLog::syncWrite is locked and *before* the first write into the new serial log record.

Similarly, SRLUpdateRecords::thaw() should also lock SerialLog::syncWrite before attempting to restore a record from the serial log.
[13 Nov 2008 9:07] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58610

2909 Christopher Powers	2008-11-13
      Bug#40690, "Falcon: Deferred index thaw crashes in addNode()"
      Bug#40691, "Falcon: Chill/thaw operations do not lock SerialLog::syncWrite"
[14 Nov 2008 15:42] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58805

2913 Christopher Powers	2008-11-14
      Bug#40690, "Falcon: Deferred index thaw crashes in addNode()"
      Bug#40691, "Falcon: Chill/thaw operations do not lock SerialLog::syncWrite"
      
      Closed synchronization gaps for record+index chill/thaw
[20 Nov 2008 12:56] Bugs System
Pushed into 6.0.9-alpha  (revid:cpowers@mysql.com-20081114153844-wgyg2mm8ihig75el) (version source revid:cpowers@mysql.com-20081114153844-wgyg2mm8ihig75el) (pib:5)
[13 Dec 2008 9:16] MC Brown
No documentation needed (internal change only)