MySQL Bugs: #32301: Falcon storage engine does not support consistent read via handler methods

Bug #32301	Falcon storage engine does not support consistent read via handler methods
Submitted:	13 Nov 2007 0:57	Modified:	3 Dec 2007 14:26
Reporter:	Chuck Bell	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S3 (Non-critical)
Version:	6.0	OS:	Any
Assigned to:	Kevin Lewis	CPU Architecture:	Any

Description:
I have been working with the mysql-6.0 code (from the tree of the same name) and noticed that the falcon engine doesn't support consistent snapshot via the handler methods -- at least not in the manner expected. 

Does falcon support consistent read? If it does, then we need some additional functionality added to falcon in order to read falcon tables in online backup via consistent read.

Allow me to explain how we are using consistent read with InnoDB.

When a table is included in a backup, the online backup code interrogates the handler for the table to see if the start_consistent_snapshot() method is implemented. If it is, then that indicates (to us) that the engine supports consistent read and we assign that table to the consistent snapshot driver. If the storage engine does not have that method defined, the table is assigned to the default (blocking) driver.

Once we identify a table's storage engine as supporting consistent read and assign it to the consistent snapshot driver, we then use the following code to initiate the consistent read in our consistent snapshot driver's lock() method.

  m_thd->lex->start_transaction_opt|= MYSQL_START_TRANS_OPT_WITH_CONS_SNAPSHOT;
  int res= begin_trans(m_thd);

The table is read using normal handler methods for a table scan, e.g. hdl->ha_rnd_init(1) + hton->rnd_next() + hdl->ha_rnd_end().

If falcon supports consistent read and therefore will use the consistent snapshot driver to do a logical backup of tables, then the falcon engine must support these methods and capabilities:

  hton->start_consistent_snapshot();
  m_thd->lex->start_transaction_opt|= MYSQL_START_TRANS_OPT_WITH_CONS_SNAPSHOT;
  ha_start_consistent_snapshot();
  ..and, of course, transactions. 

See ha_innodb.cc for details on how InnoDB implements the above.

How to repeat:
Create a table using the falcon storage engine.
Attempt a backup while inserting data into the table.

Since the falcon engine doesn't use the handler methods, online backup will see the storage engine as not supporting consistent read and will use the default (blocking) driver. Thus, inserts that occur at the same time as backup (provided there are no lock issues) are included in the backup image.

Suggested fix:
Change the falcon storage engine to support the handler methods as stated above.

I have compiled Kevin's code in mysql-6.0-falcon-team after pulling from mysql-6.0-backup and applying the patch for BUG#31383. 

The code compiles well on Windows (great job, Falcon team!) and online backup correctly identifies the falcon table as supporting consistent read. However, when the consistent snapshot test was run, the backup image included rows inserted during the period when the consistent read should have been activated.

I have included a modified version of the backup_snapshot test that can be used to test whether the falcon storage engine is correctly engaging the consistent read. Please see WL#4120 for details on how the test was constructed and how to set breakpoints in the backup code to ensure deterministic results.

NOTICE: This bug report, once solved, will require BUG#31383 and the code from mysql-6.0-backup (if BUG#31383 has not been included already).

Modified consistent snapshot test (includes falcon tables).

Attachment: backup_snapshot.test (application/octet-stream, text), 5.35 KiB.

In ha_falcon.cpp, StorageInterface::start_consistent_snapshot(),  I used the wrong transaction isolation level;

int ret = storageHandler->startTransaction(thd, TRANSACTION_READ_COMMITTED);

It should be 

int ret = storageHandler->startTransaction(thd, TRANSACTION_CONSISTENT_READ);

I debugged it with Chuck Bell's merged tree and test case, and it does start the transaction OK.  I pushed this into the mysql-6.0-falcon-team tree.

Checked in another change which allowed the test case to work.  Forwarded a patch to Chuck for verification.

I can confirm that the changes descibed above fix the problem.

In addition, I think it best to add falcon to the existing backup_snapshot test. I have attached a patch to do this. A corresponding result file can be generated. 

Note: The new test should be pushed only after the mysql-6.0-backup tree is merged with the mysql-6.0-falcon-team tree (via the normal process). This is because the corrected test requires the latest patches from mysql-6.0-backup and mysql-6.0-falcon-team. Specifically, the patch for BUG#31383 (which was pushed on 15 Nov) and the falcon code updates.

Diffs to add falcon table to backup_snapshot.test.

Attachment: falcon_snapshot.diffs (application/octet-stream, text), 6.24 KiB.

Sample result file for updated backup_snapshot test.

Attachment: backup_snapshot.result (application/octet-stream, text), 5.69 KiB.

Upgraded because of need to provide support for Online Backup Consistent Snapshot backup in Falcon in 6.0.4

Chuck,  Is there a testcase or test sequence that shows what is still wrong with this consistent snapshot code?

> From: Kevin Lewis [mailto:klewis@mysql.com] 
> Sent: Thursday, November 29, 2007 6:17 AM
> I do not see any problems with Falcon consistent snapshot.  I 
> ran Chuck's latest testcase and code with a break on the 
> backup wait, and was sure that during the backup, another 
> client inserted and updated the table.  After the restore, 
> the new records were not visible.

-----Original Message-----
From: Chuck Bell [mailto:cbell@mysql.com] 
Sent: Thursday, November 29, 2007 8:57 AM

I concur. The falcon code works. I did a rebuild and the issue I was seeing
went away. Operator error let's call it. ;)

Pushed into 6.0.4-alpha

A note has been added to the 6.0.4 changelog: 

Falcon would incorrectly return the supported repeatable-read level when queried by the Online Backup system, preventing the ability to create a consistent snapshot backup.