Bug #39845 Falcon: Concurrent online DROP INDEX causes assertion
Submitted: 3 Oct 2008 23:53 Modified: 9 Jan 2009 14:10
Reporter: Christopher Powers Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S1 (Critical)
Version:6.07 OS:Any
Assigned to: Christopher Powers CPU Architecture:Any
Tags: F_ONLINE ALTER

[3 Oct 2008 23:53] Christopher Powers
Description:
The concurrent online alter stress test, falcon_online_alter, causes a fatal assertion in Index::makeKey().

Table::rollbackRecord() cycles through the indexes to clean up unused keys. The assertion occurs if the rollback accesses an index that had just been dropped.

The assertion implies that indexId == -1, however, in some cases, the
debugger shows indexId to have a normal value, suggesting an A-B-A condition.

Transaction::rollback()
   RecordVersion::rollback()
      Table::rollbackRecord()
         Table::garbageCollect()
            Index::garbageCollect(Record *leaving, *staying)
            {
               for (record = leaving to staying)
               {
                  ...
                  IndexKey key(this);
                  makeKey(record, &key);

                  Index::makeKey(...)
                  {
                     ASSERT (indexId != -1);
                     ...
                  } 

How to repeat:
1. Install random query generator: 
https://inside.mysql.com/wiki/QARandomQueryGenerationTutorial

2. Run SystemQA falcon_online_alter

runall.pl 
   --basedir=<mysql directory> \
   --engine=Falcon \
   --grammar=conf/falcon_online_alter.yy \
   --threads=10 \
   --queries=100000

Suggested fix:
A shared Table lock in Table::rollbackRecord() prevents the assert. The larger question is whether all concurrent operations are sufficiently protected from online drop index (Falcon does not support online drop column.)
[3 Oct 2008 23:55] Christopher Powers
Verified during debugging of Bug#39795, "Falcon: Online add index does not support index with non-null columns"
[3 Oct 2008 23:59] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/55298

2852 Christopher Powers	2008-10-03
      Bug#39845, "Falcon: Concurrent online DROP INDEX causes assertion"
[9 Jan 2009 14:10] MC Brown
A note has been added to the 6.0.8 changelog: 

When performing an online DROP INDEX on a Falcon table, the operation may conflict with other index operations such as including index scans. When one client drops an index, another client may initiate a concurrent index operation that accesses the mapping object of the index being dropped, and this can cause a crash