MySQL Bugs: #38041: Bizarre errors when ALTER ADD/DROP KEY on Falcon tables

Bug #38041	Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
Submitted:	11 Jul 2008 9:46	Modified:	30 Sep 2008 20:31
Reporter:	Philip Stoev	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S1 (Critical)
Version:	6.0-falcon	OS:	Any
Assigned to:	Christopher Powers	CPU Architecture:	Any

Description:
When executing a sequence of ALTER ADD/DROP key statements, a range of bizarre errors are produced. I am filing those as a single bug because I do not think it is possible that they are all unrelated. Here is a list of messages seen so far:

ALTER TABLE C DROP KEY int_nokey;
ERROR HY000: Got error 124 from storage engine

ALTER TABLE C ADD KEY ( int_nokey );
ERROR HY000: Error on rename of './test/#sql-11a2_1' to './test/C' (errno: 121)

ALTER TABLE B DROP KEY int_key;
ERROR HY000: Got error 124 'duplicate values for key INDEXES..PRIMARY_KEY in table SYSTEM.INDEXES' from Falcon

ALTER TABLE B DROP KEY int_key
1296 Got error 124 'duplicate values for key #SQL-E3C_3$1 in table TEST.#SQL-E3C_3' from Falcon

ALTER TABLE B DROP KEY int_key
1296 Got error 124 'duplicate values for key B$1 in table TEST.B' from Falcon

ALTER TABLE B DROP KEY int_key
1296 Got error 124 'duplicate values for key C..PRIMARY_KEY in table TEST.C' from Falcon

Please note that the last message says error in table C, but we are altering table B.

Let me know if you want those errors broken down into individual bug reports.

How to repeat:
A test case will be uploaded shortly.

Test case for bug 38041

Attachment: bug38041.test (application/octet-stream, text), 2.77 KiB.

Chris, These are error message problems related to WL#4048Online Add/Drop Index.  They should probably not stop the 6.0.6 release with 4048, but if they can be partially cleaned up before then, please try.  This bug can be separated into quick fixes and longer fixes if you want to make multiple pushes.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50471

2745 Christopher Powers	2008-07-24
   Bug#38039 Assertion lockState == 0 failed in StorageTableShare::deleteTable
   Bug#38041 Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
   Bug#38043 Deadlock between server and falcon when executing concurrent ALTER
      
   Improvements to online add/drop index:
      
   - StorageTableShare::indexes[] is now a resizeable DenseArray
   - Added sync lock protection to get/set index
   - Call add/drop index from StorageTable rather than StorageTableShare
   - Rename internal Falcon indexes to preserve index name/id mapping after drop
   - Immediately update StorageTableShare::indexes[] after add/drop rather than
     wait for subsequent open()

Test case passes. Errors were due to DROP INDEX not being handled properly.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51820

2787 Christopher Powers	2008-08-18
      Bug#38044 Falcon crash in StorageTable::compareKey at StorageTable.cpp:412
      Bug#38043 Deadlock between server and falcon when executing concurrent ALTER + DML
      Bug#38039 Assertion lockState == 0 failed in StorageTableShare::deleteTable SyncObject
      Bug#38041 Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
      
      Improved index synchronization in the StorageInterface:
      - Falcon internal indexes no longer dependent upon server index id
      - Client connections lock StorageTableShare::syncIndexes to protect indexes across calls into StorageInterface
      - Changes to StorageTableShare::indexes are now protected by exclusive lock

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51859

2787 Christopher Powers	2008-08-18
      Bug#38044 Falcon crash in StorageTable::compareKey at StorageTable.cpp:412
      Bug#38043 Deadlock between server and falcon when executing concurrent ALTER + DML
      Bug#38039 Assertion lockState == 0 failed in StorageTableShare::deleteTable SyncObject
      Bug#38041 Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
      
      Improved index synchronization in the StorageInterface:
      - Falcon internal indexes no longer dependent upon server index id
      - Client connections lock StorageTableShare::syncIndexes to protect indexes across calls into StorageInterface
      - Changes to StorageTableShare::indexes are now protected by exclusive lock

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51863

2787 Christopher Powers	2008-08-18
      Bug#38044 Falcon crash in StorageTable::compareKey at StorageTable.cpp:412
      Bug#38043 Deadlock between server and falcon when executing concurrent ALTER + DML
      Bug#38039 Assertion lockState == 0 failed in StorageTableShare::deleteTable SyncObject
      Bug#38041 Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
      
      Improved index synchronization in the StorageInterface:
      - Falcon internal indexes no longer dependent upon server index id
      - Client connections lock StorageTableShare::syncIndexes to protect indexes across calls into StorageInterface
      - Changes to StorageTableShare::indexes are now protected by exclusive lock

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52089

2792 Christopher Powers	2008-08-20
      Bug #38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      - Corrected mixup with StorageTableShare::syncObject and ::syncIndexes
      
      
      Bug #38377 "Options falcon_serial_log_dir does not have any effect"
      - Temporarily disabled fatal exception in Configuration::Configuration() until ScanDir.isDirectory() be fixed on Linux.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52090

2792 Christopher Powers	2008-08-20
      Bug #38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      - Corrected mixup with StorageTableShare::syncObject and ::syncIndexes
      
      
      Bug #38377 "Options falcon_serial_log_dir does not have any effect"
      - Temporarily disabled fatal exception in Configuration::Configuration() until ScanDir.isDirectory() be fixed on Linux.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52091

2792 Christopher Powers	2008-08-20
      Bug #38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      - Corrected mixup with StorageTableShare::syncObject and ::syncIndexes
      
      
      Bug #38377 "Options falcon_serial_log_dir does not have any effect"
      - Temporarily disabled fatal exception in Configuration::Configuration() until ScanDir.isDirectory() be fixed on Linux.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52263

2793 Christopher Powers	2008-08-21
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52264

2793 Christopher Powers	2008-08-21
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52265

2793 Christopher Powers	2008-08-21
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52266

2793 Christopher Powers	2008-08-21
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52284

2794 Christopher Powers	2008-08-22
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original
      - Reverted page cache changes, commmit 2793

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52285

2794 Christopher Powers	2008-08-22
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original
      - Reverted page cache changes, commmit 2793

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52286

2794 Christopher Powers	2008-08-22
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original
      - Reverted page cache changes, commmit 2793

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52288

2794 Christopher Powers	2008-08-22
      Bug#38041 "Bizarre errors when ALTER ADD/DROP KEY on Falcon tables"
      
      Resolved memory corruption stemming from incompatible allocation between Falcon and the StorageInterface.
      
      - Converted JString fields in StorageIndexDesc to char[]
      - Changed StorageTableShare::indexes[] from DenseArray to linked list
      - Reverted DenseArray to original
      - Reverted page cache changes, commmit 2793

Pushed into 6.0.7-alpha  (revid:cpowers@mysql.com-20080822064740-21oz10rpfyreebxa) (version source revid:vvaintroub@mysql.com-20080827094944-sh582y3m7duhzrpj) (pib:3)

I am still getting such messages, see bug#39132

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53546

2814 Christopher Powers	2008-09-08
      Bug#38041 Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
      
      - Added validation methods to detect out-of-sync condition between server and falcon indexes
      - Don't create StorageIndexDesc objects if internal falcon index does not exist
      - Enabled HA_ADD/DROP_UNIQUE_INDEX
      - Rebuild index mapping when table is renamed

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53547

2814 Christopher Powers	2008-09-08
      Bug#38041 Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
      
      - Added validation methods to detect out-of-sync condition between server and falcon indexes
      - Don't create StorageIndexDesc objects if internal falcon index does not exist
      - Enabled HA_ADD/DROP_UNIQUE_INDEX
      - Rebuild index mapping when table is renamed

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53548

2814 Christopher Powers	2008-09-08
      Bug#38041 Bizarre errors when ALTER ADD/DROP KEY on Falcon tables
      
      - Added validation methods to detect out-of-sync condition between server and falcon indexes
      - Don't create StorageIndexDesc objects if internal falcon index does not exist
      - Enabled HA_ADD/DROP_UNIQUE_INDEX
      - Rebuild index mapping when table is renamed

Pushed into 6.0.6-alpha  (revid:cpowers@mysql.com-20080818054529-s9swke0xd15rx1fu) (version source revid:hakan@mysql.com-20080716175219-8unfm2ikarselqpw) (pib:3)

Documented as follows in the 6.0.6 changelog:

        ALTER TABLE ... ADD KEY and ALTER TABLE ... DROP KEY were not always
        handled correctly for Falcon tables, resulting in spurious duplicate key
        and other errors.