MySQL Bugs: #42342: Falcon crash in Index::duplicateKey

Bug #42342	Falcon crash in Index::duplicateKey
Submitted:	26 Jan 2009 9:53	Modified:	7 May 2009 16:15
Reporter:	Philip Stoev	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S1 (Critical)
Version:	6.0-falcon-team	OS:	Any
Assigned to:	Christopher Powers	CPU Architecture:	Any
Tags:	F_RECORD TREE

Description:
When executing a random workload in 32 threads, Falcon crashed as follows:

#2  0x000000000081cae3 in handle_segfault (sig=11) at mysqld.cc:2679
#3  <signal handler called>
#4  0x0000000000f4daf6 in Index::duplicateKey (this=0x7f0870359e20,
    key=0x7f085c5a24a0, record=0x42e7a50) at Index.cpp:688
#5  0x0000000000f4ed5e in Index::update (this=0x7f0870359e20,
    oldRecord=0x42e7a50, record=0x3ec7940, transaction=0x7f08702b2aa0)
    at Index.cpp:608
#6  0x0000000000e862f7 in Table::updateIndexes (this=0x7f087067e860,
    transaction=0x7f08702b2aa0, record=0x3ec7940, oldRecord=0x42e7a50)
    at Table.cpp:1246
#7  0x0000000000e8ba91 in Table::update (this=0x7f087067e860,
    transaction=0x7f08702b2aa0, orgRecord=0x42e7a50, stream=0x7f08706dff18)
    at Table.cpp:3179
#8  0x0000000000e52f75 in StorageDatabase::updateRow (this=0x7f087030e210,
    storageConnection=0x7f087034cfb8, table=0x7f087067e860,
    oldRecord=0x42e7a50, stream=0x7f08706dff18) at StorageDatabase.cpp:674
#9  0x0000000000e6392e in StorageTable::updateRow (this=0x7f08706da978,
    recordNumber=5679) at StorageTable.cpp:132
#10 0x0000000000e46ac8 in StorageInterface::update_row (this=0x3487420,
    oldData=0x37352b0 "О©ҐО©ҐО©Ґ", newData=0x3734d40 "О©ҐО©ҐО©Ґ") at ha_falcon.cpp:1214
#11 0x0000000000aea9f8 in handler::ha_update_row (this=0x3487420,
    old_data=0x37352b0 "О©ҐО©ҐО©Ґ", new_data=0x3734d40 "О©ҐО©ҐО©Ґ") at handler.cc:5463
#12 0x00000000009e49e0 in mysql_update (thd=0x7f086cea7138,
    table_list=0x3436bb0, fields=@0x7f086cea9108, values=@0x7f086cea9510,
    conds=0x0, order_num=0, order=0x0, limit=18446744073709545936,
    handle_duplicates=DUP_ERROR, ignore=false) at sql_update.cc:649
#13 0x000000000084b6ab in mysql_execute_command (thd=0x7f086cea7138)
    at sql_parse.cc:3005
#14 0x00000000008588c5 in mysql_parse (thd=0x7f086cea7138,
    inBuf=0x34369b0 "UPDATE `table1000_falcon_int_autoinc` SET `datetime_key_not_null` = '20040720052614'", length=84, found_semicolon=0x7f085c5a6ef0)
    at sql_parse.cc:5735
#15 0x000000000085a0e1 in dispatch_command (command=COM_QUERY,
    thd=0x7f086cea7138,
    packet=0x7f086ceb1b49 "UPDATE `table1000_falcon_int_autoinc` SET `datetime_key_not_null` = '20040720052614'   ", packet_length=87) at sql_parse.cc:1008
#16 0x000000000085ce2d in do_command (thd=0x7f086cea7138) at sql_parse.cc:691
#17 0x00000000008331ed in handle_one_connection (arg=0x7f086cea7138)
    at sql_connect.cc:1145
#18 0x000000315b0073da in start_thread () from /lib64/libpthread.so.0
#19 0x000000315a4e627d in clone () from /lib64/libc.so.6

683
684     }
685
686     bool Index::duplicateKey(IndexKey *key, Record * record)
687     {
688             for (Record *oldie = record; oldie; oldie = oldie->getPriorVersion()) <<<<<<<<<<<<<<<<<< HERE
689                     if (oldie->hasRecord())
690                             {
691                             IndexKey oldKey(this);
692                             makeKey (oldie, &oldKey);

(gdb) print oldie
$1 = (class Record *) 0x3e403c8
(gdb) print *oldie
$2 = {_vptr.Record = 0xeeeeeeeeeeeeeeee, data = {record = 0xeeeeeeeeeeeeeeee <Address 0xeeeeeeeeeeeeeeee out of bounds>}, useCount = -286331154,
  format = 0xeeeeeeeeeeeeeeee, recordNumber = -286331154, size = -286331154, generation = 17216961135462248174, highWater = -4370, encoding = 238 'Н',
  state = 238 'Н'}
(gdb) print record
$3 = (class Record *) 0x42e7a50

How to repeat:
A test case will be uploaded shortly.

Verified by code inspection.  Unlike other places where Index::duplicateKey can be called, Table::update() does not get a shared lock on syncPrior first.  So it is possible for a records to be scvenged and deleted while this function is traversing the prior record chain.

It just so happens that Chris an I noticed this last Friday while discussing another issue.  He made this change on Saturday which includes this fix as well.
http://lists.mysql.com/commits/63991

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/64059

2975 Christopher Powers	2009-01-26
      Bug #42342, "Falcon crash in Index::duplicateKey"
      
      Protect record version chain while searching for duplicate keys.

Pushed into 6.0.10-alpha (revid:alik@sun.com-20090211182317-uagkyj01fk30p1f8) (version source revid:vvaintroub@mysql.com-20090126231319-7b6yu4zqp2xfxuqg) (merge vers: 6.0.10-alpha) (pib:6)

Internal/test fix. No changelog entry required.