Bug #44233 Crash in Table::validateAndInsert when updating cardinalities
Submitted: 13 Apr 2009 10:34 Modified: 15 May 2009 14:50
Reporter: Philip Stoev Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S2 (Serious)
Version:6.0-falcon-team OS:Any
Assigned to: Kevin Lewis CPU Architecture:Any
Triage: Triaged: D1 (Critical)

[13 Apr 2009 10:34] Philip Stoev
Description:
When executing a simple RQG workload, Falcon crashed as follows:

# 03:03:07 #2  0x00000000006c2e15 in handle_segfault (sig=11) at mysqld.cc:2693
# 03:03:07 #3  <signal handler called>
# 03:03:07 #4  0x000000000099f27e in interlockedIncrement (ptr=0xb8) at Interlock.h:287
# 03:03:07 #5  0x00000000009b95c0 in TransactionState::addRef (this=0x0)
# 03:03:07     at TransactionState.cpp:43
# 03:03:07 #6  0x00000000009a65e5 in Table::validateAndInsert (this=0x2aaab02dcfc8,
# 03:03:07     transaction=0x2aaab0328dc0, record=0x2aaae1e91458) at Table.cpp:3360
# 03:03:07 #7  0x00000000009aaa92 in Table::update (this=0x2aaab02dcfc8,
# 03:03:07     transaction=0x2aaab0328dc0, oldRecord=0x2aaaaacb0558, numberFields=1,
# 03:03:07     updateFields=0x2aaab038b130, values=0x2aaab0357fa8) at Table.cpp:1391
# 03:03:07 #8  0x0000000000a1f11c in NUpdate::evalStatement (this=0x2aaab03857a8,
# 03:03:07     statement=0x2aaab03567b8) at NUpdate.cpp:143
# 03:03:07 #9  0x0000000000a6fb48 in Nfs::Statement::start (this=0x2aaab03567b8,
# 03:03:07     node=0x2aaab03857a8) at Statement.cpp:492
# 03:03:07 #10 0x0000000000a229e6 in PreparedStatement::executeUpdate (
# 03:03:07     this=0x2aaab03567b8) at PreparedStatement.cpp:86
# 03:03:07 #11 0x00000000009d699c in Database::updateCardinalities (this=0x2aaab0141be0)
# 03:03:07     at Database.cpp:2580
# 03:03:07 #12 0x00000000009d6b47 in Database::cardinalityThreadMain (this=0x2aaab0141be0)
# 03:03:07     at Database.cpp:2519
# 03:03:07 #13 0x00000000009d6baf in Database::cardinalityThreadMain (
# 03:03:07     database=0x2aaab0141be0) at Database.cpp:2502
# 03:03:07 #14 0x00000000009b04cf in Thread::thread (this=0x2aaab034d348)

note that this=0x0 in frame #5 -- the bad pointer to a TransactionState object should have been detected somewhere earlier in the call stack, and not in the interlockedIncrement.

How to repeat:
This was seen using the following RQG command:

 perl runall.pl    --engine=Falcon   --reporters=Deadlock,ErrorLog,Backtrace,Recovery,Shutdown   --mysqld=--loose-falcon-lock-wait
-timeout=1   --mysqld=--loose-innodb-lock-wait-timeout=1   --mysqld=--log-output=none   --mysqld=--skip-safemalloc   --mysqld=--transaction-isolation=SERIALIZ
ABLE --mysqld=--falcon-page-size=8K --mem --rows=1000 --threads=4 --mysqld=--falcon-checkpoint-schedule='1 1 1 1 1'   --mask=31588   --queries=100000000   --d
uration=900   --basedir=/export/home/pb2/test/sb_1-462247-1239581132.78/mysql-6.0.11-alpha-linux-x86_64-test  --gendata=conf/combinations.zz  --grammar=conf/c
ombinations.yy
[13 Apr 2009 15:13] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/71941

3118 Kevin Lewis	2009-04-13
      Bug#44233 - The priorRecord for the record to be inserted  might not be the current record if that prior record rolled back.  This may have happend by a system transactions during updateCarbinalities.  If that happened, the current record may not be a RecordVersion, and may not have a TransactionState.
[13 Apr 2009 20:18] Hakan Küçükyılmaz
I saw the same stacktrace running DBT2 this weekend with 100 warehouses.
[15 Apr 2009 16:59] Bugs System
Pushed into 6.0.11-alpha (revid:hky@sun.com-20090415164923-9arx29ye5pzxd4zf) (version source revid:kevin.lewis@sun.com-20090413151255-h64gg0x3rihwpp2e) (merge vers: 6.0.11-alpha) (pib:6)
[15 May 2009 14:50] MC Brown
Internal/test fix. No changelog entry required.