Bug #36269 Thread stalls during DBT2 run
Submitted: 22 Apr 2008 23:32 Modified: 8 Jan 2009 9:48
Reporter: Vladislav Vaintroub Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S3 (Non-critical)
Version:6.0 OS:Any
Assigned to: Christopher Powers
Triage: D2 (Serious)

[22 Apr 2008 23:32] Vladislav Vaintroub
Description:
During DBT2 benchmark  run a situation similar to deadlock (may not necessarily be a real deadlock, but massive thread stalls) has occured.

The callstack will follow shortly.

Interesting details:
- Most threads have  Database::retireRecords on stack 

2  0x00000000007e0129 in SyncObject::wait (this=0x2aaaab2efc10, type=<value optimized out>, thread=0x2aab48db3f00, sync=<value optimized out>, timeout=0) at SyncObject.cpp:412
#3  0x00000000007df1e0 in Sync::lock (this=0x45ad2aa0, type=Exclusive) at Sync.cpp:58
#4  0x000000000081120e in Database::retireRecords (this=0x2aaaab2eeba0, forced=true) at Database.cpp:1762
#5  0x00000000007e11c5 in Table::allocRecordVersion (this=0x2aab1c8ed838, format=0x0, transaction=0x2aab2acb55c8, priorVersion=0x2aab346a8330) at Table.cpp:3619
#6  0x00000000007e571d in Table::fetchForUpdate (this=0x2aab1c8ed838, transaction=0x2aab2acb55c8, source=<value optimized out>, usingIndex=<value optimized out>) at Table.cpp:3510
#7  0x00000000007d5da4 in StorageDatabase::nextIndexed (this=<value optimized out>, storageTable=0x2aab30028d30, recordBitmap=0x2aab44103c68, recordNumber=4, lockForUpdate=true) at StorageDatabase.cpp:390

- Most threads have Table::fetchForUpdate on stack, smth like

2  0x00000000007e0129 in SyncObject::wait (this=0x2aaaab2efc10, type=<value optimized out>, thread=0x2aab48db4760, sync=<value optimized out>, timeout=0) at SyncObject.cpp:412
#3  0x00000000007df1e0 in Sync::lock (this=0x45b541c0, type=Exclusive) at Sync.cpp:58
#4  0x000000000081120e in Database::retireRecords (this=0x2aaaab2eeba0, forced=true) at Database.cpp:1762
#5  0x00000000007e11c5 in Table::allocRecordVersion (this=0x2aab1c8ecc48, format=0x0, transaction=0x2aab5583f408, priorVersion=0x2aab3b85f890) at Table.cpp:3619
#6  0x00000000007e571d in Table::fetchForUpdate (this=0x2aab1c8ecc48, transaction=0x2aab5583f408, source=<value optimized out>, usingIndex=<value optimized out>) at Table.cpp:3510

- Some threads have Transaction::waitForTransaction on stack like
#4  0x00000000007f0771 in Transaction::waitForTransaction (this=<value optimized out>) at Transaction.cpp:1025
#5  0x00000000007f19c6 in Transaction::getRelativeState (this=0x2aab232b4a28, transaction=0x2aab335497b0, transId=136416, flags=4294967295) at Transaction.cpp:892
#6  0x00000000007f1aee in Transaction::getRelativeState (this=0x2aab232b4a28, record=0x2aab25804408, flags=1) at Transaction.cpp:837
#7  0x00000000007e55f0 in Table::fetchForUpdate (this=0x2aab1c8ed838, transaction=0x2aab232b4a28, source=0x2aab25804408, usingIndex=<value optimized out>) at Table.cpp:3482

How to repeat:
No idea,  with luck DBT2 run with 192 clients may result in this situation
[22 Apr 2008 23:36] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45854

ChangeSet@1.2639, 2008-04-22 18:33:52-05:00, cpowers@xeno.mysql.com +1 -0
  Bug#36269: Thread stalls during DBT2 run
  - Stack trace shows 179 threads blocked on a forced scavenge.
  - Removed syncSysDDL shared lock and corrected scavenge cycle bug.
[22 Apr 2008 23:52] Vladislav Vaintroub
stacktraces

Attachment: dbt2_deadlock.txt.gz (application/x-gzip, text), 59.85 KiB.

[2 Jul 2008 12:41] Kevin Lewis
This change is in the 6.0.5 release.
[2 Jul 2008 14:10] Kevin Lewis
Sorry, it was pushed AFTER the 6.0.5 release. It is in 6.0.6.
[8 Jan 2009 9:48] MC Brown
Internal change only. No documentation needed.