Bug #36269 Thread stalls during DBT2 run
Submitted: 23 Apr 2008 1:32 Modified: 8 Jan 10:48
Reporter: Vladislav Vaintroub
Status: Closed
Category:Server: Falcon Severity:S3 (Non-critical)
Version:6.0 OS:Any
Assigned to: Christopher Powers Target Version:6.0.6
Triage: D2 (Serious)

[23 Apr 2008 1:32] Vladislav Vaintroub
Description:
During DBT2 benchmark  run a situation similar to deadlock (may not necessarily be a real
deadlock, but massive thread stalls) has occured.

The callstack will follow shortly.

Interesting details:
- Most threads have  Database::retireRecords on stack 

2  0x00000000007e0129 in SyncObject::wait (this=0x2aaaab2efc10, type=<value optimized
out>, thread=0x2aab48db3f00, sync=<value optimized out>, timeout=0) at SyncObject.cpp:412
#3  0x00000000007df1e0 in Sync::lock (this=0x45ad2aa0, type=Exclusive) at Sync.cpp:58
#4  0x000000000081120e in Database::retireRecords (this=0x2aaaab2eeba0, forced=true) at
Database.cpp:1762
#5  0x00000000007e11c5 in Table::allocRecordVersion (this=0x2aab1c8ed838, format=0x0,
transaction=0x2aab2acb55c8, priorVersion=0x2aab346a8330) at Table.cpp:3619
#6  0x00000000007e571d in Table::fetchForUpdate (this=0x2aab1c8ed838,
transaction=0x2aab2acb55c8, source=<value optimized out>, usingIndex=<value optimized
out>) at Table.cpp:3510
#7  0x00000000007d5da4 in StorageDatabase::nextIndexed (this=<value optimized out>,
storageTable=0x2aab30028d30, recordBitmap=0x2aab44103c68, recordNumber=4,
lockForUpdate=true) at StorageDatabase.cpp:390

- Most threads have Table::fetchForUpdate on stack, smth like

2  0x00000000007e0129 in SyncObject::wait (this=0x2aaaab2efc10, type=<value optimized
out>, thread=0x2aab48db4760, sync=<value optimized out>, timeout=0) at SyncObject.cpp:412
#3  0x00000000007df1e0 in Sync::lock (this=0x45b541c0, type=Exclusive) at Sync.cpp:58
#4  0x000000000081120e in Database::retireRecords (this=0x2aaaab2eeba0, forced=true) at
Database.cpp:1762
#5  0x00000000007e11c5 in Table::allocRecordVersion (this=0x2aab1c8ecc48, format=0x0,
transaction=0x2aab5583f408, priorVersion=0x2aab3b85f890) at Table.cpp:3619
#6  0x00000000007e571d in Table::fetchForUpdate (this=0x2aab1c8ecc48,
transaction=0x2aab5583f408, source=<value optimized out>, usingIndex=<value optimized
out>) at Table.cpp:3510

- Some threads have Transaction::waitForTransaction on stack like
#4  0x00000000007f0771 in Transaction::waitForTransaction (this=<value optimized out>) at
Transaction.cpp:1025
#5  0x00000000007f19c6 in Transaction::getRelativeState (this=0x2aab232b4a28,
transaction=0x2aab335497b0, transId=136416, flags=4294967295) at Transaction.cpp:892
#6  0x00000000007f1aee in Transaction::getRelativeState (this=0x2aab232b4a28,
record=0x2aab25804408, flags=1) at Transaction.cpp:837
#7  0x00000000007e55f0 in Table::fetchForUpdate (this=0x2aab1c8ed838,
transaction=0x2aab232b4a28, source=0x2aab25804408, usingIndex=<value optimized out>) at
Table.cpp:3482

How to repeat:
No idea,  with luck DBT2 run with 192 clients may result in this situation
[23 Apr 2008 1:36] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45854

ChangeSet@1.2639, 2008-04-22 18:33:52-05:00, cpowers@xeno.mysql.com +1 -0
  Bug#36269: Thread stalls during DBT2 run
  - Stack trace shows 179 threads blocked on a forced scavenge.
  - Removed syncSysDDL shared lock and corrected scavenge cycle bug.
[23 Apr 2008 1:52] Vladislav Vaintroub
stacktraces

Attachment: dbt2_deadlock.txt.gz (application/x-gzip, text), 59.85 KiB.

[2 Jul 2008 14:41] Kevin Lewis
This change is in the 6.0.5 release.
[2 Jul 2008 16:10] Kevin Lewis
Sorry, it was pushed AFTER the 6.0.5 release. It is in 6.0.6.
[8 Jan 10:48] MC Brown
Internal change only. No documentation needed.