MySQL Bugs: #38254: Falcon dead lock for sysbench read-only test with the newest lock implementation

Bug #38254	Falcon dead lock for sysbench read-only test with the newest lock implementation
Submitted:	21 Jul 2008 9:31	Modified:	16 Mar 2009 22:46
Reporter:	kai xiang	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S2 (Serious)
Version:	mysql-6.0-falcon bzr	OS:	Linux (64-bit)
Assigned to:	Kelly Long	CPU Architecture:	Any
Tags:	F_PERFORMANCE

Description:
When compile mysql-6.0-falcon at rev.2719, we use the FAST_SHARED in the SyncObject.cpp #50

with the fast locking, running sysbench with read-only test, the system going to a state of never ending deadlocks with all the CPU idle but not finishing the job any more.

How to repeat:
use #define FAST_SHARED in SyncObject.cpp #50
complile the souce code and use sysbench kit running read only test:

prepare:
sysbench --test=oltp --mysql-table-engine=falcon --mysql-engine-trx=yes --mysql-user=mysql --mysql-password=xxxxxx --oltp-table-size=1000000 --mysql-socket=/tmp/mysql.sock prepare

run:
sysbench --num-threads=8 --max-time=200 --max-requests=1000000000 --mysql-engine-trx=yes --mysql-table-engine=falcon --test=oltp --oltp-read-only=on --mysql-user=mysql --mysql-password=xxxxxx --oltp-table-size=1000000 --mysql-socket=/tmp/mysql.sock run

Kelly, this problem is preventing us from turning on this new performance enhancement, which Intel has shown really works.  Intel indicates that the enhancement increases SyncTest performance 2x or 3x depending on the client load and processor.  SyncTest is nothing but Read-Read concurrency on a single SyncObject, but since locking SyncObjects can be as much as 30% of a DBT2 run, this improvement will give us measurable improvement...  if we can solve this deadlock.

I tried FAST_SHARED enabled with
revno: 2758
committer: Hakan Kuecuekyilmaz <hakan@mysql.com>
branch nick: mysql-6.0-falcon-to-merge
timestamp: Mon 2008-07-21 11:56:25 +0200

=== modified file 'storage/falcon/SyncObject.cpp'
--- storage/falcon/SyncObject.cpp       2008-07-08 16:01:41 +0000
+++ storage/falcon/SyncObject.cpp       2008-07-21 22:37:18 +0000
@@ -47,7 +47,7 @@
 #include "Stream.h"
 #include "InfoTable.h"

-//#define FAST_SHARED
+#define FAST_SHARED
 //#define STALL_THRESHOLD      1000

 #define BACKOFF        \

After compiling with BUILD/compile-amd64-max-no-ndb mysqld does not even start.

I will attach a full backtrace of all threads.

thread apply all backtrace full

Attachment: bug_38254_thread_apply_all_bt_full.txt (text/plain), 22.67 KiB.

WL#4611 tracks the implementation of the FAST_SHARED method of implementing SyncObjects.  This task is now complete, without the deadlock mentioned in this bug.  And it is being tested in the Performance tree.  This code was initially added to the Falcon tree but never turned on or used.  So the deadlock never actually occurred in a released Falcon engine.