MySQL Bugs: #27979: Running sysbench (0.4.8) against NDB Cluster causes error 4350

Bug #27979	Running sysbench (0.4.8) against NDB Cluster causes error 4350
Submitted:	20 Apr 2007 11:52	Modified:	24 Sep 2010 11:32
Reporter:	Peter Welker	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S3 (Non-critical)
Version:	mysql-5.1	OS:	Linux (SLES10)
Assigned to:	Jonas Oreland	CPU Architecture:	Any
Tags:	5.1.17

Description:
If we run a 2/4/8 node (1 replica) cluster and try to issue sysbench complex OLTP test, we got the following error in all cases with more than 1 transaction:

sysbench --test=oltp --mysql-table-engine=ndbcluster --oltp-table-size=1000 --mysql-user=root --mysql-password=<removed> --mysql-socket=/u00/app/mysql/admin/mysqld5/socket/mysqld5.sock --oltp-test-mode=complex --max-requests=10000 --num-threads=8 run

...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
...

How to repeat:
Just use sysbench like described above. here is our log...

mysql@NODE01:~/local/sysbench/bin/ [mysqld5] ./sysbench --test=oltp --mysql-table-engine=ndbcluster --oltp-table-size=1000 --mysql-user=root --mysql-password=<removed> --mysql-socket=/u00/app/mysql/admin/mysqld5/socket/mysqld5.sock --verbosity=5 --debug=ON --oltp-test-mode=complex --max-requests=1000 --num-threads=8 prepare
sysbench v0.4.8:  multi-threaded system evaluation benchmark

No DB drivers specified, using mysql
Creating table 'sbtest'...
Creating 1000 records in table 'sbtest'...
mysql@NODE01:~/local/sysbench/bin/ [mysqld5] ./sysbench --test=oltp --mysql-table-engine=ndbcluster --oltp-table-size=1000 --mysql-user=root --mysql-password=<removed> --mysql-socket=/u00/app/mysql/admin/mysqld5/socket/mysqld5.sock --verbosity=5 --debug=ON --oltp-test-mode=complex --max-requests=1000 --num-threads=8 run
sysbench v0.4.8:  multi-threaded system evaluation benchmark

No DB drivers specified, using mysql
WARNING: Preparing of "BEGIN" is unsupported, using emulation
(last message repeated 7 times)
Running the test with following options:
Number of threads: 8

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 25
DEBUG: Runner thread started (0)!
DEBUG: Runner thread started (1)!
DEBUG: Runner thread started (2)!
Threads started!
DEBUG: Runner thread started (3)!
DEBUG: Runner thread started (4)!
DEBUG: Runner thread started (5)!
DEBUG: Runner thread started (6)!
DEBUG: Runner thread started (7)!
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
(last message repeated 2 times)
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
Done.

Thank you for the report.

Verified as described.

Hi,

I was watching push build today due to pushing in test changes I noticed that rpl_ndb_mix_innodb.test has been failing for a while now. The test fails with "could not sync with master ('select master_pos_wait('master-bin.000001', 228163)' returned NULL)"

Looking at it, the cause of the slave not syncing was due to the slave failing  with:

070607 22:13:52 [ERROR] Slave: Error in Update_rows event: row application failed, Error_code: 0
070607 22:13:52 [ERROR] Slave: Error in Update_rows event: error during transaction execution on table tpcb.branch, Error_code: 1105
070607 22:13:52 [Warning] Slave: Got error 4350 'Transaction already aborted' from NDB Error_code: 1296
070607 22:13:52 [Warning] Slave: Unknown error Error_code: 1105
070607 22:13:52 [Warning] Slave: Unknown error Error_code: 1105

I am experiencing the same errors with 5.1.22rc on a 4 node ndbcluster. Here's the test config:

sysbench --max-requests=1000 --test=oltp --mysql-user=USER --mysql-password=PASS --mysql-db=test --mysql-host='HOST' --num-threads=100 run

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 1000
Threads started!
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
FATAL: database error, exiting...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
FATAL: database error, exiting...
[...]
Done.

If I lower num-threads to, let's say, 10, it passes, but is also 3 times slower than doing the same test on an innodb table.

my guess is that has been fixed in 7.0 long time ago.
setting need feedback

Correct.

Problem is not repeatable anymore. Tested with 7.0.16