MySQL Bugs: #27979: Running sysbench (0.4.8) against NDB Cluster causes error 4350

Bug #27979	Running sysbench (0.4.8) against NDB Cluster causes error 4350
Submitted:	20 Apr 2007 13:52	Modified:	26 Apr 2007 20:48
Reporter:	Peter Welker
Status:	Verified
Category:	Server: Cluster	Severity:	S3 (Non-critical)
Version:	mysql-5.1	OS:	Linux (SLES10)
Assigned to:		Target Version:	CGE
Tags:	5.1.17
Triage:	Triaged: D3 (Medium)

Description:
If we run a 2/4/8 node (1 replica) cluster and try to issue sysbench complex OLTP test,
we got the following error in all cases with more than 1 transaction:

sysbench --test=oltp --mysql-table-engine=ndbcluster --oltp-table-size=1000
--mysql-user=root --mysql-password=<removed>
--mysql-socket=/u00/app/mysql/admin/mysqld5/socket/mysqld5.sock --oltp-test-mode=complex
--max-requests=10000 --num-threads=8 run

...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
...

How to repeat:
Just use sysbench like described above. here is our log...

mysql@NODE01:~/local/sysbench/bin/ [mysqld5] ./sysbench --test=oltp
--mysql-table-engine=ndbcluster --oltp-table-size=1000 --mysql-user=root
--mysql-password=<removed>
--mysql-socket=/u00/app/mysql/admin/mysqld5/socket/mysqld5.sock --verbosity=5 --debug=ON
--oltp-test-mode=complex --max-requests=1000 --num-threads=8 prepare
sysbench v0.4.8:  multi-threaded system evaluation benchmark

No DB drivers specified, using mysql
Creating table 'sbtest'...
Creating 1000 records in table 'sbtest'...
mysql@NODE01:~/local/sysbench/bin/ [mysqld5] ./sysbench --test=oltp
--mysql-table-engine=ndbcluster --oltp-table-size=1000 --mysql-user=root
--mysql-password=<removed>
--mysql-socket=/u00/app/mysql/admin/mysqld5/socket/mysqld5.sock --verbosity=5 --debug=ON
--oltp-test-mode=complex --max-requests=1000 --num-threads=8 run
sysbench v0.4.8:  multi-threaded system evaluation benchmark

No DB drivers specified, using mysql
WARNING: Preparing of "BEGIN" is unsupported, using emulation
(last message repeated 7 times)
Running the test with following options:
Number of threads: 8

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct
cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 25
DEBUG: Runner thread started (0)!
DEBUG: Runner thread started (1)!
DEBUG: Runner thread started (2)!
Threads started!
DEBUG: Runner thread started (3)!
DEBUG: Runner thread started (4)!
DEBUG: Runner thread started (5)!
DEBUG: Runner thread started (6)!
DEBUG: Runner thread started (7)!
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
(last message repeated 2 times)
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
DEBUG: ERROR: exiting mysql_drv_execute(), database error
DEBUG: ERROR: exiting db_execute(), driver's execute method failed
FATAL: database error, exiting...
Done.

Thank you for the report.

Verified as described.

Hi,

I was watching push build today due to pushing in test changes I noticed that
rpl_ndb_mix_innodb.test has been failing for a while now. The test fails with "could not
sync with master ('select master_pos_wait('master-bin.000001', 228163)' returned NULL)"

Looking at it, the cause of the slave not syncing was due to the slave failing  with:

070607 22:13:52 [ERROR] Slave: Error in Update_rows event: row application failed,
Error_code: 0
070607 22:13:52 [ERROR] Slave: Error in Update_rows event: error during transaction
execution on table tpcb.branch, Error_code: 1105
070607 22:13:52 [Warning] Slave: Got error 4350 'Transaction already aborted' from NDB
Error_code: 1296
070607 22:13:52 [Warning] Slave: Unknown error Error_code: 1105
070607 22:13:52 [Warning] Slave: Unknown error Error_code: 1105

I am experiencing the same errors with 5.1.22rc on a 4 node ndbcluster. Here's the test
config:

sysbench --max-requests=1000 --test=oltp --mysql-user=USER --mysql-password=PASS
--mysql-db=test --mysql-host='HOST' --num-threads=100 run

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct
cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Maximum number of requests for OLTP test is limited to 1000
Threads started!
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
FATAL: database error, exiting...
ALERT: failed to execute MySQL query: `BEGIN`:
ALERT: Error 1296 Got error 4350 'Transaction already aborted' from NDBCLUSTER
FATAL: database error, exiting...
[...]
Done.

If I lower num-threads to, let's say, 10, it passes, but is also 3 times slower than
doing the same test on an innodb table.