MySQL Bugs: #62631: Query crashed all disk nodes; data was corrupted upon restart

Bug #62631	Query crashed all disk nodes; data was corrupted upon restart
Submitted:	5 Oct 2011 11:49	Modified:	10 Oct 2011 13:12
Reporter:	Steve Brown	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S1 (Critical)
Version:	7.1.15	OS:	Linux (Debian Squeeze 64bit)
Assigned to:		CPU Architecture:	Any
Tags:	cluster, crash, data corruption

Description:
Running a fairly extensive query which works fine on a single MyISAM server took much longer to run on the (unloaded) cluster with the same data set. When the query was killed the entire cluster crashed.

The cluster restarted ok, but the database was corrupted in that inserts to a 5 or 6 tables (out of 150) resulted in oddness, such as 'cannot retrieve auto-increment value) type errors.

Reloading from a mysqldump backup failed with similar errors dispute the backup having the usual 'drop table blah' statements.

The only fix was to recreate the cluster from scratch, and then the reload from backup was fine. Downtime for the (limited) production cluster was 4 hours.

error report attached.

How to repeat:
Not too sure, but willing to run test cases as required.

D'oh. File attachment limit of 500k.

Uploaded ndb_error_report_20111005121142.tar.bz2
to ftp.oracle.com/support/incoming

Edited title, query actually wasn't 'simple' ;)

Short verification question: You're using 7.1.15a, right ?
Cause 7.1.15 contained a memory corruption bug...

and FYI: I don't (yet) know how to get files from ftp.oracle.com :-(

Hey Jonas,

Actually I wasn't using 7.1.15a so I've just updated to .15a (the rolling update of which caused all manner of amusement with nodes crashing all over the place, resulting in a --initial and restore, but that's another story!)

I guess we can close this ticket and I'll raise any further issues in new tickets.

Good luck finding the credentials for your FTP server!

re credentials: thx :-)

/Jonas