Bug #17890 Storage node crashs after inserting data into table (saved on disk)
Submitted: 3 Mar 2006 9:23 Modified: 14 Mar 2006 10:05
Reporter: F Huber Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S1 (Critical)
Version:5.1.7 OS:Linux (Redhat ES 4 - x86_64)
Assigned to: Jonas Oreland CPU Architecture:Any

[3 Mar 2006 9:23] F Huber
Description:
I was creating tablespace for a Mysql Cluster to save tables to disk.

After a bunch of successful inserts (I was dumping a SQL File with single statements created by mysqldump) I got following error 

"ERROR 1297 (HY000) at line 1620988: Got temporary error 4010 'Node
failure caused abort of transaction' from NDBCLUSTER" 

I checked the cluster.log on the management node  - it shows that one of the two storage nodes crashed:

"Node 10: Forced node shutdown completed. Initiated
by signal 0. Caused by error 2341: 'Internal program error (failed ndbrequire)(Internal error
, programming error or missing error message, please report a bug). Temporary error, restart
node'."

How to repeat:
CREATE LOGFILE GROUP lg1
ADD UNDOFILE 'undofile.dat'
INITIAL_SIZE 400M
UNDO_BUFFER_SIZE = 40M
ENGINE = NDB;

CREATE TABLESPACE ts1
ADD DATAFILE 'datafile.dat'
USE LOGFILE GROUP lg1
INITIAL_SIZE 40000M
ENGINE NDB;

CREATE TABLE `testtable` (
 `ref` int(11) NOT NULL auto_increment,
 `refid` int(11) NOT NULL default '0',
 `refuserid` int(11) NOT NULL default '0',
 `time` datetime NOT NULL default '0000-00-00 00:00:00',
 `comments` int(11) NOT NULL default '0',
 PRIMARY KEY  (`ref`),
 KEY `refuserid_refid` (`refuserid`,`refid`)
)
TABLESPACE ts1 STORAGE DISK
ENGINE=NDBCLUSTER DEFAULT CHARSET=latin1; 

In our case the crash happend after 1.611.776 inserts.
[3 Mar 2006 10:26] Jonas Oreland
Hi, thx for the bug report.
This has alreay been fixed, (but not yet pushed/released)
(see bug http://bugs.mysql.com/bug.php?id=16657)

The problem is that you're using too much log buffer memory.
In 5.1.7 there will be trouble if your using more than 8M (which might be enought anyway)

So if your changing to 8M instead of 40M you should be fine.

A note is that given the size of your DB you should probably use the
parameter "DiskPageBufferMemory" to set a bigger value (default is 64M)

Let me know if decreasing the log buffer helps

/Jonas
[12 Mar 2006 12:51] Jonas Oreland
pushed into 5.1.8
(note see other bug report referenced)
[14 Mar 2006 10:05] Jon Stephens
Thank you for your bug report. This issue has been committed to our
source repository of that product and will be incorporated into the
next release.

If necessary, you can access the source repository and build the latest
available version, including the bugfix, yourself. More information 
about accessing the source trees is available at
    http://www.mysql.com/doc/en/Installing_source_tree.html

Additional info:

Documented in 5.1.8 changelog; closed.