MySQL Bugs: #39096: Creating tablespace with high extent size segfaults data nodes

Bug #39096	Creating tablespace with high extent size segfaults data nodes
Submitted:	28 Aug 2008 14:46	Modified:	19 Feb 2009 7:49
Reporter:	Geert Vanderkelen	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Disk Data	Severity:	S2 (Serious)
Version:	mysql-5.1.24-ndb-6.3.15	OS:	Any
Assigned to:	Jonas Oreland	CPU Architecture:	Any
Tags:	disk-based, segfault, Tablespace

Description:
Creating a rather big tablespace crashes all data nodes with signal 11. Trying to get the data nodes up again doesn't work, failing with error 2341 in DICT (see How To Repeat). It works with setting the tablespace smaller.

How to repeat:
mysql> CREATE LOGFILE GROUP lg_1 ADD UNDOFILE 'undo_1.dat' INITIAL_SIZE 512M UNDO_BUFFER_SIZE 32M ENGINE NDB;

(Note: had to set the SharedGlobalMemory higher, 600M, to let the above work, but size of logfile group is irrelevant)

mysql> CREATE TABLESPACE ts_1 ADD DATAFILE 'data_1.dat' USE LOGFILE GROUP lg_1 INITIAL_SIZE 1G EXTENT_SIZE 1G ENGINE NDB;
mysql> show warnings;
+-------+------+-------------------------------------------+
| Level | Code | Message                                   |
+-------+------+-------------------------------------------+
| Error | 1296 | Got error 4009 'Cluster Failure' from NDB |
| Error | 1528 | Failed to create DATAFILE                 |
+-------+------+-------------------------------------------+

Trying to get data nodes back up gives following error:

Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: dbdict/Dbdict.cpp
Error object: DBDICT (Line: 3608) 0x0000000a
Program: /data1/mysql/ndb-6.3.15/libexec/ndbd
Pid: 9409
Trace: /data2/users/geert/cluster/master/ndb_3_trace.log.1
Version: mysql-5.1.24 ndb-6.3.15-RC

Suggested fix:
Shouldn't segfault all data nodes, and at least make it possible to restart them.

Actually, It's the extend size that is the problem apparently:

mysql> CREATE TABLESPACE ts_3 ADD DATAFILE 'data_3.dat' USE LOGFILE GROUP lg_1 INITIAL_SIZE 1024M EXTENT_SIZE 100M ENGINE NDB;

mysql> CREATE TABLESPACE ts_4 ADD DATAFILE 'data_4.dat' USE LOGFILE GROUP lg_1 INITIAL_SIZE 1024M EXTENT_SIZE 1024M ENGINE NDB;
..
| Error | 1296 | Got error 4009 'Cluster Failure' from NDB |

Uploaded logs for the error after it happens

Log files and traces showing the error after trying to start data nodes

Attachment: ndb_error_report_20080828173911.tar.bz2 (, text), 230.94 KiB.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/66807

2844 Jonas Oreland	2009-02-18
      ndb - bug#39096 - fix handling of last extents

Pushed into 5.1.32-ndb-6.2.17 (revid:jonas@mysql.com-20090218215620-cnwqx2krulgxsnt1) (version source revid:jonas@mysql.com-20090218215620-cnwqx2krulgxsnt1) (merge vers: 5.1.32-ndb-6.2.17) (pib:6)

Pushed into 5.1.32-ndb-6.4.3 (revid:jonas@mysql.com-20090218220511-bgnaexvwjjfq2g6w) (version source revid:jonas@mysql.com-20090218215910-abp2s33dok9kh1g8) (merge vers: 5.1.32-ndb-6.4.3) (pib:6)

Pushed into 5.1.32-ndb-6.3.23 (revid:jonas@mysql.com-20090218220353-ih9lxz0jg5od9k2c) (version source revid:jonas@mysql.com-20090218215746-gseg9gxs7zcaqt4s) (merge vers: 5.1.32-ndb-6.3.23) (pib:6)

Documented bugfix in the NDB-6.2.17, 6.3.22, and 6.4.3 changelogs as follows:

        Creating a Disk Data tablespace with a very large extent size
        caused the data nodes to fail. The issue was observed when using
        extent sizes of 100 MB and larger.