MySQL Bugs: #48045: UNIQUE KEY cause system to crash

Bug #48045	UNIQUE KEY cause system to crash
Submitted:	14 Oct 2009 15:27	Modified:	21 Jan 2010 12:12
Reporter:	Marcus ROTSEN	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S2 (Serious)
Version:	7.0.7 \| 7.0.8	OS:	Linux (64bits red hat version binaries)
Assigned to:	Assigned Account	CPU Architecture:	Any
Tags:	Can't create table, Error: 2301, unique key

Description:
I have an annoying problem when I want to create another copy of my database on the same system.

I updated my cluster configuration to allow a very large amount of index, tables etc but I still have the problem.
I can drop my initial database then recreate from a dump but when I want to use my dump to create a new copy I have the following error:

ERROR 1005 (HY000) at line 844: Can't create table 'database.table' (errno: 4247)

It seems that UNIQUE KEY are responsible for this error. When I use the dump without
UNIQUE KEY it works very well.

In 7.0.5 the whole cluster crashed every time I tried to create the copy. I've updated to 7.0.7 and now only 1 data node crash.

Cluster overview (pre production system):

[ndbd(NDB)]	2 node(s)
id=3	@10.0.0.109  (mysql-5.1.35 ndb-7.0.7, Nodegroup: 0, Master)
id=4	@10.0.0.110  (mysql-5.1.35 ndb-7.0.7, Nodegroup: 0)

[ndb_mgmd(MGM)]	2 node(s)
id=1	@10.0.0.105  (mysql-5.1.35 ndb-7.0.7)
id=2	@10.0.0.106  (mysql-5.1.35 ndb-7.0.7)

[mysqld(API)]	2 node(s)
id=5	@10.0.0.107  (mysql-5.1.35 ndb-7.0.7)
id=6	@10.0.0.108  (mysql-5.1.35 ndb-7.0.7)

Here is my configuration:

[ndbd default]
NoOfReplicas=2 
DataMemory=200M 
IndexMemory=100M
BackupDataDir=/usr/local/mysql/backups
MaxNoOfExecutionThreads=2
MaxNoOfUniqueHashIndexes=20000
MaxNoOfOrderedIndexes=20000
MaxNoOfAttributes=20000
datadir=/usr/local/mysql/data
StringMemory=20
MaxNoOfTables=10000
MaxNoOfTriggers=50000

I think I have the same problem that this issue: http://bugs.mysql.com/bug.php?id=46295

The database is not very big (100 tables) and approximatively 3 UNIQUE KEY per table.

The ouput of one of the data node:

Time: Tuesday 13 October 2009 - 12:22:32
Status: Temporary error, restart node
Message: Assertion (Internal error, programming error or missing error message, please
report a bug)
Error: 2301
Error data: ArrayPool<T>::getPtr
Error object: ../../../../../storage/ndb/src/kernel/vm/ArrayPool.hpp line: 422 (block:
DBDICT)
Program: ndbmtd
Pid: 1294
Trace: /usr/local/mysql/data/ndb_3_trace.log.17
/usr/local/mysql/data/ndb_3_trace.log.17_t1 /usr/local/mysql/data/ndb_3_trace.log.17_t2
/usr/local/mysql 

How to repeat:
Use the dump file for at least 2 databases with the same cluster configuration.

Suggested fix:
unknow

Marcus,

Could you please provide the output from ndb_error_reporter?

/Gustaf

I sent the archive of the ndb reporter script a few weeks ago.
Do you have some an idea of where this problem come from ?

My cluster is running on 7.0.8A version now and I still have the problem.

Hi,

I tried to load your dump into two databases wo/ problem.
I tried with you config, on both 7.0.5 and 7.0.9
I didn't find any logs(traces) from a crashing 7.0.8a, only from 7.0.5
I tried with a clean cluster every time (i.e started with --initial)

Only think I can think of is if you have upgraded somehow.

Setting status to need-feedback, since I can't repeat

Hello,
I just tried again with 7.0.8A.
I started my storage nodes with --initial option like you.
I used the same database structure.

The first time worked well, but at the second try I got the same error and my master storage node crashed.

This is the logs from this node (node #3):
Time: Monday 16 November 2009 - 16:05:15
Status: Temporary error, restart node
Message: Assertion (Internal error, programming error or missing error message, please report a bug)
Error: 2301
Error data: ArrayPool<T>::getPtr
Error object: ../../../../../storage/ndb/src/kernel/vm/ArrayPool.hpp line: 422 (block: DBDICT)
Program: ndbmtd
Pid: 27562 thr: 0
Version: mysql-5.1.37 ndb-7.0.8a
Trace: /usr/local/mysql/data/ndb_3_trace.log.22 /usr/local/mysql/data/ndb_3_trace.log.22_t1 /usr/local/mysql/da
Time: Monday 16 November 2009 - 16:09:35
Status: Temporary error, restart node
Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error)
Error: 2308
Error data: Node 4 disconnected
Error object: QMGR (Line: 2971) 0x0000000e
Program: ndbmtd
Pid: 16934 thr: 0
Version: mysql-5.1.37 ndb-7.0.8a
Trace: /usr/local/mysql/data/ndb_3_trace.log.23 /usr/local/mysql/data/ndb_3_trace.log.23_t1 /usr/local/mysql/data/ndb_3_trace.log.23_t2 /usr/local/mysql
Time: Monday 16 November 2009 - 16:21:20
Status: Temporary error, restart node
Message: Assertion (Internal error, programming error or missing error message, please report a bug)
Error: 2301
Error data: ArrayPool<T>::getPtr
Error object: ../../../../../storage/ndb/src/kernel/vm/ArrayPool.hpp line: 422 (block: DBDICT)
Program: ndbmtd
Pid: 16998 thr: 0
Version: mysql-5.1.37 ndb-7.0.8a
Trace: /usr/local/mysql/data/ndb_3_trace.log.24 /usr/local/mysql/data/ndb_3_trace.log.24_t1 /usr/local/mysql/da

And this is what I got in the management node console:

Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)]	2 node(s)
id=3	@10.0.0.109  (mysql-5.1.37 ndb-7.0.8, Nodegroup: 0, Master)
id=4	@10.0.0.110  (mysql-5.1.37 ndb-7.0.8, Nodegroup: 0)

[ndb_mgmd(MGM)]	2 node(s)
id=1	@10.0.0.105  (mysql-5.1.37 ndb-7.0.8)
id=2	@10.0.0.106  (mysql-5.1.37 ndb-7.0.8)

[mysqld(API)]	2 node(s)
id=5	@10.0.0.107  (mysql-5.1.37 ndb-7.0.8)
id=6	@10.0.0.108  (mysql-5.1.37 ndb-7.0.8)

ndb_mgm> Node 3: Forced node shutdown completed. Caused by error 2301: 'Assertion(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.

Maybe I can try with 7.0.9, is there any way to try it ?

version 7.0.8 has the same problem

src release of 7.0.10 is available at ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/mysql-5.1.39-ndb-7.0.10/mysql-cluster...

if still encountering same problem, be sure to upload new logs/traces/etc.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".