Bug #52055 | NDB node shutdown and NDB misbehavior version 7.0.7 | ||
---|---|---|---|
Submitted: | 15 Mar 2010 8:34 | Modified: | 24 May 2010 20:13 |
Reporter: | Rosario Arun | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Cluster: Cluster (NDB) storage engine | Severity: | S1 (Critical) |
Version: | mysql-5.1-telco-7.0 | OS: | Linux (SUSE Enterprise Linux) |
Assigned to: | Jonas Oreland | CPU Architecture: | Any |
Tags: | database import issue version 7.0.7, ndbd node shutdown |
[15 Mar 2010 8:34]
Rosario Arun
[16 Mar 2010 13:52]
Gustaf Thorslund
Hi Rosario, Could you please provide the full ndb_error_reporter output as described in: http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-programs-ndb-error-reporter.html If you upload it to our ftp server please put 52055 somewhere in the filename. /Gustaf
[18 Mar 2010 15:17]
Rosario Arun
ndb_error_reporter
Attachment: 52055__ndb_error_report_20100318180120.tar.bz2 (application/octet-stream, text), 154.58 KiB.
[18 Mar 2010 15:32]
Rosario Arun
Error log or all the nodes Complete version
Attachment: 52055_ALLNODES_ndb_error_report_20100318181802.tar.bz2 (application/octet-stream, text), 256.85 KiB.
[18 Mar 2010 15:33]
Rosario Arun
Hi Gustaf Thorslund , Thanks for your response. I have uploaded the error dump from ndb_error_reporter. The second file 52055_ALLNODES_ndb_error_report_20100318181802.tar.bz2 holds the complete dump. Thanks in advance.
[3 May 2010 12:23]
Jonas Oreland
Hi, 1) Can you please try with a newer version, e.g 7.0.14 or 7.1.3 2) If this still fails, can you please upload the schema definition. 3) Also, have you upgrade from 6.3 ? /Jonas
[11 May 2010 6:31]
Geert Vanderkelen
This is repeatable with MySQL Cluster 7.1.3 (so I guess also with 7.0.14). * Creating 300 tables, failed with out of attributes * Increasing MaxNoAttributes to 10000 * Failure after 107 where created, error 2301 * Cleaning up everything i.e. removing all ndb related files and MySQL's datadir * Start again: error 136 No more room in index file * Reconfiguring to give more index memory * Again, same error 2301 with trace here below Configuration: [ndbd default] NoOfReplicas = 1 datamemory = 200M indexmemory = 70M datadir = /opt/mysql/clusters/machA/ndb MaxNoOfAttributes=10000 MaxNoOfConcurrentOperations=500000 MaxNoOfConcurrentTransactions=20000 [ndb_mgmd default] datadir = /opt/mysql/clusters/machA/ndb [ndb_mgmd] hostname = localhost PortNumber = 1186 id = 1 [ndbd] hostname = localhost id = 2 [api] id = 20 [api] id = 21 [api] id = 22 Attaching file which has error and trace.
[11 May 2010 6:34]
Geert Vanderkelen
Just a FYI, the Python script creating a rather basic table 319 times: import mysql.connector as db def main(): cnx = db.connect(user='root',db='test') cur = cnx.cursor() create = "CREATE TABLE t%d (id INT UNSIGNED NOT NULL AUTO_INCREMENT,"\ " c1 INT, c2 INT, c3 VARCHAR(400), c4 VARCHAR(200), c5 INT, c6 BIT,"\ "PRIMARY KEY (id)) ENGINE=ndb" for i in xrange(1,320): cur.execute("DROP TABLE IF EXISTS t%d" % i) cur.execute(create % i) cur.close() cnx.close()
[11 May 2010 7:05]
Geert Vanderkelen
Yes, repeatable: 1) Make sure you create enough tables so error "error code 136: No more room in index file" is generated. This can be done with the above configuration 2) Restart Cluster 3) Try again recreating these tables. 4) Data Node shuts down.
[11 May 2010 7:13]
Geert Vanderkelen
Seems not to be related to setting MaxNoOfOrderedIndexes higher (defaults to 128). Setting it to 500 crashes the data node as well, at creation of table t217 or t128.
[24 May 2010 8:14]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/108999 3544 Jonas Oreland 2010-05-24 ndb - bug#52055 - fix problem where index could get too high id, causing assertion
[24 May 2010 8:17]
Jonas Oreland
pushed to 7.0.15 and 7.1.4 DOCS: when creating an index, code failed to check that id allocate was within range of possible id's for an index, causing assertion
[24 May 2010 20:13]
Jon Stephens
Documented bugfix in the NDB-7.0.15 and 7.1.4 changelogs as follows: When creating an index, NDB failed to check whether the internal ID allocated to the index was within the allowable range, leading to an assertion. This issue could manifest itself as a data node failure with NDB error 707 (No more table metadata records (increase MaxNoOfTables)), when creating tables in rapid succession (for example, by a script, or when importing from mysqldump), even with a relatively high value for MaxNoOfTables and a relatively low number of tables. Closed.