Bug #44396 NDBD / NDBMTD would not start
Submitted: 22 Apr 2009 0:32 Modified: 23 Apr 2009 9:12
Reporter: Phil Bayfield Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S1 (Critical)
Version:7.0.5 OS:Linux
Assigned to: Assigned Account CPU Architecture:Any
Tags: ndbd, ndbmtd

[22 Apr 2009 0:32] Phil Bayfield
Description:
Following upgrade from 6.3.23 to 7.0.5 ndbd and ndbmtd would not start on the data nodes.

A full shutdown of the cluster was performed prior to upgrade due to advertised issues with rolling upgrade.

Log from NDB server:

Time: Tuesday 21 April 2009 - 23:28:15
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: dbdict/Dbdict.cpp
Error object: DBDICT (Line: 25084) 0x0000000a
Program: ndbmtd
Pid: 7600
Trace: /var/lib/mysql-cluster/ndb_3_trace.log.11 /var/lib/mysql-cluster/ndb_3_trace.log.11_t1 /var/lib/mysql-cluster/ndb_3_trace.log.11_t2 /var/lib/mysql-cluster/ndb_3_trace
Time: Tuesday 21 April 2009 - 23:30:09
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: dbdict/Dbdict.cpp
Error object: DBDICT (Line: 25084) 0x0000000a
Program: ndbd
Pid: 7652
Trace: /var/lib/mysql-cluster/ndb_3_trace.log.12
Version: mysql-5.1.32 ndb-7.0.5-beta

Will attach trace logs shortly.

How to repeat:
Run either ndbd or ndbmtd
[22 Apr 2009 0:37] Phil Bayfield
Trace logs for 3 separate failed starts.

Attachment: Trace.zip (application/zip, text), 310.67 KiB.

[22 Apr 2009 0:47] Phil Bayfield
Following downgrade to 6.3.24 cluster started normally without issue.
[22 Apr 2009 13:48] Jonas Oreland
Hi,

This is a duplication of
http://bugs.mysql.com/bug.php?id=44294

/Jonas
[22 Apr 2009 15:26] Phil Bayfield
Is an initial start required to upgrade to 7.0.5 at this time?
[23 Apr 2009 0:28] Jonas Oreland
yes, until we release 7.0.6, sorry
/Jonas
[23 Apr 2009 9:12] Phil Bayfield
Ahh I see, I had assumed that as we could not perform an online upgrade that it would work if we simply shutdown the cluster completely, upgraded and restarted.

Thanks for you help Jonas, we'll wait for the next release.