Bug #40089 ndbd fails to start after hard kill due to corrupt Dbdict.
Submitted: 16 Oct 2008 20:19 Modified: 4 Oct 2009 12:58
Reporter: Matthew Montgomery Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:mysql-5.1-telco-7.0 OS:Linux
Assigned to: Assigned Account CPU Architecture:Any

[16 Oct 2008 20:19] Matthew Montgomery
Description:
After killing both ndbd nodes with "killall ndbd" command cluster nodes would not start without a initial system restart.  Both nodes were corrupted.  No schema operations or transactions were in progress at the time.

Complained of 

Time: Thursday 16 October 2008 - 14:32:36
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error
 message, please report a bug)
Error: 2341
Error data: dbdict/Dbdict.cpp
Error object: DBDICT (Line: 5327) 0x0000000a
Program: ndbd
Pid: 13465
Trace: /home/matt/mysql/sandbox/64/cluster/ndb_2_trace.log.4
Version: mysql-5.1.28 ndb-6.4.0-alpha
***EOM***

Another attempt with --initial restart of node 2. 
                            
Time: Thursday 16 October 2008 - 14:38:41
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error
 message, please report a bug)
Error: 2341
Error data: dbdict/Dbdict.cpp
Error object: DBDICT (Line: 5327) 0x0000000a
Program: ./bin/ndbd
Pid: 13650
Trace: /home/matt/mysql/sandbox/64/cluster/ndb_3_trace.log.4
Version: mysql-5.1.28 ndb-6.4.0-alpha
***EOM***

Traces to follow.

How to repeat:
Kill 6.4 hard with command line kill. 

Suggested fix:
?
[16 Oct 2008 20:25] MySQL Verification Team
Trace log file

Attachment: 40089_traces.tgz (application/x-compressed-tar, text), 104.22 KiB.

[4 Dec 2008 0:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[14 Jan 2009 21:13] MySQL Verification Team
Using version mysql-5.1.30 ndb-6.4.1 this is not reproduced.
[17 Feb 2009 16:31] Theodor Reppe
Here at mysql-5.1.31-ndb-6.4.1-innodb, I just experienced this bug again.
[17 Feb 2009 16:32] Theodor Reppe
Time: Tuesday 17 February 2009 - 17:25:47
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: dbdict/Dbdict.cpp
Error object: DBDICT (Line: 3967) 0x0000000a
Program: /usr/local/mysql//mysql/bin//ndbmtd
Pid: 3090
Trace: /var/lib/mysqlcluster//ndb_2_trace.log.4 /var/lib/mysqlcluster//ndb_2_trace.log.4_t1 /var/lib/mysqlcluster//ndb_2_trace.log.4_t2 /var/l
[17 Feb 2009 16:36] Theodor Reppe
my traces

Attachment: 40089_traces_theodor_reppe.tar.gz (application/x-gzip, text), 125.75 KiB.

[20 Feb 2009 10:08] Jonas Oreland
maybe related to bug#42962
[4 Sep 2009 12:55] Jonas Oreland
Hi,

I'm reasonably sure that this has been fixed with (numerous) bug fixes
made in the area.

Can you please retest with 7.0.7

/Jonas

ps. setting status to need feedback
[4 Oct 2009 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".