Bug #40229 Cluster Node Start Fail - err 2311 - Internal Error
Submitted: 22 Oct 2008 4:39 Modified: 1 Dec 2010 13:48
Reporter: James Knight Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:5.0.67 OS:Linux (CentOs 5.2)
Assigned to: Jonas Oreland CPU Architecture:Any
Tags: 2311, Internal error, selecting restart type, startphase 1, Temporary error
Triage: Needs Triage: D2 (Serious)

[22 Oct 2008 4:39] James Knight
Description:
After attempting to start a cluster node, the node prompts with the following error and then terminates:
 "Node 3: Forced node shutdown completed. Occured during startphase 1. Caused by error 2311: 'Conflict when selecting restart type(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'."

At one point the cluster appeared to be completely operational - after a series of simulated power downs (force quit on the machines) the error appeared.

[ndb_3_out]
---
2008-10-21 16:05:07 [ndbd] INFO     -- Angel pid: 2500 ndb pid: 2501
2008-10-21 16:05:07 [ndbd] INFO     -- NDB Cluster -- DB node 3
2008-10-21 16:05:07 [ndbd] INFO     -- Version 5.0.67 --
2008-10-21 16:05:07 [ndbd] INFO     -- Configuration fetched at 192.168.1.161 port 1186
2008-10-21 16:05:08 [ndbd] INFO     -- Start initiated (version 5.0.67)
2008-10-21 16:05:41 [ndbd] INFO     -- Error handler startup shutting down system
2008-10-21 16:05:45 [ndbd] INFO     -- Angel received ndbd startup failure count 1.
2008-10-21 16:05:45 [ndbd] INFO     -- Error handler shutdown completed - exiting
2008-10-21 16:05:45 [ndbd] ALERT    -- Node 3: Forced node shutdown completed. Occured during startphase 1. Caused by error 2311: 'Conflict when selecting restart type(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.
2008-10-21 16:05:54 [ndbd] WARNING  -- Unable to report shutdown reason to 192.168.1.160:1186: Could not connect to socket : Unable to connect with connect string: nodeid=0,192.168.1.160:1186
---

[ndb_3_error.log]
---
Current byte-offset of file-pointer is: 568                       

Time: Tuesday 21 October 2008 - 16:05:41
Status: Temporary error, restart node
Message: Conflict when selecting restart type (Internal error, programming error or missing error message, please report a bug)
Error: 2311
Error data: Unable to start missing node group!  starting: 0000000000000008 (missing fs for: 0000000000000000)
Error object: QMGR (Line: 1402) 0x0000000a
Program: ndbd
Pid: 2501
Trace: /var/lib/mysql-cluster/ndb_3_trace.log.1
Version: Version 5.0.67
***EOM***
                   
---

[ndb_3_trace.log.next]
---
1
---

Only SQL data is default with the following commands:
---
DELETE FROM mysql.user WHERE User='';
DELETE FROM mysql.user WHERE User='root' AND Host != 'localhost';
DROP DATABASE test;
DELETE FROM mysql.db WHERE Db = 'test' OR Db = 'test\\_%';
FLUSH PRIVILEGES;
---

How to repeat:
Unknown.

Suggested fix:
N/A.
[22 Oct 2008 4:43] James Knight
Trace log, cluster management configuration and node configuration.

Attachment: bug-data-20229.zip (application/zip, text), 23.55 KiB.

[6 Nov 2008 7:37] Guangbao Ni
Hi James,

Is there a lot of data in table?
Is the ndbd restarted with --initial option?

I can't reproduce it.

/Guangbao
[6 Nov 2008 19:27] James Knight
Guangbao,

There is no data, other than what is given when mySQL is setup, present in the tables.

The error message (error 2311) appears only when attempting to start the nodes with no arguments. If I apply the '--initial' option on all nodes, they start up with no signs of issues.
[27 Nov 2008 7:40] Guangbao Ni
Hi James,

Please attach your config.ini file.

From source codes, seems that it can't elect the president.

/Guangbao
[28 Dec 2008 0:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[17 Nov 2010 15:16] Martin Herbort
Hello,

I have exactly the same issue, using MySQL Cluster 5.0.46.

The config.ini which was requested was already attached by James Knight on 22 Oct 2008 6:43.

So, please continue to investigate this issue or state with what MySQL Cluster version it was fixed.

Thanks,

Martin
[1 Dec 2010 13:48] Jonas Oreland
Hi,

5.0.67 is very very old and has not seen any bug fixes in a long time.
As for current releases, we fixed many many bugs in 7.1 since 5.0
and it's not practical to check all too see when this was fixed.

I suggest you try with latest 7.1, and if you can still repeat the problem
please reopen bug (or a new one) and upload new tracefiles/config.ini + description.

Setting status to Can't repeat

/Jonas
[12 Jan 2012 2:00] Alexandru Thirtheu
Hello,

I am experiencing the the same error with MySQL Cluster version 7.1.18, 64-bit.
One of the NDBD nodes starts up, but shuts down a few seconds later (2-5 s) with error 2311. I am using two Debian 6.0.3 amd64, each hosting one mgmd, one ndbd, and one mysqld.
I would upload any file needed, but I don't know where the log files are located on Linux.

I am also posing on http://forums.mysql.com/read.php?10,507734 and http://forums.techguy.org/linux-unix/1034338-mysql-cluster-sartup-failure.html, where I've provided other details.