Bug #71825 All ndb nodes crash! Forced node shutdown completed. Occured during startphase 5
Submitted: 25 Feb 2014 2:12 Modified: 13 Aug 2018 18:32
Reporter: Kaiying Hu Email Updates:
Status: Unsupported Impact on me:
None 
Category:MySQL Cluster: NDB API Severity:S3 (Non-critical)
Version:7.3.3 OS:Linux
Assigned to: Bogdan Kecman CPU Architecture:Any

[25 Feb 2014 2:12] Kaiying Hu
Description:
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=13   @10.148.66.37  (mysql-5.6.15 ndb-7.3.4, starting, Nodegroup: 0)
id=14   @10.39.88.214  (mysql-5.6.14 ndb-7.3.3, starting, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @10.148.66.45  (mysql-5.6.14 ndb-7.3.3)

[mysqld(API)]   6 node(s)
id=21 (not connected, accepting connect from 10.148.66.37)
id=22 (not connected, accepting connect from 10.39.88.214)
id=23 (not connected, accepting connect from any host)
id=24 (not connected, accepting connect from any host)

ndb_mgm> Node 13: Forced node shutdown completed. Occured during startphase 5. Caused by error 2341: 'Internal program error (failed ndbrequire)(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.
Node 14: Forced node shutdown completed. Occured during startphase 5. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.

error log:
Time: Tuesday 25 February 2014 - 09:37:24
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: NdbcntrMain.cpp
Error object: NDBCNTR (Line: 2178) 0x00000006
Program: ndbd
Pid: 23650
Version: mysql-5.6.15 ndb-7.3.4
Trace: /usr/local/src/mysql/data/ndb_13_trace.log.3 [t1..t1]
***EOM***

ndb node 13(@10.148.66.37) is 32bit OS
ndb node 14(@10.39.88.214) and mgnt node are 64bit OS

MySQL Cluster version: mysql-cluster-gpl-7.3.3

config.ini

[ndbd default]
NoOfReplicas= 2

[ndb_mgmd]
nodeid=1
HostName= 10.148.66.45
datadir= /usr/local/src/mysql-mgm/data

[ndbd]
nodeid = 13
hostname=10.148.66.37
datadir=/usr/local/src/mysql/data

[ndbd]
nodeid = 14
hostname=10.39.88.214
datadir=/usr/local/src/mysql/data

[mysqld]
nodeid=21
ArbitrationRank=2
hostname=10.148.66.37

[mysqld]
nodeid=22
ArbitrationRank=2
hostname=10.39.88.214

[mysqld]

[mysqld]

my.cnf
[mysqld]
default-storage-engine=ndbcluster
ndbcluster
ndb-connectstring=10.148.66.45
basedir=/usr/local/src/mysql
datadir=/usr/local/src/mysql/data
socket=/tmp/mysql.sock
user=mysql
explicit_defaults_for_timestamp=true
port=3307
log-bin=/usr/local/src/mysql
binlog_format=mixed

[mysql_cluster]
ndb-connectstring=10.148.66.45

How to repeat:
Before, i installed on another servers(all servers are 64-bit system) with same configurations, that's ok. And this difference is one of servers is 32-bit system. Every time, when all ndb nodes will crash
[13 Aug 2018 18:32] Bogdan Kecman
Hi,

While I verified this behavior with 7.3.3 and 7.3.4 (crash when mixing 32bit and 64bit data nodes) I cannot reproduce it with 7.6.7 (latest GA at this moment).

I'd be tempted to tell you to upgrade, but we do not guarantee this type if mix will work so while I was not able to reproduce it with latest GA it's not a guarantee it will work in all conditions.

Having a management node and sql nodes on other architecture is ok but all your data nodes should use same architecture.

all best
Bogdan