Bug #69959 unable to start data node

Description:
I have a cluster with 4 data nodes, 2 replicas, 2 management nodes and 1 sql node.
I stopped a data node for a few days and continue running the cluster.
After that I tried to start the node and got this error:

 "Node 13: Forced node shutdown completed. Occured during startphase 5. Initiated by signal 11. Caused by error 6000: 'Error OS signal received(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'."

If I try to use ndbd --initial, it can start, but as far as I know, --initial will cause data losing, it not safe except some special case (ex upgrade version...).

Here's my config.ini:

[ndbd default]
# Options affecting ndbd processes on all data nodes:
datadir=/var/lib/mysql_ndbd/data   # Directory for this data node's data files
NoOfReplicas=2    # Number of replicas
DataMemory=2000M    # How much memory to allocate for data storage
IndexMemory=512M   # How much memory to allocate for index storage
                  # For DataMemory and IndexMemory, we have used the
                  # default values. Since the "world" database takes up
                  # only about 500KB, this should be more than enough for
                  # this example Cluster setup.
MaxNoOfTables=3000
MaxNoOfAttributes=30000
DiskPageBufferMemory=128M
MaxNoOfOrderedIndexes= 512
MaxNoOfConcurrentOperations=250000 # 800000
MaxNoOfLocalOperations=270000

[tcp default]
# TCP/IP options:
portnumber=2202   # This the default; however, you can use any
                  # port that is free for all the hosts in the cluster
                  # Note: It is recommended that you do not specify the port
                  # number at all and simply allow the default value to be used
                  # instead

[ndb_mgmd]
# Management process options:
NodeId=1
hostname=192.168.1.51           # Hostname or IP address of MGM node
datadir=/var/lib/mysql-cluster  # Directory for MGM node log files
LogDestination=FILE:filename=cluster.log,maxsize=1000000,maxfiles=6

[ndb_mgmd]
# Management process options:
NodeId=2
hostname=192.168.1.55           # Hostname or IP address of MGM node
datadir=/var/lib/mysql-cluster  # Directory for MGM node log files
LogDestination=FILE:filename=cluster.log,maxsize=1000000,maxfiles=6

[ndbd]
NodeId=11
NodeGroup=0
hostname=192.168.1.51           # Hostname or IP address

[ndbd]
NodeId=12
NodeGroup=0
hostname=192.168.1.52           # Hostname or IP address

[ndbd]
NodeId=13
NodeGroup=1
hostname=192.168.1.54           # Hostname or IP address

[ndbd]
NodeId=14
NodeGroup=1
hostname=192.168.1.59           # Hostname or IP address

[mysqld]
# SQL node options:
NodeId=41
hostname=192.168.1.51           # Hostname or IP address

[mysqld]
NodeId=42
hostname=192.168.1.56

[mysqld]

Does anyone has any idea of this? 
Thanks in advanced.

How to repeat:
Stop a data node for a few days.
Continue running the cluster (including import new data).
Start the node again.

Would be interesting to see a core dump for this ...

But if your node was down for several days already then doing an --initial restart doesn't harm, its local checkpoints will be so out of date by now that it would do a full sync anyway ...

Hi Hartmut,
Thanks for replying.
I'm gonna try to enable OS's core dump file then paste it in here.
By the way, may I ask if one day I have to use ndbd --initial for all data node, have I got any chance to get my data back?

If you do a full initial restart ... well, better have a recent backup that you can restore from then, and binlogs enabled on mysqld nodes so that you can do point-in-time recovery from these for the span between time of the backup and time when cluster stopped working ...

Hello Firzen,

Thank you for the report.
I couldn't reproduce the reported issue. Is this issue still repeatable(with latest GA 7.3.3)? 

Also, could you please attach the cluster logs? Preferably using the ndb_error_reporter utility:

  http://dev.mysql.com/doc/refman/5.5/en/mysql-cluster-programs-ndb-error-reporter.html

Thanks,
Umesh

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".