Description:
I have a ndb cluster whit 2 ndbd node and 1 manager node ,it is running several weeks ,but today one of the ndbd service shutdown itself and I cannot start it ,please help me to find what I should do to start the ndb service,Thanks.
the error log is:
------------------------------ndb_1_cluster.log-----------
2018-05-22 17:28:35 [MgmtSrvr] INFO -- Node 2: NR Status: node=3,OLD=Allocated node id,NEW=Allocated node id
2018-05-22 17:28:35 [MgmtSrvr] INFO -- Alloc node id 3 succeeded
2018-05-22 17:28:35 [MgmtSrvr] INFO -- Nodeid 3 allocated for NDB at 10.10.134.69
2018-05-22 17:28:35 [MgmtSrvr] INFO -- Node 1: Node 3 Connected
2018-05-22 17:28:36 [MgmtSrvr] ALERT -- Node 3: Forced node shutdown completed. Occured during startphase 0. Initiated by signal 6. Caused by error 6000: 'Error OS signal received(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.
2018-05-22 17:28:36 [MgmtSrvr] ALERT -- Node 1: Node 3 Disconnected
2018-05-22 17:28:55 [MgmtSrvr] WARNING -- Node 2: Releasing node id allocation for node 3
------------------------------ndb_3_out.log-----------------
2018-05-22 17:26:29 [ndbd] INFO -- Angel pid: 17649 started child: 17650
2018-05-22 17:26:29 [ndbd] INFO -- Normal start of data node using checkpoint and log info if existing
2018-05-22 17:26:29 [ndbd] INFO -- Configuration fetched from '10.10.134.71:1186', generation: 1
2018-05-22 17:26:29 [ndbd] INFO -- Changing directory to '/mysqlCluster/mysql-cluster-gpl-7.5.7-linux-glibc2.12-x86_64/data'
2018-05-22 17:26:30 [ndbd] INFO -- MaxNoOfTriggers set to 142424
NDBMT: non-mt
2018-05-22 17:26:30 [ndbd] INFO -- NDB Cluster -- DB node 3
2018-05-22 17:26:30 [ndbd] INFO -- mysql-5.7.19 ndb-7.5.7 --
2018-05-22 17:26:30 [ndbd] INFO -- Memory Allocation for global memory pools Starting
2018-05-22 17:26:30 [ndbd] INFO -- numa_set_interleave_mask(numa_all_nodes) : OK
2018-05-22 17:26:30 [ndbd] INFO -- Ndbd_mem_manager::init(1) min: 23580Mb initial: 23708Mb
2018-05-22 17:26:30 [ndbd] INFO -- Touch Memory Starting, 32896 pages, page size = 32768
2018-05-22 17:26:30 [ndbd] INFO -- Touch Memory Completed
2018-05-22 17:26:30 [ndbd] INFO -- Adding 1028Mb to ZONE_LO (1,32895)
2018-05-22 17:26:30 [ndbd] INFO -- Memory Allocation for global memory pools Completed
2018-05-22 17:26:30 [ndbd] INFO -- Loading blocks for data node run-time environment
2018-05-22 17:26:30 [ndbd] INFO -- Starting Sending and Receiving services
2018-05-22 17:26:30 [ndbd] INFO -- Failed to set real-time prio on tid = 17660, error_no = 1
2018-05-22 17:26:30 [ndbd] INFO -- Received signal 6. Running error handler.
2018-05-22 17:26:30 [ndbd] INFO -- Signal 6 received; Aborted
2018-05-22 17:26:30 [ndbd] INFO -- /export/home/pb2/build/sb_0-23963488-1498206279.11/mysql-cluster-gpl-7.5.7/storage/ndb/src/kernel/ndbd.cpp
2018-05-22 17:26:30 [ndbd] INFO -- Error handler signal shutting down system
2018-05-22 17:26:30 [ndbd] INFO -- Error handler shutdown completed - exiting
2018-05-22 17:26:30 [ndbd] ALERT -- Node 3: Forced node shutdown completed. Occured during startphase 0. Initiated by signal 6. Caused by error 6000: 'Error OS signal received(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.
--------------------------------------ndb_3_error.log -----------------
Time: Tuesday 22 May 2018 - 16:02:37
Status: Temporary error, restart node
Message: Error OS signal received (Internal error, programming error or missing error message, please report a bug)
Error: 6000
Error data: Signal 6 received; Aborted
Error object: /export/home/pb2/build/sb_0-23963488-1498206279.11/mysql-cluster-gpl-7.5.7/storage/ndb/src/kernel/ndbd.cpp
Program: ndbd
Pid: 3267
Version: mysql-5.7.19 ndb-7.5.7
Trace file name: ndb_3_trace.log.25
Trace file path: /mysqlCluster/mysql-cluster-gpl-7.5.7-linux-glibc2.12-x86_64/data/ndb_3_trace.log.25 [t1..t1]
***EOM***
How to repeat:
I do not know
