Bug #60654 Error data: Killed by node 3 as copyfrag failed, error: 1501
Submitted: 26 Mar 2011 10:47 Modified: 25 May 2012 10:23
Reporter: Rob Tousain Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S1 (Critical)
Version:mysql-5.1.30 ndb-6.3.20-GA OS:Linux (Redhat 5)
Assigned to: CPU Architecture:Any
Tags: Error data: Killed by node 3 as copyfrag failed, error: 1501

[26 Mar 2011 10:47] Rob Tousain
Description:

Hi,
my node 3 was down for 1 day. after restarting the clust won't start anymore.
Wat does the error means?
I als tried the ndbd --initial but the node won't restart.

Time: Saturday 26 March 2011 - 04:02:27
Status: Temporary error, restart node
Message: System error, node killed during node restart by other node (Internal e
rror, programming error or missing error message, please report a bug)
Error: 2303
Error data: Killed by node 3 as copyfrag failed, error: 1501
Error object: NDBCNTR (Line: 249) 0x0000000a
Program: /usr/sbin/ndbd
Pid: 26727
Trace: /data/mysql/ndb_3_trace.log.25
Version: mysql-5.1.30 ndb-6.3.20-GA
***EOM***

Didnt find any LCP for node: 3 tab: 278 frag: 1
execSTART_RECREQ chaning srnodes from 000000000000000c to 0000000000000008
Applying undo to LCP: 0
2011-03-26 11:11:57 [ndbd] INFO     -- Undo head - undo_02.dat page: 1 lsn: 0
2011-03-26 11:11:57 [ndbd] INFO     --    - next - undo_01.dat(0)
2011-03-26 11:11:57 [ndbd] INFO     -- Logfile group: 2
2011-03-26 11:11:57 [ndbd] INFO     --   head: undo_02.dat page: 1
2011-03-26 11:11:57 [ndbd] INFO     --   tail: undo_02.dat page: 1
2011-03-26 11:11:57 [ndbd] INFO     -- Flushing page cache after undo completion
2011-03-26 11:11:57 [ndbd] INFO     -- Flushing complete
changing file from 2154508 to 2154496
2011-03-26 11:31:41 [ndbd] INFO     -- Killed by node 3 as copyfrag failed, error: 1501
2011-03-26 11:31:41 [ndbd] INFO     -- NDBCNTR (Line: 249) 0x0000000a
2011-03-26 11:31:41 [ndbd] INFO     -- Error handler startup shutting down system
2011-03-26 11:31:41 [ndbd] INFO     -- Error handler shutdown completed - exiting
2011-03-26 11:31:41 [ndbd] INFO     -- Angel received ndbd startup failure count 1.
2011-03-26 11:31:41 [ndbd] ALERT    -- Node 3: Forced node shutdown completed. Occured during startphase 5. Caused by error 2303: 'System error, node killed during node restart by other node(Internal error, programming error or missing error message, please report a bug). Temporary error, rest

How to repeat:
ndbd or ndbd --initial

Suggested fix:
?
[25 May 2012 10:23] Gustaf Thorslund
See:
  http://bugs.mysql.com/bug.php?id=50979
Setting as duplicate of bug 50979