2006-12-18 16:40:50 [MgmSrvr] INFO     -- Node 2: We are Node 2 with dynamic ID 1, our left neighbour is Node 65535, our right is Node 65535
2006-12-18 16:40:50 [MgmSrvr] INFO     -- Node 2: Communication to Node 3 closed
2006-12-18 16:40:50 [MgmSrvr] ALERT    -- Node 2: Network partitioning - arbitration required
2006-12-18 16:40:50 [MgmSrvr] INFO     -- Node 2: President restarts arbitration thread [state=7]
2006-12-18 16:40:50 [MgmSrvr] INFO     -- Node 1: Node 3 Connected
2006-12-18 16:40:50 [MgmSrvr] ALERT    -- Node 3: Forced node shutdown completed. Occured during startphase 5. Initiated by signal 0. Caused 
by error 2310: 'Error while reading the REDO log(Ndbd file system inconsistency error, please report a bug). Ndbd file system error, restart 
node initial'.
2006-12-18 16:40:50 [MgmSrvr] ALERT    -- Node 2: Arbitration won - positive reply from node 1
2006-12-18 16:40:50 [MgmSrvr] INFO     -- Node 2: DICT: remove lock by failed node 3 for NodeRestart
2006-12-18 16:40:50 [MgmSrvr] INFO     -- Node 2: DICT: lock bs: 0 ops: 0 poll: 0 cnt: 0 queue: 
2006-12-18 16:40:51 [MgmSrvr] INFO     -- Node 2: Started arbitrator node 1 [ticket=04cc00249639529a]
2006-12-18 16:40:54 [MgmSrvr] INFO     -- Node 2: Communication to Node 3 opened
2006-12-18 16:41:36 [MgmSrvr] INFO     -- Mgmt server state: nodeid 3 reserved for ip 192.168.99.18, m_reserved_nodes 000000000000000a.
2006-12-18 16:41:36 [MgmSrvr] INFO     -- Node 1: Node 3 Connected
2006-12-18 16:41:36 [MgmSrvr] INFO     -- Node 3: Start phase 0 completed 
2006-12-18 16:41:36 [MgmSrvr] INFO     -- Node 3: Communication to Node 2 opened
2006-12-18 16:41:36 [MgmSrvr] INFO     -- Node 3: CM_REGREF from Node 3 to our Node 3. Cause = Election without selecting new candidate
2006-12-18 16:41:36 [MgmSrvr] INFO     -- Node 3: Initial start, waiting for 0000000000000004 to connect,  nodes [ all: 000000000000000c conn
ected: 0000000000000008 no-wait: 0000000000000000 ]
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 2: Node 3 Connected
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 2: We are Node 2 with dynamic ID 1, our left neighbour is Node 3, our right is Node 3
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 2: Node 3: API version 5.1.14
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 3: Node 2 Connected
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 3: CM_REGCONF president = 2, own Node = 3, our dynamic id = 37
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 3: We are Node 3 with dynamic ID 37, our left neighbour is Node 2, our right is Node 2
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 3: Node 2: API version 5.1.14
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 3: Start phase 1 completed 
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 3: Start phase 2 completed (initial node restart)
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 2: DICT: locked by node 3 for NodeRestart
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 2: DICT: lock bs: 4 ops: 0 poll: 0 cnt: 0 queue: 3L
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Mgmt server state: nodeid 3 freed, m_reserved_nodes 0000000000000002.
2006-12-18 16:41:37 [MgmSrvr] INFO     -- Node 3: Receive arbitrator node 1 [ticket=04cc00249639529a]
2006-12-18 16:41:40 [MgmSrvr] INFO     -- Node 3: Start phase 3 completed (initial node restart)
2006-12-18 16:41:44 [MgmSrvr] INFO     -- Node 3: Start phase 4 completed (initial node restart)

2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 3 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 10 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 11 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 13 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 14 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 16 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 17 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 18 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 19 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 21 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 23 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 25 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 27 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 29 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 31 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 33 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 35 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 37 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 39 activated
2006-12-18 16:41:50 [MgmSrvr] INFO     -- Node 3: DICT: index 41 activated
2006-12-18 16:41:51 [MgmSrvr] INFO     -- Node 2: Node: 3 StartLog: [GCI Keep: 241461 LastCompleted: 237156 NewestRestorable: 242760]
2006-12-18 16:41:51 [MgmSrvr] INFO     -- Node 2: Local checkpoint 193 started. Keep GCI = 242659 oldest restorable GCI = 237933
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Suma: asking node 2 to recreate subscriptions on me
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Suma: node 2 has completed restoring me
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Start phase 5 completed (initial node restart)
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Start phase 6 completed (initial node restart)
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Start phase 7 completed (initial node restart)
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 4 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 5 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 6 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 7 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 8 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 9 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 10 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 11 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Communication to Node 12 opened
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Start phase 8 completed (initial node restart)
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Start phase 9 completed (initial node restart)
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Start phase 100 completed (initial node restart)
2006-12-18 16:42:03 [MgmSrvr] INFO     -- Node 3: Node 1: API version 5.1.14

ndb_3_out.log :
2006-12-18 16:40:46 [ndbd] INFO     -- Error handler startup restarting system
2006-12-18 16:40:46 [ndbd] INFO     -- Error handler shutdown completed - exiting
2006-12-18 16:40:46 [ndbd] INFO     -- Angel received ndbd startup failure count 2.
2006-12-18 16:40:46 [ndbd] ALERT    -- Node 3: Forced node shutdown completed, restarting. Occured during startphase 5. Initiated by signal 0. Caused by error 2310: 'Error while reading the REDO log(Ndbd file system inconsistency error, please report a bug). Ndbd file system error, restart nod
2006-12-18 16:40:46 [ndbd] INFO     -- Ndb has terminated (pid 27710) restarting
2006-12-18 16:40:51 [ndbd] INFO     -- Angel pid: 27664 ndb pid: 27742
2006-12-18 16:40:51 [ndbd] INFO     -- NDB Cluster -- DB node 3
2006-12-18 16:40:51 [ndbd] INFO     -- Version 5.1.14 (beta) --
2006-12-18 16:40:51 [ndbd] INFO     -- Configuration fetched at 192.168.99.99 port 1186
2006-12-18 16:40:51 [ndbd] INFO     -- Start initiated (version 5.1.14)
2006-12-18 16:40:51 [ndbd] INFO     -- Ndbd_mem_manager::init(1) min: 20Mb initial: 20Mb
WOPool::init(61, 9)
RWPool::init(82, 13)
RWPool::init(a2, 18)
RWPool::init(c2, 13)
RWPool::init(122, 17)
RWPool::init(142, 15)
WOPool::init(41, 8)
RWPool::init(e2, 12)
RWPool::init(102, 51)
WOPool::init(21, 6)
RESTORE table: 0 1046 rows applied
RESTORE table: 0 1020 rows applied
RESTORE table: 1 14 rows applied
RESTORE table: 1 7 rows applied
RESTORE table: 2 0 rows applied
RESTORE table: 2 0 rows applied
RESTORE table: 4 12 rows applied
RESTORE table: 5 2 rows applied
RESTORE table: 5 7 rows applied
RESTORE table: 6 0 rows applied
RESTORE table: 6 0 rows applied
RESTORE table: 7 6 rows applied
RESTORE table: 7 4 rows applied
RESTORE table: 8 140 rows applied
RESTORE table: 8 190 rows applied
RESTORE table: 9 2085 rows applied
RESTORE table: 9 1995 rows applied
RESTORE table: 12 6 rows applied
RESTORE table: 12 5 rows applied
RESTORE table: 15 5 rows applied
RESTORE table: 15 6 rows applied
RESTORE table: 18 7 rows applied
RESTORE table: 18 3 rows applied
RESTORE table: 20 1 rows applied
RESTORE table: 20 1 rows applied
RESTORE table: 22 0 rows applied
RESTORE table: 22 2 rows applied
RESTORE table: 24 0 rows applied
RESTORE table: 24 3 rows applied
RESTORE table: 26 3 rows applied
RESTORE table: 26 5 rows applied
RESTORE table: 28 0 rows applied
RESTORE table: 28 0 rows applied
RESTORE table: 30 0 rows applied
RESTORE table: 30 0 rows applied
RESTORE table: 32 0 rows applied
RESTORE table: 32 0 rows applied
RESTORE table: 34 0 rows applied
RESTORE table: 34 0 rows applied
RESTORE table: 36 0 rows applied
RESTORE table: 36 0 rows applied
RESTORE table: 38 1 rows applied
RESTORE table: 38 2 rows applied
RESTORE table: 40 10 rows applied
RESTORE table: 40 10 rows applied
2006-12-18 16:41:05 [ndbd] INFO     -- Error handler startup restarting system
2006-12-18 16:41:06 [ndbd] INFO     -- Error handler shutdown completed - exiting
2006-12-18 16:41:06 [ndbd] INFO     -- Angel received ndbd startup failure count 3.
2006-12-18 16:41:06 [ndbd] ALERT    -- Ndbd has failed 3 consecutive startups. Not restarting
2006-12-18 16:41:06 [ndbd] ALERT    -- Node 3: Forced node shutdown completed. Occured during startphase 5. Initiated by signal 0. Caused by error 2310: 'Error while reading the REDO log(Ndbd file system inconsistency error, please report a bug). Ndbd file system error, restart node initial'.
2006-12-18 16:41:51 [ndbd] INFO     -- Angel pid: 27797 ndb pid: 27798
2006-12-18 16:41:51 [ndbd] INFO     -- NDB Cluster -- DB node 3
2006-12-18 16:41:51 [ndbd] INFO     -- Version 5.1.14 (beta) --
2006-12-18 16:41:51 [ndbd] INFO     -- Configuration fetched at 192.168.99.99 port 1186
2006-12-18 16:41:52 [ndbd] INFO     -- Start initiated (version 5.1.14)
2006-12-18 16:41:52 [ndbd] INFO     -- Ndbd_mem_manager::init(1) min: 20Mb initial: 20Mb
WOPool::init(61, 9)
RWPool::init(82, 13)
RWPool::init(a2, 18)
RWPool::init(c2, 13)
RWPool::init(122, 17)
RWPool::init(142, 15)
WOPool::init(41, 8)
RWPool::init(e2, 12)
RWPool::init(102, 51)
WOPool::init(21, 6)
NR: setLcpActiveStatusEnd - m_participatingLQH
table 4 options 0
table 5 options 0