10-08-24 10:32:30 [MgmtSrvr] INFO     -- Node 3: Data usage is 34%(902223 32K pages of total 2624000)
2010-08-24 10:32:30 [MgmtSrvr] INFO     -- Node 3: Index usage is 31%(302361 8K pages of total 960128)
2010-08-24 10:32:41 [MgmtSrvr] WARNING  -- Node 4: Node 3 missed heartbeat 2
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 8: Data usage is 34%(902403 32K pages of total 2624000)
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 8: Index usage is 31%(302274 8K pages of total 960128)
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 7: Node 3 Disconnected
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 1: Node 3 Disconnected
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 6: Node 3 Disconnected
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 4: Communication to Node 3 closed
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 4: Arbitration check won - node group majority
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 4: President restarts arbitration thread [state=6]
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 4: GCP Take over started
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 4: Node 4 taking over as DICT master
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 5: Communication to Node 3 closed
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 6: Communication to Node 3 closed
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 7: Communication to Node 3 closed
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 8: Communication to Node 3 closed
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 4: GCP Take over completed
2010-08-24 10:32:43 [MgmtSrvr] INFO     -- Node 4: kk: 3423743/4 0 0
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 4: Node 3 Disconnected
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 5: Node 3 Disconnected
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 8: Node 3 Disconnected
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 8: Backup 1 started from 1 has been aborted. Error: 1326
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 5: Backup 1 started from 1 has been aborted. Error: 1326
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 4: Backup 1 started from 1 has been aborted. Error: 1326
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 7: Backup 1 started from 1 has been aborted. Error: 1326
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 6: Backup 1 started from 1 has been aborted. Error: 1326
2010-08-24 10:32:43 [MgmtSrvr] ALERT    -- Node 3: Forced node shutdown completed. Caused by error 2301: 'Assertion(Internal error, programmin
g error or missing error message, please report a bug). Temporary error, restart node'.
2010-08-24 10:32:46 [MgmtSrvr] INFO     -- Node 6: Data usage is 34%(903518 32K pages of total 2624000)
2010-08-24 10:32:46 [MgmtSrvr] INFO     -- Node 6: Index usage is 31%(302311 8K pages of total 960128)
2010-08-24 10:32:51 [MgmtSrvr] INFO     -- Node 7: Data usage is 34%(902425 32K pages of total 2624000)
2010-08-24 10:32:51 [MgmtSrvr] INFO     -- Node 7: Index usage is 31%(302274 8K pages of total 960128)
2010-08-24 10:32:57 [MgmtSrvr] INFO     -- Node 4: Data usage is 34%(902116 32K pages of total 2624000)
2010-08-24 10:32:57 [MgmtSrvr] INFO     -- Node 4: Index usage is 31%(302361 8K pages of total 960128)
2010-08-24 10:32:57 [MgmtSrvr] INFO     -- Node 5: Data usage is 34%(903481 32K pages of total 2624000)
2010-08-24 10:32:57 [MgmtSrvr] INFO     -- Node 5: Index usage is 31%(302309 8K pages of total 960128)
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: LCP Take over started
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: ParticipatingDIH = 00000000000001f0
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: ParticipatingLQH = 00000000000001f0
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LCP_COMPLETE_REP_Counter_DIH = [SignalCounter: m_count=0 0000000000000000]
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LCP_COMPLETE_REP_Counter_LQH = [SignalCounter: m_count=5 00000000000001f0]
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LAST_LCP_FRAG_ORD = [SignalCounter: m_count=0 0000000000000000]
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LCP_COMPLETE_REP_From_Master_Received = 0
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: LCP Take over completed (state = 5)
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: ParticipatingDIH = 00000000000001f0
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: ParticipatingLQH = 00000000000001f0
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LCP_COMPLETE_REP_Counter_DIH = [SignalCounter: m_count=5 00000000000001f0]
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LCP_COMPLETE_REP_Counter_LQH = [SignalCounter: m_count=5 00000000000001f0]
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LAST_LCP_FRAG_ORD = [SignalCounter: m_count=5 00000000000001f0]
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 4: m_LCP_COMPLETE_REP_From_Master_Received = 0
2010-08-24 10:33:00 [MgmtSrvr] INFO     -- Node 5: Communication to Node 3 opened
2010-08-24 10:33:01 [MgmtSrvr] INFO     -- Node 4: Communication to Node 3 opened
2010-08-24 10:33:01 [MgmtSrvr] INFO     -- Node 6: Communication to Node 3 opened
2010-08-24 10:33:01 [MgmtSrvr] INFO     -- Node 7: Communication to Node 3 opened
2010-08-24 10:33:01 [MgmtSrvr] INFO     -- Node 8: Communication to Node 3 opened
2010-08-24 10:33:13 [MgmtSrvr] INFO     -- Node 8: Data usage is 34%(902403 32K pages of total 2624000)
2010-08-24 10:33:13 [MgmtSrvr] INFO     -- Node 8: Index usage is 31%(302274 8K pages of total 960128)
2010-08-24 10:33:16 [MgmtSrvr] INFO     -- Node 6: Data usage is 34%(903520 32K pages of total 2624000)
2010-08-24 10:33:16 [MgmtSrvr] INFO     -- Node 6: Index usage is 31%(302312 8K pages of total 960128)
2010-08-24 10:33:21 [MgmtSrvr] INFO     -- Node 7: Data usage is 34%(902425 32K pages of total 2624000)
2010-08-24 10:33:21 [MgmtSrvr] INFO     -- Node 7: Index usage is 31%(302274 8K pages of total 960128)
2010-08-24 10:33:27 [MgmtSrvr] INFO     -- Node 4: Data usage is 34%(902116 32K pages of total 2624000)
2010-08-24 10:33:27 [MgmtSrvr] INFO     -- Node 4: Index usage is 31%(302361 8K pages of total 960128)
2010-08-24 10:33:27 [MgmtSrvr] INFO     -- Node 5: Data usage is 34%(903483 32K pages of total 2624000)
2010-08-24 10:33:27 [MgmtSrvr] INFO     -- Node 5: Index usage is 31%(302310 8K pages of total 960128)
2010-08-24 10:33:43 [MgmtSrvr] INFO     -- Node 8: Data usage is 34%(902403 32K pages of total 2624000)
2010-08-24 10:33:43 [MgmtSrvr] INFO     -- Node 8: Index usage is 31%(302274 8K pages of total 960128)
2010-08-24 10:33:46 [MgmtSrvr] INFO     -- Node 6: Data usage is 34%(903522 32K pages of total 2624000)
2010-08-24 10:33:46 [MgmtSrvr] INFO     -- Node 6: Index usage is 31%(302312 8K pages of total 960128)
2010-08-24 10:33:51 [MgmtSrvr] INFO     -- Node 7: Data usage is 34%(902426 32K pages of total 2624000)
2010-08-24 10:33:51 [MgmtSrvr] INFO     -- Node 7: Index usage is 31%(302275 8K pages of total 960128)
2010-08-24 10:33:57 [MgmtSrvr] INFO     -- Node 4: Data usage is 34%(902118 32K pages of total 2624000)
2010-08-24 10:33:57 [MgmtSrvr] INFO     -- Node 4: Index usage is 31%(302361 8K pages of total 960128)
2010-08-24 10:33:57 [MgmtSrvr] INFO     -- Node 5: Data usage is 34%(903483 32K pages of total 2624000)
2010-08-24 10:33:57 [MgmtSrvr] INFO     -- Node 5: Index usage is 31%(302310 8K pages of total 960128)
2010-08-24 10:34:13 [MgmtSrvr] INFO     -- Node 8: Data usage is 34%(902405 32K pages of total 2624000)
2010-08-24 10:34:13 [MgmtSrvr] INFO     -- Node 8: Index usage is 31%(302276 8K pages of total 960128)
2010-08-24 10:34:16 [MgmtSrvr] INFO     -- Node 6: Data usage is 34%(903522 32K pages of total 2624000)
2010-08-24 10:34:16 [MgmtSrvr] INFO     -- Node 6: Index usage is 31%(302313 8K pages of total 960128)
2010-08-24 10:34:21 [MgmtSrvr] INFO     -- Node 4: Local checkpoint 2829 completed
2010-08-24 10:34:21 [MgmtSrvr] INFO     -- Node 7: Data usage is 34%(902427 32K pages of total 2624000)
2010-08-24 10:34:21 [MgmtSrvr] INFO     -- Node 7: Index usage is 31%(302277 8K pages of total 960128)
2010-08-24 10:34:22 [MgmtSrvr] INFO     -- Node 4: Local checkpoint 2830 started. Keep GCI = 3422683 oldest restorable GCI = 3423832
2010-08-24 10:34:27 [MgmtSrvr] INFO     -- Node 4: Data usage is 34%(902120 32K pages of total 2624000)
2010-08-24 10:34:27 [MgmtSrvr] INFO     -- Node 4: Index usage is 31%(302361 8K pages of total 960128)
2010-08-24 10:34:27 [MgmtSrvr] INFO     -- Node 5: Data usage is 34%(903483 32K pages of total 2624000)
2010-08-24 10:34:27 [MgmtSrvr] INFO     -- Node 5: Index usage is 31%(302311 8K pages of total 960128)
2010-08-24 10:34:29 [MgmtSrvr] ALERT    -- Node 4: Backup 1 started from 1 has been aborted. Error: 1350
2010-08-24 10:34:40 [MgmtSrvr] WARNING  -- Node 5: Node 4 missed heartbeat 2
2010-08-24 10:34:41 [MgmtSrvr] ALERT    -- Node 4: Forced node shutdown completed. Caused by error 2301: 'Assertion(Internal error, programmin
g error or missing error message, please report a bug). Temporary error, restart node'.
2010-08-24 10:34:41 [MgmtSrvr] INFO     -- Node 5: Communication to Node 4 closed
2010-08-24 10:34:41 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 closed
2010-08-24 10:34:41 [MgmtSrvr] INFO     -- Node 7: Communication to Node 4 closed
2010-08-24 10:34:41 [MgmtSrvr] ALERT    -- Node 7: Node 4 Disconnected
2010-08-24 10:34:41 [MgmtSrvr] INFO     -- Node 7: Communication to Node 4 closed
2010-08-24 10:34:41 [MgmtSrvr] ALERT    -- Node 8: Node 4 Disconnected
2010-08-24 10:34:41 [MgmtSrvr] INFO     -- Node 8: Communication to Node 4 closed
2010-08-24 10:34:42 [MgmtSrvr] ALERT    -- Node 1: Node 4 Disconnected
2010-08-24 10:34:42 [MgmtSrvr] ALERT    -- Node 6: Node 4 Disconnected
2010-08-24 10:34:42 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 closed
2010-08-24 10:34:43 [MgmtSrvr] INFO     -- Node 8: Data usage is 34%(902406 32K pages of total 2624000)
2010-08-24 10:34:43 [MgmtSrvr] INFO     -- Node 8: Index usage is 31%(302277 8K pages of total 960128)
2010-08-24 10:34:45 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 opened
2010-08-24 10:34:45 [MgmtSrvr] INFO     -- Node 7: Communication to Node 4 opened
2010-08-24 10:34:46 [MgmtSrvr] WARNING  -- Node 7: Node 5 missed heartbeat 2
2010-08-24 10:34:46 [MgmtSrvr] ALERT    -- Node 6: Node 5 Disconnected
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 closed
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 6: Communication to Node 5 closed
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 7: Communication to Node 4 closed
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 7: Communication to Node 5 closed
2010-08-24 10:34:46 [MgmtSrvr] ALERT    -- Node 7: Node 4 Disconnected
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 7: Communication to Node 4 closed
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 8: Communication to Node 4 closed
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 8: Communication to Node 5 closed
2010-08-24 10:34:46 [MgmtSrvr] ALERT    -- Node 1: Node 5 Disconnected
2010-08-24 10:34:46 [MgmtSrvr] ALERT    -- Node 6: Node 4 Disconnected
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 closed
2010-08-24 10:34:46 [MgmtSrvr] ALERT    -- Node 8: Node 5 Disconnected
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 8: Communication to Node 5 closed
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 6: Data usage is 34%(903522 32K pages of total 2624000)
2010-08-24 10:34:46 [MgmtSrvr] INFO     -- Node 6: Index usage is 31%(302313 8K pages of total 960128)
2010-08-24 10:34:46 [MgmtSrvr] ALERT    -- Node 5: Forced node shutdown completed. Caused by error 2305: 'Node lost connection to other nodes
and can not form a unpartitioned cluster, please investigate if there are error(s) on other node(s)(Arbitration error). Temporary error, resta
rt node'.
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 opened
2010-08-24 10:34:50 [MgmtSrvr] WARNING  -- Node 8: Node 7 missed heartbeat 2
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 8: Communication to Node 5 opened
2010-08-24 10:34:50 [MgmtSrvr] ALERT    -- Node 6: Node 7 Disconnected
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 8: Communication to Node 4 closed
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 8: Communication to Node 5 closed
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 8: Communication to Node 7 closed
2010-08-24 10:34:50 [MgmtSrvr] ALERT    -- Node 8: Node 5 Disconnected
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 8: Communication to Node 5 closed
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 closed
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 6: Communication to Node 5 closed
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 6: Communication to Node 7 closed
2010-08-24 10:34:50 [MgmtSrvr] ALERT    -- Node 1: Node 7 Disconnected
2010-08-24 10:34:50 [MgmtSrvr] ALERT    -- Node 7: Forced node shutdown completed. Caused by error 2305: 'Node lost connection to other nodes
and can not form a unpartitioned cluster, please investigate if there are error(s) on other node(s)(Arbitration error). Temporary error, resta
rt node'.
2010-08-24 10:34:50 [MgmtSrvr] ALERT    -- Node 6: Node 4 Disconnected
2010-08-24 10:34:50 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 closed
2010-08-24 10:34:54 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 opened
2010-08-24 10:34:55 [MgmtSrvr] WARNING  -- Node 6: Node 8 missed heartbeat 2
2010-08-24 10:34:55 [MgmtSrvr] ALERT    -- Node 6: Node 8 Disconnected
2010-08-24 10:34:55 [MgmtSrvr] INFO     -- Node 6: Communication to Node 4 closed
2010-08-24 10:34:55 [MgmtSrvr] INFO     -- Node 6: Communication to Node 5 closed
2010-08-24 10:34:55 [MgmtSrvr] INFO     -- Node 6: Communication to Node 7 closed
2010-08-24 10:34:55 [MgmtSrvr] INFO     -- Node 6: Communication to Node 8 closed
2010-08-24 10:34:55 [MgmtSrvr] ALERT    -- Node 6: Node 4 Disconnected
2010-08-24 10:34:55 [MgmtSrvr] ALERT    -- Node 1: Node 8 Disconnected
2010-08-24 10:34:55 [MgmtSrvr] ALERT    -- Node 8: Forced node shutdown completed. Caused by error 2305: 'Node lost connection to other nodes
and can not form a unpartitioned cluster, please investigate if there are error(s) on other node(s)(Arbitration error). Temporary error, resta
rt node'.
2010-08-24 10:34:59 [MgmtSrvr] ALERT    -- Node 1: Node 6 Disconnected
2010-08-24 10:34:59 [MgmtSrvr] ALERT    -- Node 6: Forced node shutdown completed. Caused by error 2305: 'Node lost connection to other nodes
and can not form a unpartitioned cluster, please investigate if there are error(s) on other node(s)(Arbitration error). Temporary error, resta
rt node'.