2006-07-12 20:05:48 [MgmSrvr] INFO -- Node 2: Local checkpoint 1717 started. Keep GCI = 29245 oldest restorable GCI = 29257 2006-07-12 20:07:49 [MgmSrvr] INFO -- Node 2: Local checkpoint 1718 started. Keep GCI = 29267 oldest restorable GCI = 29279 2006-07-12 20:09:51 [MgmSrvr] INFO -- Node 2: Local checkpoint 1719 started. Keep GCI = 29288 oldest restorable GCI = 29299 2006-07-12 20:11:52 [MgmSrvr] INFO -- Node 2: Local checkpoint 1720 started. Keep GCI = 29310 oldest restorable GCI = 29322 2006-07-12 20:13:55 [MgmSrvr] INFO -- Node 2: Local checkpoint 1721 started. Keep GCI = 29331 oldest restorable GCI = 29343 2006-07-12 20:15:56 [MgmSrvr] INFO -- Node 2: Local checkpoint 1722 started. Keep GCI = 29353 oldest restorable GCI = 29365 2006-07-12 20:17:54 [MgmSrvr] INFO -- Node 2: Local checkpoint 1723 started. Keep GCI = 29375 oldest restorable GCI = 29386 2006-07-12 20:19:56 [MgmSrvr] INFO -- Node 2: Local checkpoint 1724 started. Keep GCI = 29396 oldest restorable GCI = 29408 2006-07-12 20:21:57 [MgmSrvr] INFO -- Node 2: Local checkpoint 1725 started. Keep GCI = 29417 oldest restorable GCI = 29429 2006-07-12 20:23:59 [MgmSrvr] INFO -- Node 2: Local checkpoint 1726 started. Keep GCI = 29438 oldest restorable GCI = 29450 2006-07-12 20:25:58 [MgmSrvr] INFO -- Node 2: Local checkpoint 1727 started. Keep GCI = 29460 oldest restorable GCI = 29472 2006-07-12 20:27:59 [MgmSrvr] INFO -- Node 2: Local checkpoint 1728 started. Keep GCI = 29482 oldest restorable GCI = 29493 2006-07-12 20:30:00 [MgmSrvr] INFO -- Node 2: Local checkpoint 1729 started. Keep GCI = 29503 oldest restorable GCI = 29515 2006-07-12 20:32:03 [MgmSrvr] INFO -- Node 2: Local checkpoint 1730 started. Keep GCI = 29523 oldest restorable GCI = 29535 2006-07-12 20:34:04 [MgmSrvr] INFO -- Node 2: Local checkpoint 1731 started. Keep GCI = 29544 oldest restorable GCI = 29556 2006-07-12 20:36:02 [MgmSrvr] INFO -- Node 2: Local checkpoint 1732 started. Keep GCI = 29565 oldest restorable GCI = 29577 2006-07-12 20:38:03 [MgmSrvr] INFO -- Node 2: Local checkpoint 1733 started. Keep GCI = 29586 oldest restorable GCI = 29598 2006-07-12 20:40:03 [MgmSrvr] INFO -- Node 2: Local checkpoint 1734 started. Keep GCI = 29607 oldest restorable GCI = 29619 2006-07-12 20:42:05 [MgmSrvr] INFO -- Node 2: Local checkpoint 1735 started. Keep GCI = 29629 oldest restorable GCI = 29641 2006-07-12 20:44:09 [MgmSrvr] INFO -- Node 2: Local checkpoint 1736 started. Keep GCI = 29651 oldest restorable GCI = 29663 2006-07-12 20:46:09 [MgmSrvr] INFO -- Node 2: Local checkpoint 1737 started. Keep GCI = 29673 oldest restorable GCI = 29685 2006-07-12 20:48:07 [MgmSrvr] INFO -- Node 2: Local checkpoint 1738 started. Keep GCI = 29694 oldest restorable GCI = 29705 2006-07-12 20:50:07 [MgmSrvr] INFO -- Node 2: Local checkpoint 1739 started. Keep GCI = 29715 oldest restorable GCI = 29727 2006-07-12 20:52:07 [MgmSrvr] INFO -- Node 2: Local checkpoint 1740 started. Keep GCI = 29736 oldest restorable GCI = 29748 2006-07-12 20:54:08 [MgmSrvr] INFO -- Node 2: Local checkpoint 1741 started. Keep GCI = 29759 oldest restorable GCI = 29770 2006-07-12 20:56:12 [MgmSrvr] INFO -- Node 2: Local checkpoint 1742 started. Keep GCI = 29780 oldest restorable GCI = 29792 2006-07-12 20:58:13 [MgmSrvr] INFO -- Node 2: Local checkpoint 1743 started. Keep GCI = 29803 oldest restorable GCI = 29815 2006-07-12 21:00:12 [MgmSrvr] INFO -- Node 2: Local checkpoint 1744 started. Keep GCI = 29824 oldest restorable GCI = 29836 2006-07-12 21:02:16 [MgmSrvr] INFO -- Node 2: Local checkpoint 1745 started. Keep GCI = 29845 oldest restorable GCI = 29857 2006-07-12 21:04:18 [MgmSrvr] INFO -- Node 2: Local checkpoint 1746 started. Keep GCI = 29867 oldest restorable GCI = 29879 2006-07-12 21:06:18 [MgmSrvr] INFO -- Node 2: Local checkpoint 1747 started. Keep GCI = 29888 oldest restorable GCI = 29899 2006-07-12 21:08:17 [MgmSrvr] INFO -- Node 2: Local checkpoint 1748 started. Keep GCI = 29909 oldest restorable GCI = 29921 2006-07-12 21:10:18 [MgmSrvr] INFO -- Node 2: Local checkpoint 1749 started. Keep GCI = 29929 oldest restorable GCI = 29942 2006-07-12 21:12:18 [MgmSrvr] INFO -- Node 2: Local checkpoint 1750 started. Keep GCI = 29952 oldest restorable GCI = 29964 2006-07-12 21:14:21 [MgmSrvr] INFO -- Node 2: Local checkpoint 1751 started. Keep GCI = 29974 oldest restorable GCI = 29986 2006-07-12 21:16:23 [MgmSrvr] INFO -- Node 2: Local checkpoint 1752 started. Keep GCI = 29995 oldest restorable GCI = 30006 2006-07-12 21:18:24 [MgmSrvr] INFO -- Node 2: Local checkpoint 1753 started. Keep GCI = 30017 oldest restorable GCI = 30029 2006-07-12 21:20:29 [MgmSrvr] INFO -- Node 2: Local checkpoint 1754 started. Keep GCI = 30039 oldest restorable GCI = 30050 2006-07-12 21:22:09 [MgmSrvr] INFO -- Node 6: Event buffer status: used=4235KB(91%) alloc=4620KB(0%) max=0B apply_gci=30078 latest_gci=30078 2006-07-12 21:22:09 [MgmSrvr] INFO -- Node 4: Event buffer status: used=4226KB(91%) alloc=4622KB(0%) max=0B apply_gci=30078 latest_gci=30078 2006-07-12 21:22:09 [MgmSrvr] INFO -- Node 5: Event buffer status: used=4195KB(91%) alloc=4597KB(0%) max=0B apply_gci=30078 latest_gci=30078 2006-07-12 21:22:19 [MgmSrvr] INFO -- Node 6: Event buffer status: used=5111KB(100%) alloc=5111KB(0%) max=0B apply_gci=30079 latest_gci=30079 2006-07-12 21:22:19 [MgmSrvr] INFO -- Node 4: Event buffer status: used=5113KB(100%) alloc=5113KB(0%) max=0B apply_gci=30079 latest_gci=30079 2006-07-12 21:22:19 [MgmSrvr] INFO -- Node 5: Event buffer status: used=5089KB(100%) alloc=5089KB(0%) max=0B apply_gci=30079 latest_gci=30079 2006-07-12 21:22:31 [MgmSrvr] INFO -- Node 2: Local checkpoint 1755 started. Keep GCI = 30061 oldest restorable GCI = 30072 2006-07-12 21:24:31 [MgmSrvr] INFO -- Node 2: Local checkpoint 1756 started. Keep GCI = 30083 oldest restorable GCI = 30094 2006-07-12 21:26:31 [MgmSrvr] INFO -- Node 2: Local checkpoint 1757 started. Keep GCI = 30103 oldest restorable GCI = 30114 2006-07-12 21:28:33 [MgmSrvr] INFO -- Node 2: Local checkpoint 1758 started. Keep GCI = 30123 oldest restorable GCI = 30134 2006-07-12 21:30:33 [MgmSrvr] INFO -- Node 2: Local checkpoint 1759 started. Keep GCI = 30145 oldest restorable GCI = 30157 2006-07-12 21:32:33 [MgmSrvr] INFO -- Node 2: Local checkpoint 1760 started. Keep GCI = 30167 oldest restorable GCI = 30178 2006-07-12 21:34:34 [MgmSrvr] INFO -- Node 2: Local checkpoint 1761 started. Keep GCI = 30188 oldest restorable GCI = 30200 2006-07-12 21:36:35 [MgmSrvr] INFO -- Node 2: Local checkpoint 1762 started. Keep GCI = 30209 oldest restorable GCI = 30220 2006-07-12 21:37:58 [MgmSrvr] INFO -- Node 3: Data usage increased to 97%(90420 32K pages of total 92800) 2006-07-12 21:37:59 [MgmSrvr] INFO -- Node 2: Data usage increased to 97%(90660 32K pages of total 92800) 2006-07-12 21:38:38 [MgmSrvr] INFO -- Node 2: Local checkpoint 1763 started. Keep GCI = 30227 oldest restorable GCI = 30247 2006-07-12 21:40:41 [MgmSrvr] INFO -- Node 2: Local checkpoint 1764 started. Keep GCI = 30249 oldest restorable GCI = 30261 2006-07-12 21:42:43 [MgmSrvr] INFO -- Node 2: Local checkpoint 1765 started. Keep GCI = 30271 oldest restorable GCI = 30283 2006-07-12 21:44:47 [MgmSrvr] INFO -- Node 2: Local checkpoint 1766 started. Keep GCI = 30293 oldest restorable GCI = 30305 2006-07-12 21:46:52 [MgmSrvr] INFO -- Node 2: Local checkpoint 1767 started. Keep GCI = 30314 oldest restorable GCI = 30326 2006-07-12 21:48:54 [MgmSrvr] INFO -- Node 2: Local checkpoint 1768 started. Keep GCI = 30336 oldest restorable GCI = 30349 2006-07-12 21:50:58 [MgmSrvr] INFO -- Node 2: Local checkpoint 1769 started. Keep GCI = 30359 oldest restorable GCI = 30371 2006-07-12 21:53:04 [MgmSrvr] INFO -- Node 2: Local checkpoint 1770 started. Keep GCI = 30380 oldest restorable GCI = 30392 2006-07-12 21:55:09 [MgmSrvr] INFO -- Node 2: Local checkpoint 1771 started. Keep GCI = 30402 oldest restorable GCI = 30414 2006-07-12 21:57:14 [MgmSrvr] INFO -- Node 2: Local checkpoint 1772 started. Keep GCI = 30424 oldest restorable GCI = 30436 2006-07-12 21:59:20 [MgmSrvr] INFO -- Node 2: Local checkpoint 1773 started. Keep GCI = 30447 oldest restorable GCI = 30460 2006-07-12 22:01:23 [MgmSrvr] INFO -- Node 2: Local checkpoint 1774 started. Keep GCI = 30469 oldest restorable GCI = 30481 2006-07-12 22:03:28 [MgmSrvr] INFO -- Node 2: Local checkpoint 1775 started. Keep GCI = 30490 oldest restorable GCI = 30501 2006-07-12 22:05:35 [MgmSrvr] INFO -- Node 2: Local checkpoint 1776 started. Keep GCI = 30513 oldest restorable GCI = 30523 2006-07-12 22:07:42 [MgmSrvr] INFO -- Node 2: Local checkpoint 1777 started. Keep GCI = 30534 oldest restorable GCI = 30547 2006-07-12 22:09:50 [MgmSrvr] INFO -- Node 2: Local checkpoint 1778 started. Keep GCI = 30558 oldest restorable GCI = 30569 2006-07-12 22:11:56 [MgmSrvr] INFO -- Node 2: Local checkpoint 1779 started. Keep GCI = 30581 oldest restorable GCI = 30593 2006-07-12 22:13:53 [MgmSrvr] INFO -- Node 3: Data usage increased to 100%(92800 32K pages of total 92800) 2006-07-12 22:13:57 [MgmSrvr] WARNING -- Node 3: Node 2 missed heartbeat 2 2006-07-12 22:13:57 [MgmSrvr] INFO -- Node 1: Node 2 Connected 2006-07-12 22:13:58 [MgmSrvr] WARNING -- Node 3: Node 2 missed heartbeat 3 2006-07-12 22:14:00 [MgmSrvr] WARNING -- Node 3: Node 2 missed heartbeat 4 2006-07-12 22:14:00 [MgmSrvr] ALERT -- Node 3: Node 2 declared dead due to missed heartbeat 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: Communication to Node 2 closed 2006-07-12 22:14:00 [MgmSrvr] ALERT -- Node 3: Network partitioning - arbitration required 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: President restarts arbitration thread [state=7] 2006-07-12 22:14:00 [MgmSrvr] ALERT -- Node 3: Arbitration won - positive reply from node 1 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: GCP Take over started 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: LCP Take over started 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: ParticipatingDIH = 0000000000000008 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: ParticipatingLQH = 0000000000000008 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LCP_COMPLETE_REP_Counter_DIH = [SignalCounter: m_count=0 0000000000000000] 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LCP_COMPLETE_REP_Counter_LQH = [SignalCounter: m_count=1 0000000000000008] 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LAST_LCP_FRAG_ORD = [SignalCounter: m_count=1 0000000000000008] 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LCP_COMPLETE_REP_From_Master_Received = 0 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: GCP Take over completed 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: LCP Take over completed (state = 5) 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: ParticipatingDIH = 0000000000000008 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: ParticipatingLQH = 0000000000000008 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LCP_COMPLETE_REP_Counter_DIH = [SignalCounter: m_count=1 0000000000000008] 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LCP_COMPLETE_REP_Counter_LQH = [SignalCounter: m_count=1 0000000000000008] 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LAST_LCP_FRAG_ORD = [SignalCounter: m_count=1 0000000000000008] 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: m_LCP_COMPLETE_REP_From_Master_Received = 0 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: Node 3: started: table=14 fragment=0 replica=57 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: Node 3: started: table=14 fragment=1 replica=59 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: Node 3: queued: table=15 fragment=0 replica=61 2006-07-12 22:14:00 [MgmSrvr] INFO -- Node 3: Node 3: queued: table=15 fragment=1 replica=63 2006-07-12 22:14:01 [MgmSrvr] ALERT -- Node 3: Node 2 Disconnected 2006-07-12 22:14:01 [MgmSrvr] INFO -- Node 3: Started arbitrator node 1 [ticket=1734000363164743] 2006-07-12 22:14:06 [MgmSrvr] INFO -- Node 3: Communication to Node 2 opened 2006-07-12 22:14:37 [MgmSrvr] INFO -- Node 1: Node 3 Connected 2006-07-12 22:15:30 [MgmSrvr] INFO -- Shutting down server... 2006-07-12 22:15:32 [MgmSrvr] INFO -- Mgmt server state: nodeid 6 freed, m_reserved_nodes 0000000000000002. 2006-07-12 22:15:32 [MgmSrvr] INFO -- Shutdown complete