2006-05-10 08:32:51 [ndbd] INFO -- Start initiated (version 5.0.21) NR: setLcpActiveStatusEnd - !m_participatingLQH NR: setLcpActiveStatusEnd - m_participatingLQH LP 0 state: 0 WW_Gci: 1 gcprec: 0 flq: -256 currfile: 29 tailFileNo: 29 logTailMbyte: 9 Ndb kernel is stuck in: file 0(0) FileChangeState: 0 logFileStatus: 20 currentMbyte: 15 currentFilepage Job Handling file 1(1) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 2(2) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 3(3) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 4(4) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 5(5) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 6(6) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 7(7) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 8(8) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 9(9) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 10(10) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 11(11) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 12(12) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 13(13) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 14(14) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 15(15) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 16(16) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 17(17) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 18(18) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 19(19) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 20(20) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 21(21) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 22(22) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 23(23) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 24(24) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 25(25) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 26(26) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 27(27) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 28(28) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 29(29) FileChangeState: 0 logFileStatus: 20 currentMbyte: 9 currentFilepage file 30(30) FileChangeState: 0 logFileStatus: 20 currentMbyte: 0 currentFilepage file 31(31) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage LP 1 state: 0 WW_Gci: 1 gcprec: 0 flq: -256 currfile: 35 tailFileNo: 3 logTailMbyte: 6 file 0(32) FileChangeState: 0 logFileStatus: 20 currentMbyte: 15 currentFilepage file 1(33) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 2(34) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 3(35) FileChangeState: 0 logFileStatus: 20 currentMbyte: 6 currentFilepage file 4(36) FileChangeState: 0 logFileStatus: 20 currentMbyte: 0 currentFilepage file 5(37) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 6(38) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 7(39) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 8(40) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 9(41) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 10(42) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 11(43) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 12(44) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 13(45) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 14(46) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 15(47) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 16(48) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 17(49) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 18(50) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 19(51) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 20(52) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 21(53) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 22(54) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 23(55) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 24(56) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 25(57) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 26(58) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 27(59) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 28(60) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 29(61) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 30(62) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 31(63) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage LP 2 state: 0 WW_Gci: 1 gcprec: 0 flq: -256 currfile: 93 tailFileNo: 29 logTailMbyte: 12 file 0(64) FileChangeState: 0 logFileStatus: 20 currentMbyte: 15 currentFilepage file 1(65) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 2(66) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 3(67) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 4(68) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 5(69) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 6(70) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 7(71) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 8(72) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 9(73) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 10(74) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 11(75) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 12(76) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 13(77) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 14(78) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 15(79) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 16(80) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 17(81) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 18(82) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 19(83) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 20(84) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 21(85) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 22(86) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 23(87) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 24(88) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 25(89) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 26(90) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 27(91) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 28(92) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 29(93) FileChangeState: 0 logFileStatus: 20 currentMbyte: 12 currentFilepage file 30(94) FileChangeState: 0 logFileStatus: 20 currentMbyte: 0 currentFilepage file 31(95) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage LP 3 state: 0 WW_Gci: 1 gcprec: 0 flq: -256 currfile: 125 tailFileNo: 29 logTailMbyte: 9 file 0(96) FileChangeState: 0 logFileStatus: 20 currentMbyte: 15 currentFilepage file 1(97) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 2(98) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 3(99) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 4(100) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 5(101) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 6(102) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 7(103) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 8(104) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 9(105) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 10(106) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 11(107) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 12(108) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 13(109) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 14(110) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 15(111) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 16(112) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 17(113) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 18(114) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 19(115) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 20(116) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 21(117) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 22(118) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 23(119) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 24(120) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 25(121) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 26(122) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 27(123) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 28(124) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage file 29(125) FileChangeState: 0 logFileStatus: 20 currentMbyte: 9 currentFilepage file 30(126) FileChangeState: 0 logFileStatus: 20 currentMbyte: 0 currentFilepage file 31(127) FileChangeState: 0 logFileStatus: 1 currentMbyte: 15 currentFilepage GCP 0 file: 29 state: 1 sync: 0 page: 298 word: 3980 GCP 1 file: 35 state: 1 sync: 0 page: 211 word: 577 GCP 2 file: 93 state: 1 sync: 0 page: 400 word: 7076 GCP 3 file: 125 state: 1 sync: 0 page: 299 word: 6686 2006-09-02 04:09:22 [ndbd] INFO -- Error handler shutting down system 2006-09-02 04:09:24 [ndbd] INFO -- Error handler shutdown completed - exiting 2006-09-02 04:09:24 [ndbd] ALERT -- Node 12: Forced node shutdown completed. Initiated by signal 0. Caused by error 2303: 'System error, node killed during node restart by other node(Internal error, programming error or missing error message, please report a bug). Temporary error, restart n 2006-09-02 04:57:09 [ndbd] INFO -- Angel pid: 5930 ndb pid: 5931 2006-09-02 04:57:09 [ndbd] INFO -- NDB Cluster -- DB node 12 2006-09-02 04:57:09 [ndbd] INFO -- Version 5.0.21 -- 2006-09-02 04:57:09 [ndbd] INFO -- Configuration fetched at 10.10.250.1 port 1186 2006-09-02 04:57:09 [ndbd] INFO -- WatchDog timer is set to 6000 ms 2006-09-02 04:57:09 [ndbd] INFO -- Start initiated (version 5.0.21) 2006-09-02 04:57:11 [ndbd] INFO -- Start phase 0 completed 2006-09-02 04:57:11 [ndbd] INFO -- CM_REGREF from Node 12 to our Node 12. Cause = Election without selecting new candidate 2006-09-02 04:57:11 [ndbd] INFO -- Waiting 30 sec for nodes 000000000007e800 to connect, nodes [ all: 000000000007f800 connected: 0000000000001000 no-wait: 0000000000000000 ] NR: setLcpActiveStatusEnd - !m_participatingLQH NR: setLcpActiveStatusEnd - !m_participatingLQH NR: setLcpActiveStatusEnd - m_participatingLQH