Current byte-offset of file-pointer is: 21027 Time: Wednesday 17 April 2019 - 17:05:52 Status: Temporary error, restart node Message: LCP fragment scan watchdog detected a problem. Please report a bug. (Internal error, programming error or missing error message, please report a bug) Error: 7200 Error data: Please report this as a bug. Provide as much info as possible, expecially all the ndb_*_out.log files, Thanks. Shutting down node due to lack of LCP fragment scan progress Error object: DBLQH (Line: 28861) 0x00000002 Program: ndbmtd Pid: 12188 thr: 2 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.1_t2 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.1 [t1..t4] ***EOM*** Time: Tuesday 23 April 2019 - 17:03:31 Status: Temporary error, restart node Message: LCP fragment scan watchdog detected a problem. Please report a bug. (Internal error, programming error or missing error message, please report a bug) Error: 7200 Error data: Please report this as a bug. Provide as much info as possible, expecially all the ndb_*_out.log files, Thanks. Shutting down node due to lack of LCP fragment scan progress Error object: DBLQH (Line: 28861) 0x00000002 Program: ndbmtd Pid: 15010 thr: 2 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.2_t2 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.2 [t1..t4] ***EOM*** Time: Tuesday 23 April 2019 - 17:13:22 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 32 failed Error object: QMGR (Line: 4310) 0x00000002 Program: ndbmtd Pid: 31350 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.3 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.3 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 10:04:03 Status: Temporary error, restart node Message: Node lost connection to other nodes and can not form a unpartitioned cluster, please investigate if there are error(s) on other node(s) (Arbitration error) Error: 2305 Error data: Arbitrator decided to shutdown this node Error object: QMGR (Line: 6381) 0x00000002 Program: ndbmtd Pid: 31431 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.4 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.4 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 10:22:23 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: DbtcMain.cpp Error object: DBTC (Line: 1812) 0x00000002 Check false failed Program: ndbmtd Pid: 20070 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.5 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.5 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 11:27:13 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: DbtcMain.cpp Error object: DBTC (Line: 1812) 0x00000002 Check false failed Program: ndbmtd Pid: 20184 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.6 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.6 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 11:49:14 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: DbtcMain.cpp Error object: DBTC (Line: 1812) 0x00000002 Check false failed Program: ndbmtd Pid: 20683 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.7 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.7 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 11:55:21 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 34 failed Error object: QMGR (Line: 4310) 0x00000002 Program: ndbmtd Pid: 20938 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.8 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.8 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 12:12:26 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 31 failed Error object: QMGR (Line: 4310) 0x00000002 Program: ndbmtd Pid: 21092 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.9 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.9 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 12:19:43 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 31 failed Error object: QMGR (Line: 4310) 0x00000002 Program: ndbmtd Pid: 21205 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.10 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.10 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 12:26:24 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 34 disconnected Error object: QMGR (Line: 3539) 0x00000002 Program: ndbmtd Pid: 21286 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.11 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.11 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 12:42:03 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 34 disconnected Error object: QMGR (Line: 3539) 0x00000002 Program: ndbmtd Pid: 21363 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.12 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.12 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 16:49:26 Status: Temporary error, restart node Message: Node lost connection to other nodes and can not form a unpartitioned cluster, please investigate if there are error(s) on other node(s) (Arbitration error) Error: 2305 Error data: Arbitrator decided to shutdown this node Error object: QMGR (Line: 6381) 0x00000002 Program: ndbmtd Pid: 21470 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.13 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.13 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 17:09:54 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: DbtcMain.cpp Error object: DBTC (Line: 1812) 0x00000002 Check false failed Program: ndbmtd Pid: 23008 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.14 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.14 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 17:17:24 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 31 failed Error object: QMGR (Line: 4310) 0x00000002 Program: ndbmtd Pid: 23158 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.15 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.15 [t1..t4] ***EOM*** Time: Friday 3 May 2019 - 17:24:06 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 34 disconnected Error object: QMGR (Line: 3539) 0x00000002 Program: ndbmtd Pid: 23239 thr: 0 Version: mysql-5.7.23 ndb-7.6.7 Trace file name: ndb_33_trace.log.16 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.16 [t1..t4] ***EOM*** Time: Wednesday 8 May 2019 - 10:06:07 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: DbtcMain.cpp Error object: DBTC (Line: 1823) 0x00000002 Check false failed Program: ndbmtd Pid: 31948 thr: 0 Version: mysql-5.7.26 ndb-7.6.10 Trace file name: ndb_33_trace.log.17 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.17 [t1..t4] ***EOM*** Time: Wednesday 8 May 2019 - 10:17:04 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 32 failed Error object: QMGR (Line: 4319) 0x00000002 Program: ndbmtd Pid: 24082 thr: 0 Version: mysql-5.7.26 ndb-7.6.10 Trace file name: ndb_33_trace.log.18 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.18 [t1..t4] ***EOM*** Time: Wednesday 8 May 2019 - 10:23:55 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 34 disconnected Error object: QMGR (Line: 3548) 0x00000002 Program: ndbmtd Pid: 24172 thr: 0 Version: mysql-5.7.26 ndb-7.6.10 Trace file name: ndb_33_trace.log.19 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.19 [t1..t4] ***EOM*** Time: Wednesday 8 May 2019 - 10:40:00 Status: Temporary error, restart node Message: Another node failed during system restart, please investigate error(s) on other node(s) (Restart error) Error: 2308 Error data: Node 34 disconnected Error object: QMGR (Line: 3548) 0x00000002 Program: ndbmtd Pid: 24271 thr: 0 Version: mysql-5.7.26 ndb-7.6.10 Trace file name: ndb_33_trace.log.20 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.20 [t1..t4] ***EOM*** Time: Wednesday 8 May 2019 - 10:53:23 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: DblqhMain.cpp Error object: DBLQH (Line: 10641) 0x00000002 Check false failed Program: ndbmtd Pid: 24355 thr: 2 Version: mysql-5.7.26 ndb-7.6.10 Trace file name: ndb_33_trace.log.21_t2 Trace file path: /var/lib/mysql-cluster/data_cluster_node/data/ndb_33_trace.log.21 [t1..t4] ***EOM***