Mar 6 13:58:01 HXVM-DNS-02 pdns_recursor: Mar 06 13:58:01 Inserting rfc 1918 private space zones Mar 6 14:00:01 HXVM-DNS-02 systemd: Started Session 592790 of user root. Mar 6 14:00:01 HXVM-DNS-02 systemd: Starting Session 592790 of user root. Mar 6 14:00:01 HXVM-DNS-02 pdns_recursor: Mar 06 14:00:01 Reloading zones, purging data from cache Mar 6 14:00:01 HXVM-DNS-02 pdns_recursor: Mar 06 14:00:01 Redirecting queries for zone '.' with recursion to: 114.114.114.114:53, 223.5.5.5:53, 119.29.29.29:53, 202.96.209.133:53 Mar 6 14:00:01 HXVM-DNS-02 pdns_recursor: Mar 06 14:00:01 Reading zone forwarding information from '/etc/pdns-recursor/zone' Mar 6 14:00:01 HXVM-DNS-02 pdns_recursor: Mar 06 14:00:01 Done parsing 14 forwarding instructions from file '/etc/pdns-recursor/zone' Mar 6 14:00:01 HXVM-DNS-02 pdns_recursor: Mar 06 14:00:01 Inserting rfc 1918 private space zones Mar 6 14:01:02 HXVM-DNS-02 kernel: vmtoolsd invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 Mar 6 14:01:02 HXVM-DNS-02 kernel: vmtoolsd cpuset=/ mems_allowed=0 Mar 6 14:01:02 HXVM-DNS-02 kernel: CPU: 1 PID: 811 Comm: vmtoolsd Kdump: loaded Not tainted 3.10.0-862.el7.x86_64 #1 Mar 6 14:01:02 HXVM-DNS-02 kernel: Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016 Mar 6 14:01:02 HXVM-DNS-02 kernel: Call Trace: Mar 6 14:01:02 HXVM-DNS-02 kernel: [] dump_stack+0x19/0x1b Mar 6 14:01:02 HXVM-DNS-02 kernel: [] dump_header+0x90/0x229 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] ? cred_has_capability+0x6b/0x120 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] oom_kill_process+0x254/0x3d0 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] ? selinux_capable+0x1c/0x40 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] out_of_memory+0x4b6/0x4f0 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] __alloc_pages_nodemask+0xaa4/0xbb0 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] alloc_pages_current+0x98/0x110 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] __page_cache_alloc+0x97/0xb0 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] filemap_fault+0x298/0x490 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] ext4_filemap_fault+0x36/0x50 [ext4] Mar 6 14:01:02 HXVM-DNS-02 kernel: [] __do_fault.isra.58+0x8a/0x100 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] ? memcg_check_events+0xbe/0x210 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] do_read_fault.isra.60+0x4c/0x1a0 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] handle_pte_fault+0x2dc/0xc30 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] handle_mm_fault+0x39d/0x9b0 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] __do_page_fault+0x197/0x4f0 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] do_page_fault+0x35/0x90 Mar 6 14:01:02 HXVM-DNS-02 kernel: [] page_fault+0x28/0x30 Mar 6 14:01:02 HXVM-DNS-02 kernel: Mem-Info: Mar 6 14:01:02 HXVM-DNS-02 kernel: active_anon:6 inactive_anon:19 isolated_anon:0#012 active_file:42 inactive_file:16 isolated_file:0#012 unevictable:1780836 dirty:0 writeback:0 unstable:0#012 slab_reclaimable:8254 slab_unreclaimable:11 563#012 mapped:9267 shmem:0 pagetables:4991 bounce:0#012 free:26276 free_pcp:2 free_cma:0 Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 DMA free:15900kB min:88kB low:108kB high:132kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:1 5904kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:4kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Mar 6 14:01:02 HXVM-DNS-02 kernel: lowmem_reserve[]: 0 2830 11818 11818 Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 DMA32 free:51876kB min:16164kB low:20204kB high:24244kB active_anon:24kB inactive_anon:76kB active_file:60kB inactive_file:0kB unevictable:1140036kB isolated(anon):0kB isolated(file):0kB present :3129152kB managed:2898720kB mlocked:1140036kB dirty:0kB writeback:0kB mapped:7096kB shmem:0kB slab_reclaimable:6772kB slab_unreclaimable:9012kB kernel_stack:384kB pagetables:2848kB unstable:0kB bounce:0kB free_pcp:8kB local_pcp:0kB free _cma:0kB writeback_tmp:0kB pages_scanned:18475 all_unreclaimable? yes Mar 6 14:01:02 HXVM-DNS-02 kernel: lowmem_reserve[]: 0 0 8987 8987 Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 Normal free:37328kB min:51328kB low:64160kB high:76992kB active_anon:0kB inactive_anon:0kB active_file:108kB inactive_file:128kB unevictable:5983308kB isolated(anon):0kB isolated(file):0kB prese nt:9437184kB managed:9206476kB mlocked:5983308kB dirty:0kB writeback:0kB mapped:29972kB shmem:0kB slab_reclaimable:26244kB slab_unreclaimable:37236kB kernel_stack:6352kB pagetables:17116kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0 kB free_cma:0kB writeback_tmp:0kB pages_scanned:384 all_unreclaimable? yes Mar 6 14:01:02 HXVM-DNS-02 kernel: lowmem_reserve[]: 0 0 0 0 Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15900kB Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 DMA32: 6673*4kB (UEM) 3158*8kB (UM) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 51956kB Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 Normal: 76*4kB (EM) 86*8kB (EM) 43*16kB (EM) 64*32kB (EM) 29*64kB (UEM) 14*128kB (UEM) 17*256kB (UEM) 14*512kB (UEM) 10*1024kB (M) 4*2048kB (M) 0*4096kB = 37328kB Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB Mar 6 14:01:02 HXVM-DNS-02 kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Mar 6 14:01:02 HXVM-DNS-02 kernel: 9341 total pagecache pages Mar 6 14:01:02 HXVM-DNS-02 kernel: 1 pages in swap cache Mar 6 14:01:02 HXVM-DNS-02 kernel: Swap cache stats: add 519493, delete 519473, find 255424615/255453013 Mar 6 14:01:02 HXVM-DNS-02 kernel: Free swap = 3473192kB Mar 6 14:01:02 HXVM-DNS-02 kernel: Total swap = 4194300kB Mar 6 14:01:02 HXVM-DNS-02 kernel: 3145581 pages RAM Mar 6 14:01:02 HXVM-DNS-02 kernel: 0 pages HighMem/MovableOnly Mar 6 14:01:02 HXVM-DNS-02 kernel: 115306 pages reserved Mar 6 14:01:02 HXVM-DNS-02 kernel: [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 562] 0 562 25155 206 55 85 0 systemd-journal Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 578] 0 578 50247 196 31 835 0 lvmetad Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 586] 0 586 12189 218 26 902 -1000 systemd-udevd Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 778] 0 778 13877 116 27 132 -1000 auditd Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 800] 81 800 16608 240 32 169 -900 dbus-daemon Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 803] 0 803 119047 181 84 597 0 NetworkManager Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 807] 999 807 134641 233 59 1690 0 polkitd Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 808] 0 808 6627 221 18 106 0 systemd-logind Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 809] 0 809 5412 191 17 85 0 irqbalance Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 810] 0 810 24914 294 40 402 0 VGAuthService Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 811] 0 811 93111 345 60 361 0 vmtoolsd Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 816] 0 816 31579 193 20 171 0 crond Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 818] 998 818 29438 244 29 115 0 chronyd Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 829] 0 829 27522 172 10 33 0 agetty Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 1061] 0 1061 28199 248 58 259 -1000 sshd Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 1062] 0 1062 143463 249 95 2805 0 tuned Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 1064] 0 1064 155979 192 131 626 0 rsyslogd Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 1300] 0 1300 22405 205 42 263 0 master Mar 6 14:01:02 HXVM-DNS-02 kernel: [ 1303] 89 1303 22475 290 45 267 0 qmgr Mar 6 14:01:02 HXVM-DNS-02 kernel: [109900] 0 109900 209294 87 39 3197 0 ndb_mgmd Mar 6 14:01:02 HXVM-DNS-02 kernel: [110008] 0 110008 7161 127 17 519 0 ndbd Mar 6 14:01:02 HXVM-DNS-02 kernel: [110009] 0 110009 1675019 1622313 3183 0 0 ndbd Mar 6 14:01:02 HXVM-DNS-02 kernel: [110313] 1001 110313 413710 158672 351 4303 0 mysqld Mar 6 14:01:02 HXVM-DNS-02 kernel: [114909] 997 114909 92170 389 141 3847 0 pdns_recursor Mar 6 14:01:02 HXVM-DNS-02 kernel: [130172] 1000 130172 202892 265 75 3084 0 pdns_server Mar 6 14:01:02 HXVM-DNS-02 kernel: [73726] 996 73726 19595 120 38 195 0 zabbix_agentd Mar 6 14:01:02 HXVM-DNS-02 kernel: [73727] 996 73727 19595 81 38 195 0 zabbix_agentd Mar 6 14:01:02 HXVM-DNS-02 kernel: [73728] 996 73728 19595 117 38 204 0 zabbix_agentd Mar 6 14:01:02 HXVM-DNS-02 kernel: [73729] 996 73729 19595 141 38 202 0 zabbix_agentd Mar 6 14:01:02 HXVM-DNS-02 kernel: [73730] 996 73730 19595 138 38 202 0 zabbix_agentd Mar 6 14:01:02 HXVM-DNS-02 kernel: [73731] 996 73731 19595 109 38 209 0 zabbix_agentd Mar 6 14:01:02 HXVM-DNS-02 kernel: [59358] 89 59358 22431 320 47 265 0 pickup Mar 6 14:01:02 HXVM-DNS-02 kernel: Out of memory: Kill process 110009 (ndbd) score 386 or sacrifice child Mar 6 14:01:02 HXVM-DNS-02 kernel: Killed process 110009 (ndbd) total-vm:6700076kB, anon-rss:6477888kB, file-rss:11364kB, shmem-rss:0kB Mar 6 14:01:02 HXVM-DNS-02 pdns_server: Mar 06 14:01:02 Backend reported permanent error which prevented lookup (GSQLBackend lookup query:Could not store mysql statement: SELECT content,ttl,prio,type,domain_id,disabled,name,auth FROM re cords WHERE disabled=0 and type=? and name=?: Got temporary error 4025 'Node failure caused abort of transaction' from NDBCLUSTER), aborting Mar 6 14:01:02 HXVM-DNS-02 pdns[130172]: Backend reported permanent error which prevented lookup (GSQLBackend lookup query:Could not store mysql statement: SELECT content,ttl,prio,type,domain_id,disabled,name,auth FROM records WHERE dis abled=0 and type=? and name=?: Got temporary error 4025 'Node failure caused abort of transaction' from NDBCLUSTER), aborting Mar 6 14:01:02 HXVM-DNS-02 pdns_server: Mar 06 14:01:02 Backend reported permanent error which prevented lookup (GSQLBackend lookup query:Could not store mysql statement: SELECT content,ttl,prio,type,domain_id,disabled,name,auth FROM re cords WHERE disabled=0 and type=? and name=?: Got temporary error 4025 'Node failure caused abort of transaction' from NDBCLUSTER), aborting