Bug #70987 NDB_MGMD consumes a lot of CPU and no reponding on OpenVZ
Submitted: 23 Nov 2013 14:54 Modified: 24 Mar 2016 23:30
Reporter: None None Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:7.3.2 OS:Linux (CentOS)
Assigned to: MySQL Verification Team CPU Architecture:Any
Tags: ndb_mgmd, openvz, time_wait

[23 Nov 2013 14:54] None None 
Description:
I'm trying to build a "shared nothing" system:

For a few days everything seemed fine but after about a week ndb_mgmd started to eat 80% CPU and stopped responding to 1186 port (ndb_mgm says "can't connect to ndb_ngnd").

netstat reports many local connections to port 1186 with TIME_WAIT state.
After rebooting the system everithing is fine for from a day to a week.

Two nodes (configered identicaly):

Management node
NDB node
Mysql node

I use CentOS OpenVZ virtual machines with 1Gb RAM and a single network adapter on each node.

Nodes were set up with Auto installer.

How to repeat:
This happens from time to time. I don't know the reason
[23 Nov 2013 14:55] None None 
Arbitration is disabled.
[23 Nov 2013 15:21] None None 
I'm raising Severity because consider this important
[25 Nov 2013 19:31] None None 
/home/mysql/MySQL_Cluster/49/ndb_49_cluster.log
is ful of:

[MgmtSrvr] WARNING  -- Failed to convert connection from '10.0.0.1:XXX' to transporter: line: 497 : Incorrect state for node 1 state: does nothing (2)
[25 Nov 2013 19:57] None None 
strace shows:

select(0, NULL, NULL, NULL, {0, 500000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 500000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 500000}) = 0 (Timeout)
....
[24 Mar 2016 23:30] MySQL Verification Team 
duplicate of 21141495
fixed in 7.3.10 and 7.4.7