Bug #70452 Internal program error (failed ndbrequire)
Submitted: 27 Sep 2013 15:09 Modified: 23 Mar 2016 13:06
Reporter: Pavel Baranov Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:7.3.2 OS:Linux (centos 6.4 64bit)
Assigned to: MySQL Verification Team CPU Architecture:Any

[27 Sep 2013 15:09] Pavel Baranov 
Description:
All of a sudden 2 NDB nodes disconnected. Happened after running this query:
set global max_connections = 1000;
On mysqld.

Will be attaching logs.

ndb_12_error.log :

Current byte-offset of file-pointer is: 568

Time: Friday 27 September 2013 - 07:56:03
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: SimulatedBlock.cpp
Error object: DBTC (Line: 1374) 0x00000002
Program: ndbd
Pid: 11716
Version: mysql-5.6.11 ndb-7.3.2
Trace: /mysql/cluster/ndb_data/ndb_12_trace.log.1 [t1..t1]
***EOM***

How to repeat:
Change default max_connections after starting a cluster. (not sure if that caused the issue though)
[30 Sep 2013 21:12] Pavel Baranov 
this happens NOT exactly after doing "set global ..."
but actually connecting with application with 400+ connections.
[2 Dec 2013 20:27] Pavel Baranov 
63 days without a single response... says a lot about oracle
[13 Nov 2014 13:45] Mikael Ronström 
The problem is that you run out of the resource LongMessageBuffer.
The default for this buffer has been stepped considerably in later versions, it is now
64 MB by default.

The problem is that many connections starts large queries at the same time and then this
resource isn't big enough.