Bug #67709 forced cluster shutdown after one node failed with error 2341
Submitted: 26 Nov 2012 12:47 Modified: 18 Jul 2016 20:07
Reporter: Benjamin J. Email Updates:
Status: Closed Impact on me:
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S1 (Critical)
Version:7.2.8 OS:Linux (Debian 6.0.1)
Assigned to: MySQL Verification Team CPU Architecture:Any

[26 Nov 2012 12:47] Benjamin J.
I get the following error every few days on one of my data nodes resulting in a fatal shutdown of all other nodes.

Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programmin$
Error: 2341
Error data: SimulatedBlock.cpp
Error object: DBTC (Line: 1386) 0x00000002
Program: ndbmtd
Pid: 1142 thr: 0
Version: mysql-5.5.27 ndb-7.2.8
Trace: /usr/local/mysql/analyse_cluster/ndb_data/ndb_5_trace.log.5 [t1..t7]

I can't say which query is causing the failure and I don't know which additional helpful informations I could provide as I can't run ndb_error_reporter on my Windows management node. I posted a new topic regarding error reports in the MySQL Cluster forum, I hope someone can tell me what to do.

How to repeat:
I don't know how to repeat this bug. We're running 2 clusters with 8 data nodes and 12 SQL nodes each. They have the same configuration and one is running without any problems. The other keeps on crashing and I haven't found any significant difference between both clusters.
[10 Jan 2013 17:55] MySQL Verification Team
Thank you for the report.

I can not repeat described behavior and also couldn't locate the uploaded file. Could you please help us and compress/upload the relevant files(cluster log, Node logs(*.out, ndb_<node_id>_trace.log.x, *.err)) once again? 

Also, similar issue is reported against 7.1.22 and tracked in Internal Bug DB Bug#14488185
[11 Feb 2013 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[18 Jul 2016 20:07] MySQL Verification Team
bug #14488185, fixed in
 - 7.0.38
 - 7.1.27
 - 7.2.11
 - 7.3.3

Exaustion of LongMessageBuffer memory while under heavy load cause data nodes running ndbmtd to fail.