Bug #47528 Data Node fails in SimulatedBlock.cpp -- when using NoOfReplicas=3
Submitted: 22 Sep 2009 21:19 Modified: 18 Oct 2010 8:53
Reporter: David Enbal Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:mysql-5.1-telco-7.0 OS:Solaris
Assigned to: CPU Architecture:Any
Tags: 7.0.7
Triage: Triaged: D2 (Serious) / R6 (Needs Assessment) / E6 (Needs Assessment)

[22 Sep 2009 21:19] David Enbal
Description:
I did a database import from a mysqldump dump file (8GB in size) and two data nodes crashed. One because of: 

Time: Tuesday 22 September 2009 - 23:46:50
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: SimulatedBlock.cpp
Error object: QMGR (Line: 510) 0x0000000e
Program: /opt/mysql/mysql/bin/ndbmtd
Pid: 5498
Trace: /opt/mysql/mysql/data/ndb_2_trace.log.2 /opt/mysql/mysql/data/ndb_2_trace.log.2_t1 /opt/mysql/mysql/data/ndb_2_trace.log.2_t2 /opt/mysql/mysql/d

The other because of:
Time: Tuesday 22 September 2009 - 23:47:03
Status: Temporary error, restart node
Message: System error, node killed during node restart by other node (Internal error, programming error or missing error message, please report a bug)
Error: 2303
Error data: Node 4 killed this node because GCP stop was detected
Error object: NDBCNTR (Line: 263) 0x0000000a
Program: /opt/mysql/mysql/bin/ndbmtd
Pid: 24599
Trace: /opt/mysql/mysql/data/ndb_4_trace.log.8 /opt/mysql/mysql/data/ndb_4_trace.log.8_t1 /opt/

How to repeat:
Import a mysqldump dump file.
[23 Sep 2009 7:37] Jonas Oreland
ndb_error_reporter
[23 Sep 2009 7:42] David Enbal
it is already at yours. Same system as in #47530. The file is at your FTP called ndb_error_report_20090923085331.tar.bz2.
[23 Sep 2009 10:31] Gustaf Thorslund
Tried it with NoOfReplicas=2?
[23 Sep 2009 10:50] David Enbal
Tried that with only 2 replica and it doesn't repeat itself.
[23 Mar 2010 7:48] Jonas Oreland
Any progress on this or ??
[23 Mar 2010 20:27] David Enbal
I am working now with just 2 replicas so the situation stays the same. It works with 2 replicas just fine. I don't know if it fails on 3 because I can't allow myself more fails with customer's system.
[18 Oct 2010 8:53] Gustaf Thorslund
if NoOfReplicas > 2 then bug in 7.0.7; end if;