MySQL Bugs: #67551: Data node crashes; Shutting down node due to lack of LCP fragment scan progress.

Bug #67551	Data node crashes; Shutting down node due to lack of LCP fragment scan progress.
Submitted:	10 Nov 2012 21:41	Modified:	14 Nov 2012 8:17
Reporter:	Sajjad Tariq	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S2 (Serious)
Version:	mysql-5.5.27 ndb-7.2.8	OS:	Linux (x86_64)
Assigned to:		CPU Architecture:	Any
Tags:	crash, data node, MySQL Cluster

Description:
Data node crashes;

ndb_7_error.log:
Time: Saturday 10 November 2012 - 04:07:31
Status: Temporary error, restart node
Message: LCP fragment scan watchdog detected a problem.  Please report a bug. (Internal error, programming error or missing error message, please report a bug)
Error: 7200
Error data: Please report this as a bug. Provide as much info as possible, expecially all the ndb_*_out.log files, Thanks. Shutting down node due to lack of LCP fragment scan progress
Error object: DBLQH (Line: 23661) 0x00000002
Program: ndbmtd

ndb_1_cluster.log:
2012-11-10 04:07:25 [MgmtSrvr] INFO     -- Node 7: -- Node 7 LCP STATE --
2012-11-10 04:07:25 [MgmtSrvr] INFO     -- Node 7: ParticipatingDIH = 00000000000001f8
2012-11-10 04:07:25 [MgmtSrvr] INFO     -- Node 7: ParticipatingLQH = 00000000000001f8
2012-11-10 04:07:25 [MgmtSrvr] INFO     -- Node 7: m_LCP_COMPLETE_REP_Counter_DIH = [SignalCounter: m_count=6 00000000000001f8]
2012-11-10 04:07:25 [MgmtSrvr] INFO     -- Node 7: m_LCP_COMPLETE_REP_Counter_LQH = [SignalCounter: m_count=6 00000000000001f8]
2012-11-10 04:07:25 [MgmtSrvr] INFO     -- Node 7: m_LAST_LCP_FRAG_ORD = [SignalCounter: m_count=6 00000000000001f8]

2012-11-10 04:07:41 [MgmtSrvr] ALERT    -- Node 7: Forced node shutdown completed. Caused by error 7200: 'LCP fragment scan watchdog detected a problem.  Please rep
ort a bug.(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.

How to repeat:
Not sure.

Suggested fix:
Fix LCP fragment scan progress.

my cluster(mysql-5.5.27 ndb-7.2.8) has this problem too, but just warning:
2012-11-14 04:50:15 [MgmtSrvr] WARNING  -- Node 17: LCP Frag watchdog : No progress on table 4294967295, frag 4294967295 for 20 s.  184467440737095

before ndb-7.2.6 does not have this problem.
in the mysql online document: in 80 seconds continuous generated 7 same warnings,
the ndbmtd will restart. 

I do not know what causes the problem ? High loadavg or network ?

Thank you for taking the time to write to us, but this is not a bug.

This is a known behavior, reported in internal Oracle bug # are 14495364, 14075825

Also, this behavior is well documented in the manual page:

see, http://dev.mysql.com/doc/refman/5.5/en/mysql-cluster-news-5-5-22-ndb-7-2-6.html