MySQL Bugs: #71098: changing NoOfFragmentLogFiles and doing a cluster restart leads to error 2815

Bug #71098	changing NoOfFragmentLogFiles and doing a cluster restart leads to error 2815
Submitted:	5 Dec 2013 20:06	Modified:	10 Mar 2016 13:21
Reporter:	Jérémie Banier	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S3 (Non-critical)
Version:	mysql-5.6.11 ndb-7.3.2	OS:	Linux (debian - Wheezy)
Assigned to:	MySQL Verification Team	CPU Architecture:	Any
Tags:	config.ini, errno 2, integrity

Description:
Changing the value of NoOfFragmentLogFiles in the config.ini and doing a full cluster restart leads to a cluster failure with error:
Forced node shutdown completed. Occured during startphase 4. Caused by error 2815: 'File not found(Ndbd file system inconsistency error, please report a bug). Ndbd file system error, restart node initial'

How to repeat:
Change the value of NoOfFragmentLogFiles
in ndb_mgm do all restart 
cluster will faill with above error.

Suggested fix:
It should have complained about an invalid restart type when I try to bring it online and provide meaningfull instruction on how to do the rolling restart.

uploaded ndb_error_report_20131205194944.tar.bz2 to //support/incoming/ndb_error_report_20131205194944.tar.bz2

http://dev.mysql.com/doc/refman/5.6/en/mysql-cluster-ndbd-definition.html#ndbparam-ndbd-no...

says "Restart Type: IN"

so you need to do a rolling restart and restart each
data node with --initial so that it recreates its local
data directory with the new settings and then re-fetches
all data from its still running peer.

Looks as if you did a simple restart without --initial?

Indeed, I did a full restart without the --initial which was a mistake, still the error is not pointing you in the proper direction; it should state that it can't use that configuration or start instead of failling but give a warning in the logs telling you it didn't update the config because that parameter require an initial restart.

Hi,

the problem you experience after wrong type of restart is not "wrong restart type" but "wrong data on disk" that can happen for some other reason other then how it happened to you so the guessing game "why you entered that state" is really not happening. We report the problem and where it happens, we can't guess you came to that state by not following a procedure to change a parameter.

take care
Bogdan Kecman