Bug #50930 Cluster crash when node is restarted 255 times and encountering error 1204
Submitted: 4 Feb 2010 20:41 Modified: 26 Feb 2010 12:15
Reporter: Jonas Oreland Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:mysql-5.1-telco-6.2 OS:Any
Assigned to: Jonas Oreland CPU Architecture:Any

[4 Feb 2010 20:41] Jonas Oreland
Description:
If nodes are restarted 256 times in one node-group
and applications are running traffic so that it would
encounter error 1204, the alive node in the node group
will crash, causing cluster crash (unless using >= 3 replicas)

Note: error does only occur if 1204 is encountered on each 256'th restart
the subsequent 255 will handle 1204 fine.

How to repeat:
.

Suggested fix:
.
[4 Feb 2010 21:13] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99352

3072 Jonas Oreland	2010-02-04
      ndb - bug#50930 - fix aritmetic when computing diff of the fragdistkeys
[4 Feb 2010 21:19] Jonas Oreland
pushed to 6.2.19 6.3.32 and 7.0.13
[4 Feb 2010 21:19] Bugs System
Pushed into 5.1.41-ndb-7.0.13 (revid:jonas@mysql.com-20100204211523-rl0t4mrgql71vj1r) (version source revid:jonas@mysql.com-20100204211523-rl0t4mrgql71vj1r) (merge vers: 5.1.41-ndb-7.0.13) (pib:16)
[8 Feb 2010 14:33] Jon Stephens
Documented in the NDB-6.2.19, 6.3.32, 7.0.13, and 7.1.2 changelogs as follows:

        When using 1 or 2 replicas, if data nodes from one node group
        were restarted 256 times and applications were running traffic
        such that it would encounter error 1204 (Temporary failure,
        distribution changed), the live node in the node group would
        crash, causing the cluster to crash as well. The crash occurred
        only when the error was encountered on the 256th restart; having
        the error on any previous or subsequent restart did not cause
        any problems.

Closed.
[26 Feb 2010 10:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/101570

3082 Jonas Oreland	2010-02-26
      ndb - correction for bug#50930, any of the 2 values can be 1 bigger...
[26 Feb 2010 11:23] Bugs System
Pushed into 5.1.41-ndb-6.3.32 (revid:jonas@mysql.com-20100226111652-0lwl2dcevyz5fwdu) (version source revid:jonas@mysql.com-20100226111415-0b6ihdp3mgt6joyp) (merge vers: 5.1.41-ndb-6.3.32) (pib:16)
[26 Feb 2010 11:24] Bugs System
Pushed into 5.1.41-ndb-7.0.13 (revid:jonas@mysql.com-20100226111758-yi418obub9ukgqy7) (version source revid:jonas@mysql.com-20100226111758-yi418obub9ukgqy7) (merge vers: 5.1.41-ndb-7.0.13) (pib:16)
[26 Feb 2010 12:15] Jonas Oreland
reclose