Bug #44291 LocalProxy does not forward signals NODE_STATE_REP and CHANGE_NODE_STATE_REQ

• Submitted: 15 Apr 2009 12:40
• Modified: 16 Apr 2009 19:18
• Reporter: Ole John Aske
• Status: Closed
• Category: MySQL Cluster: Cluster (NDB) storage engine
• Severity: S2 (Serious)
• Version: 7.0
• OS: Any
• Assigned to: Ole John Aske
• CPU Architecture: Any

[15 Apr 2009 12:40] Ole John Aske

Description:
The signals GSN_NODE_STATE_REP and GSN_CHANGE_NODE_STATE_REQ are used during gracefull shutdown in order to change NodeState.startLevel. In particular the startLevel of the NodeState is set to SL_STOPPING_4 in order to prevent LQH from writing Global Checkpoint during a controlled shutdown.

As the above signals is not handled by the LQH proxy, NodeState::SL_STOPPING_4 never becomes visible at mt-LQH's. This breake the gracefull stopping of the GCP
mechanism and may cause inconsistent REDO logs to be produced (According to Jonas)

This may be related to bug#42564

How to repeat:
Verified by reading code

Suggested fix:
Implement handling of the above signals in LocalProxy.*pp

[15 Apr 2009 13:10] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72162

2962 Ole John Aske	2009-04-15
      bug#44291 LocalProxy should forward signals NODE_STATE_REP and CHANGE_NODE_STATE_REQ

[15 Apr 2009 13:14] Ole John Aske

As I will be on vacation for the next week, (until 26/4) the patch will not be commited until 27/4 earliest.

[16 Apr 2009 8:48] Jonas Oreland

review comments (or really patch):
change so that LocalProxy block also gets the signals.

[16 Apr 2009 8:49] Jonas Oreland

changes to committed patch

Attachment: bug44291.review.patch (text/x-patch), 2.20 KiB.

[16 Apr 2009 8:50] Jonas Oreland

discovered that this bug/fix
can be reproduced using "testSystemRestart -n to I3"

[16 Apr 2009 8:56] Bugs System

Pushed into 5.1.32-ndb-7.0.6 (revid:[e-mail]-20090416085227-bua6o7t3id2j5aiw) (version source revid:[e-mail]-20090416085227-bua6o7t3id2j5aiw) (merge vers: 5.1.32-ndb-7.0.6) (pib:6)

[16 Apr 2009 10:59] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72237

2907 Jonas Oreland	2009-04-16
      ndb - bug#44291
        Node-state is not propageded to all blocks in ndbmtd
        This can cause
        - inconsistent REDO when doing graceful shutdown
        - crash in Dbtux, when restarting cluster, and some nodes
          needs to perform node-recovery during system-restart
      
        Possibly other problems...but they have not been identified

[16 Apr 2009 19:18] Jon Stephens

Documented bugfix in the NDB-7.0.6 changelog as follows:

        Node state information was not propagated to all blocks of
        ndbmtd. This could cause:

            ·Inconsistent redo logs when performing a graceful shutdown

            ·Data node crashes when later restarting the cluster, data
            nodes needing to perform node recovery during the system
            restart, or both.