MySQL Bugs: #50906: Adding data nodes online needs re-start of nodes using dynamic IDs

Bug #50906	Adding data nodes online needs re-start of nodes using dynamic IDs
Submitted:	4 Feb 2010 10:05	Modified:	11 Feb 2010 20:21
Reporter:	Geert Vanderkelen	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Documentation	Severity:	S3 (Non-critical)
Version:	mysql-5.1-ndb-7.0	OS:	Any
Assigned to:	Jon Stephens	CPU Architecture:	Any

Description:
When adding a node gorup online one needs to make sure that the node IDs for the new data nodes are not used by other nodes, e.g. SQL/API nodes.
Therefor, we advice to assign the first SQL/API nodes with a higher ID, e.g. 20, so data nodes can be added with have lower IDs.

For example, good practice is:

[ndbd]
id=3
[ndbd]
id=4
[api]
id=20
..

Adding data nodes:

[ndbd]
id=3
[ndbd]
id=4
[ndbd]
id=5
[ndbd]
id=6
[api]
id=20

How to repeat:
Add a new data node group with data nodes IDs which are used by already running SQL or API nodes, which use dynamic node IDs.

For example, if you add data nodes and the first 2 SQL nodes were not configured with explicit IDs you get this:

[ndbd(NDB)] 4 node(s)
id=3 @192.168.1.11 (mysql-5.1.39 ndb-7.0.9, Nodegroup: 0, Master)
id=4 @192.168.1.16 (mysql-5.1.39 ndb-7.0.9, Nodegroup: 0)
id=5 (not connected, accepting connect from datanodeC)
id=6 (not connected, accepting connect from datanodeD)

# The above 5 and 6 are still mysqld's running with IDs 5
# and 6. Data nodes can't be started then. Also, you
# can't restart node 3.

[ndb_mgmd(MGM)] 2 node(s)
id=1 @192.168.1.1 (mysql-5.1.39 ndb-7.0.9)
id=2 @192.168.1.6 (mysql-5.1.39 ndb-7.0.9)

[mysqld(API)] 80 node(s)
id=7 (not connected, accepting connect from mysqldA)
id=8 (not connected, accepting connect from mysqldB)
id=9 (not connected, accepting connect from any host)
id=10 (not connected, accepting connect from any host)

Suggested fix:
For example, put some extra remark in the docs near:
 4. Perform a rolling restart of any SQL or API nodes connected to the MySQL Cluster. 
On the following page:
 http://dev.mysql.com/doc/mysql-cluster-excerpt/5.1/en/mysql-cluster-online-add-node-basics...

Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Note that we already tell users to perform a rolling restart of the cluster, which is defined elsewhere as restarting all nodes in the cluster; we do not tell the user or give the user any indication that only restarting the data nodes is sufficient.

Please also add a note which indicates that for clusters using dynamically assigned API node IDs, (where no gap between NDBD and API node IDs was built in) that these API nodes need to be restarted at step 3 prior to restart of NDBD so that the API nodes and NDBD nodes both know that the APIs are using new IDs and that a new gap is formed between the statically assigned IDs of the NDBD nodes and the dynamically assigned one of the API nodes.

Huh?

Thank you for your bug report. This issue has been addressed in the documentation. The updated documentation will appear on our website shortly, and will be included in the next release of the relevant products.

New commit is here: http://lists.mysql.com/commits/100028