MySQL Bugs: #44299: Unable to explicitly specify nodeid in ndb-connectstring

Bug #44299	Unable to explicitly specify nodeid in ndb-connectstring
Submitted:	15 Apr 2009 15:47	Modified:	5 May 2010 22:56
Reporter:	Adam Dane	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S3 (Non-critical)
Version:	mysql-5.0	OS:	Linux (2.6.21)
Assigned to:	Magnus Blåudd	CPU Architecture:	Any
Tags:	mysql-5.0.67-telco-6.3.20

Description:
Two management nodes, two servers as storage + API, and two servers with just API.  I specified the connect strings for both the storage and API in the configuration, including node IDs.

I was able to start the management and storage nodes, as well as the two standalone API nodes.  The two API nodes that share servers with the storage nodes failed to start.

In addition, ndb_restore (I assume using the same connect string from the configuration) could not be run.

The error message: Configuration error: Could not alloc node id at 01.db.example.com port 1186: Id 3 configured as ndbd(NDB), connect attempted as mysqld(API).

How to repeat:
1. Start the management nodes
2. Start the storage nodes
3. Start the API nodes
4. Check error log for the API nodes on the servers with storage for error messages.

The following are the relevant configurations used:

--ndb_mgmd.cnf BEGIN--
[NDB_MGMD]
Id=1
HostName=01.db.example.com

[NDB_MGMD]
Id=2
HostName=02.db.example.com

[NDBD]
Id=3
HostName=03.db.example.com

[NDBD]
Id=4
HostName=04.db.example.com

[MYSQLD]
Id=5
HostName=03.db.example.com

[MYSQLD]
Id=6
HostName=04.db.example.com

[MYSQLD]
Id=7
HostName=05.db.example.com

[MYSQLD]
Id=8
HostName=06.db.example.com
--ndb_mgmd.cnf END--

--my.cnf BEGIN--
[mysqld]
ndbcluster
ndb-connectstring=nodeid=5,01.db.example.com,02.db.example.com
bind-address=03.db.example.com
#ndb-connectstring=nodeid=6,01.db.example.com,02.db.example.com
#bind-address=04.db.example.com
#ndb-connectstring=nodeid=7,01.db.example.com,02.db.example.com
#bind-address=05.db.example.com
#ndb-connectstring=nodeid=8,01.db.example.com,02.db.example.com
#bind-address=06.db.example.com
default-table-type=NDBCLUSTER

[MYSQL_CLUSTER]
ndb-connectstring=nodeid=3,01.db.example.com,02.db.example.com
#ndb-connectstring=nodeid=4,01.db.example.com,02.db.example.com
--my.cnf END--

Suggested fix:
mysql should look at [mysql_cluster] first, but override if there is a connect string specified in [mysqld]

Thank you, Verified with ndb-6.3.20 

Also specifying ndb-nodeid= in the my.cnf section does not function as a workaround. It gives - [ERROR] NDB: error (1) Could not initialize handle to management server: Illegal connect string :

Only removing the nodeid= portion from the connect strings allows both nodes to start.

Suggested fix: 

mysqld should look to [mysql_cluster] section first then to it's own [mysqld] section which should override.  If mysqld_multi is used each mysqld instance should look at its own [mysqldN] section for ndb-connectstring, ndb-nodeid and bind-address options.

The --ndb-nodeid and --ndb-connectstring parameters are concatenated into one string which is then passed to the NdbApi when it's started. Apparently there is a problem when concatenating.

There is one implementation for mysqld and one for ndbapi programs, they should both work the same way.

When using a ndbapi program it's also not possible to specify the same option twice(where the last should win).

For example:
$ tools/ndb_config --ndb-nodeid=3 --ndb-nodeid=4
*  1001: Illegal connect string
*        nodeid=3,nodeid=4

"Suggested solution: mysql should look at [mysql_cluster] first, but override if there is a connect string
specified in [mysqld]"

^ I probably missunderstod the problem. 

mysqld will upon startup read the [mysql_cluster] section if compiled with the WITH_NDBCLUSTER_STORAGE_ENGINE define. Will need to double check if it continues to read also the [mysqld] section if [mysql_cluster] is found.

However, the workaround to remove the "ndb-nodeid" specification from the more or less global [mysql_cluster] section is really easy. And I would recomend that.

Both [mysql_cluster] and [myqld] sections are read. Howevere in the case of ndb-nodeid, ndb-connectstring and ndb-mgmd there is the above mentioned concat bug causing non standard behaviour.

Standard behaviour for MySQL commandline arguments is to use the last given value. 

Example:
--ndb-nodeid=3 --ndb-nodeid=4
=> ndb_nodeid is set to 4

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/107355

Pushed into 5.1.44-ndb-7.0.15 (revid:magnus.blaudd@sun.com-20100505093008-81x1i3vzcyrq1y4o) (version source revid:magnus.blaudd@sun.com-20100505093008-81x1i3vzcyrq1y4o) (merge vers: 5.1.44-ndb-7.0.15) (pib:16)

Pushed to 7.0.15 and 7.1.4

Documented in the NDB-7.0.15 and 7.1.4 changelogs as follows:

      Specifying the node ID as part of the --ndb-connectstring option 
      to mysqld was not handled correctly. The fix for this issue 
      includes the following changes:

         ·Multiple occurrences of any of the mysqld options 
          --ndb-connectstring, --ndb-mgmd-host, and --ndb-nodeid are now   
          handled in the same way as with other MySQL server options, in 
          that the value set by the last occurrence of the option is the 
          value that is used by mysqld. 

          Now, if --ndb-nodeid is used, its value overrides that of any 
          nodeid setting used in --ndb-connectstring. For example, starting 
          mysqld with --ndb-connectstring=nodeid=1,10.100.1.100 
          --ndb-nodeid=3 now produces the same result as starting it with
          --ndb-connectstring=nodeid=3,10.100.1.100.

         ·The 1024-character limit on the length of the connectstring 
          is removed, and --ndb-connectstring is now handled in this 
          regard in the same way as other mysqld options.

         ·In the NDB API, a new constructor for Ndb_cluster_connection 
          is added which takes as arguments a connectstring and the node 
          ID to force the API node to use.

Also updated option descriptions and API docs.

Closed.