Description:
A MySQL NDB Cluster SQL/API node aborts during restart when the schema UUID stored in NDB differs from the schema UUID stored in the local MySQL data dictionary.
The affected SQL/API node starts normally, connects to all NDB storage nodes, reaches "ready for connections", then aborts in the NDB binlog/schema setup path:
```text
2026-06-29T03:54:46.774543Z [NDB] connection[0], NodeID: 7, all storage nodes connected
2026-06-29T03:54:46.775179Z [NDB] Binlog: Started
2026-06-29T03:54:46.775525Z [NDB] Binlog: Setup completed
2026-06-29T03:54:47.917822Z mysqld: ready for connections
2026-06-29T03:54:47.924057Z [NDB] Global schema lock acquired
2026-06-29T03:54:48.137523Z [NDB] Event 'REPL$mysql/ndb_schema' for table 'mysql.ndb_schema' already exists
2026-06-29T03:54:48.214765Z [NDB] Binlog: logging ./mysql/ndb_schema (UPDATED,USE_WRITE)
2026-06-29T03:54:48.215821Z [NDB] Schema UUID from NDB '53a92a25-6f14-11f1-a447-9ef7d609cc81' != from DD Schema UUID ''
mysqld: storage/ndb/plugin/ha_ndbcluster_binlog.cc:866:
bool Ndb_binlog_setup::setup(Thd_ndb*): Assertion `false' failed.
mysqld got signal 6
```
After this abort, the affected SQL/API node no longer listens on port 3306. `mysqladmin ping` fails with connection refused, and `ndb_mgm SHOW` reports the API node as not connected:
```text
id=7 (not connected, accepting connect from ndb2)
```
Other SQL/API nodes remain queryable, and the NDB data nodes remain connected. This appears to be a node-local SQL/API restart failure caused by NDB schema UUID vs local DD schema UUID mismatch.
The original observation happened after a full NDB cluster restart during a metadata-heavy workload. The relevant setup was:
```text
Topology:
1 management node
4 NDB data nodes, NodeId 2-5
4 SQL/API nodes, NodeId 6-9
affected SQL/API node: NodeId 7, HostName ndb2
Important config:
[ndbd default]
NoOfReplicas=2
DataMemory=512M
IndexMemory=128M
TimeBetweenGlobalCheckpoints=2000
TimeBetweenLocalCheckpoints=20
[mysqld]
NodeId=6 HostName=ndb1
NodeId=7 HostName=ndb2
NodeId=8 HostName=ndb3
NodeId=9 HostName=ndb4
```
The original run used a changed `ArbitrationDelay` value on the affected cluster:
```text
[ndb_mgmd default]
ArbitrationDelay=4294967039
```
However, after the bad metadata state had been created, the SQL/API node restart assertion remained reproducible even after restoring the cluster configuration to the baseline. Therefore, this report is not claiming that `ArbitrationDelay` alone is the root cause. The reportable issue is that mysqld aborts when restarting with an NDB schema UUID vs local DD schema UUID mismatch.
How to repeat:
The following is the sequence that produced the bad metadata state in my environment, followed by the reduced restart step that repeats the crash.
1. Start an NDB Cluster with the topology above.
2. Run metadata-heavy NDB DDL/DML on the SQL/API nodes:
```sql
CREATE DATABASE IF NOT EXISTS ndb_uuid_repro;
USE ndb_uuid_repro;
CREATE TABLE IF NOT EXISTS canary (
case_id VARCHAR(64) NOT NULL,
table_idx INT NOT NULL,
k INT NOT NULL,
node_hint INT NOT NULL,
payload VARCHAR(192) NOT NULL,
v BIGINT NOT NULL,
PRIMARY KEY(case_id, table_idx, k)
) ENGINE=NDBCLUSTER;
CREATE TABLE IF NOT EXISTS t0 (
case_id VARCHAR(64) NOT NULL,
table_idx INT NOT NULL,
k INT NOT NULL,
node_hint INT NOT NULL,
payload VARCHAR(192) NOT NULL,
v BIGINT NOT NULL,
PRIMARY KEY(case_id, table_idx, k),
KEY idx_node_hint(node_hint)
) ENGINE=NDBCLUSTER;
CREATE TABLE IF NOT EXISTS t1 (
case_id VARCHAR(64) NOT NULL,
table_idx INT NOT NULL,
k INT NOT NULL,
node_hint INT NOT NULL,
payload VARCHAR(192) NOT NULL,
v BIGINT NOT NULL,
PRIMARY KEY(case_id, table_idx, k),
KEY idx_node_hint(node_hint)
) ENGINE=NDBCLUSTER;
INSERT INTO canary VALUES
('r',-1,0,0,'pre:0',900000),
('r',-1,1,1,'pre:1',900001),
('r',-1,2,2,'pre:2',900002),
('r',-1,3,3,'pre:3',900003)
ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v);
INSERT INTO t0 VALUES
('r',0,0,0,'t0:pre:0',0),
('r',0,1,1,'t0:pre:1',1),
('r',0,2,2,'t0:pre:2',2),
('r',0,3,3,'t0:pre:3',3)
ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v);
INSERT INTO t1 VALUES
('r',1,0,1,'t1:pre:0',1000),
('r',1,1,2,'t1:pre:1',1001),
('r',1,2,3,'t1:pre:2',1002),
('r',1,3,0,'t1:pre:3',1003)
ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v);
```
3. Perform a full NDB cluster restart. In my environment this restarted the management node, data nodes, and SQL/API nodes. The original observation happened during recovery after this full cluster restart.
4. After the restart, run more metadata operations on the same schema:
```sql
USE ndb_uuid_repro;
CREATE TABLE IF NOT EXISTS canary (
case_id VARCHAR(64) NOT NULL,
table_idx INT NOT NULL,
k INT NOT NULL,
node_hint INT NOT NULL,
payload VARCHAR(192) NOT NULL,
v BIGINT NOT NULL,
PRIMARY KEY(case_id, table_idx, k)
) ENGINE=NDBCLUSTER;
INSERT INTO canary VALUES
('r',-1,0,0,'post:0',900000),
('r',-1,1,1,'post:1',900001),
('r',-1,2,2,'post:2',900002),
('r',-1,3,3,'post:3',900003)
ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v);
UPDATE t0 SET v = v + 7 WHERE case_id='r';
SELECT COUNT(*), COALESCE(SUM(v),0) FROM t0;
ALTER TABLE t0 ADD COLUMN ddl_touch_0 INT DEFAULT 0;
UPDATE t1 SET v = v + 7 WHERE case_id='r';
SELECT COUNT(*), COALESCE(SUM(v),0) FROM t1;
ALTER TABLE t1 ADD COLUMN ddl_touch_1 INT DEFAULT 0;
```
5. In the bad metadata state, restart only the affected SQL/API node, NodeId 7 / HostName ndb2:
```bash
# Replace service/container names with the local deployment names.
restart SQL/API node with NDB API NodeId 7
sleep 90
ndb_mgm -e SHOW
tail -300 /var/log/mysql/mysqld.err
ss -lntp | grep 3306 || true
mysqladmin --protocol=tcp -uroot -h127.0.0.1 -P3306 --connect-timeout=3 ping || true
```
Observed result:
The SQL/API node starts, connects to all NDB storage nodes, reaches "ready for connections", detects the schema UUID mismatch, and aborts:
```text
Schema UUID from NDB '53a92a25-6f14-11f1-a447-9ef7d609cc81' != from DD Schema UUID ''
ha_ndbcluster_binlog.cc:866: bool Ndb_binlog_setup::setup(Thd_ndb*): Assertion `false' failed.
mysqld got signal 6
```
Afterwards, port 3306 is not listening on that SQL/API node, `mysqladmin ping` returns connection refused, and `ndb_mgm SHOW` reports API NodeId 7 as not connected. Other SQL/API nodes remain usable.
Expected result:
The SQL/API node should not abort because of a schema UUID mismatch between NDB and the local DD. It should either resynchronize or repair the local DD metadata, or fail startup with a controlled diagnostic error and a clear recovery path.
Suggested fix:
Please check the schema UUID validation and error handling in `Ndb_binlog_setup::setup()`. A mismatch between the schema UUID stored in NDB and the local DD schema UUID should not lead to an unconditional assertion that terminates mysqld.
© 2026, Oracle Corporation and/or its affiliates
Description: A MySQL NDB Cluster SQL/API node aborts during restart when the schema UUID stored in NDB differs from the schema UUID stored in the local MySQL data dictionary. The affected SQL/API node starts normally, connects to all NDB storage nodes, reaches "ready for connections", then aborts in the NDB binlog/schema setup path: ```text 2026-06-29T03:54:46.774543Z [NDB] connection[0], NodeID: 7, all storage nodes connected 2026-06-29T03:54:46.775179Z [NDB] Binlog: Started 2026-06-29T03:54:46.775525Z [NDB] Binlog: Setup completed 2026-06-29T03:54:47.917822Z mysqld: ready for connections 2026-06-29T03:54:47.924057Z [NDB] Global schema lock acquired 2026-06-29T03:54:48.137523Z [NDB] Event 'REPL$mysql/ndb_schema' for table 'mysql.ndb_schema' already exists 2026-06-29T03:54:48.214765Z [NDB] Binlog: logging ./mysql/ndb_schema (UPDATED,USE_WRITE) 2026-06-29T03:54:48.215821Z [NDB] Schema UUID from NDB '53a92a25-6f14-11f1-a447-9ef7d609cc81' != from DD Schema UUID '' mysqld: storage/ndb/plugin/ha_ndbcluster_binlog.cc:866: bool Ndb_binlog_setup::setup(Thd_ndb*): Assertion `false' failed. mysqld got signal 6 ``` After this abort, the affected SQL/API node no longer listens on port 3306. `mysqladmin ping` fails with connection refused, and `ndb_mgm SHOW` reports the API node as not connected: ```text id=7 (not connected, accepting connect from ndb2) ``` Other SQL/API nodes remain queryable, and the NDB data nodes remain connected. This appears to be a node-local SQL/API restart failure caused by NDB schema UUID vs local DD schema UUID mismatch. The original observation happened after a full NDB cluster restart during a metadata-heavy workload. The relevant setup was: ```text Topology: 1 management node 4 NDB data nodes, NodeId 2-5 4 SQL/API nodes, NodeId 6-9 affected SQL/API node: NodeId 7, HostName ndb2 Important config: [ndbd default] NoOfReplicas=2 DataMemory=512M IndexMemory=128M TimeBetweenGlobalCheckpoints=2000 TimeBetweenLocalCheckpoints=20 [mysqld] NodeId=6 HostName=ndb1 NodeId=7 HostName=ndb2 NodeId=8 HostName=ndb3 NodeId=9 HostName=ndb4 ``` The original run used a changed `ArbitrationDelay` value on the affected cluster: ```text [ndb_mgmd default] ArbitrationDelay=4294967039 ``` However, after the bad metadata state had been created, the SQL/API node restart assertion remained reproducible even after restoring the cluster configuration to the baseline. Therefore, this report is not claiming that `ArbitrationDelay` alone is the root cause. The reportable issue is that mysqld aborts when restarting with an NDB schema UUID vs local DD schema UUID mismatch. How to repeat: The following is the sequence that produced the bad metadata state in my environment, followed by the reduced restart step that repeats the crash. 1. Start an NDB Cluster with the topology above. 2. Run metadata-heavy NDB DDL/DML on the SQL/API nodes: ```sql CREATE DATABASE IF NOT EXISTS ndb_uuid_repro; USE ndb_uuid_repro; CREATE TABLE IF NOT EXISTS canary ( case_id VARCHAR(64) NOT NULL, table_idx INT NOT NULL, k INT NOT NULL, node_hint INT NOT NULL, payload VARCHAR(192) NOT NULL, v BIGINT NOT NULL, PRIMARY KEY(case_id, table_idx, k) ) ENGINE=NDBCLUSTER; CREATE TABLE IF NOT EXISTS t0 ( case_id VARCHAR(64) NOT NULL, table_idx INT NOT NULL, k INT NOT NULL, node_hint INT NOT NULL, payload VARCHAR(192) NOT NULL, v BIGINT NOT NULL, PRIMARY KEY(case_id, table_idx, k), KEY idx_node_hint(node_hint) ) ENGINE=NDBCLUSTER; CREATE TABLE IF NOT EXISTS t1 ( case_id VARCHAR(64) NOT NULL, table_idx INT NOT NULL, k INT NOT NULL, node_hint INT NOT NULL, payload VARCHAR(192) NOT NULL, v BIGINT NOT NULL, PRIMARY KEY(case_id, table_idx, k), KEY idx_node_hint(node_hint) ) ENGINE=NDBCLUSTER; INSERT INTO canary VALUES ('r',-1,0,0,'pre:0',900000), ('r',-1,1,1,'pre:1',900001), ('r',-1,2,2,'pre:2',900002), ('r',-1,3,3,'pre:3',900003) ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v); INSERT INTO t0 VALUES ('r',0,0,0,'t0:pre:0',0), ('r',0,1,1,'t0:pre:1',1), ('r',0,2,2,'t0:pre:2',2), ('r',0,3,3,'t0:pre:3',3) ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v); INSERT INTO t1 VALUES ('r',1,0,1,'t1:pre:0',1000), ('r',1,1,2,'t1:pre:1',1001), ('r',1,2,3,'t1:pre:2',1002), ('r',1,3,0,'t1:pre:3',1003) ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v); ``` 3. Perform a full NDB cluster restart. In my environment this restarted the management node, data nodes, and SQL/API nodes. The original observation happened during recovery after this full cluster restart. 4. After the restart, run more metadata operations on the same schema: ```sql USE ndb_uuid_repro; CREATE TABLE IF NOT EXISTS canary ( case_id VARCHAR(64) NOT NULL, table_idx INT NOT NULL, k INT NOT NULL, node_hint INT NOT NULL, payload VARCHAR(192) NOT NULL, v BIGINT NOT NULL, PRIMARY KEY(case_id, table_idx, k) ) ENGINE=NDBCLUSTER; INSERT INTO canary VALUES ('r',-1,0,0,'post:0',900000), ('r',-1,1,1,'post:1',900001), ('r',-1,2,2,'post:2',900002), ('r',-1,3,3,'post:3',900003) ON DUPLICATE KEY UPDATE payload=VALUES(payload), v=VALUES(v); UPDATE t0 SET v = v + 7 WHERE case_id='r'; SELECT COUNT(*), COALESCE(SUM(v),0) FROM t0; ALTER TABLE t0 ADD COLUMN ddl_touch_0 INT DEFAULT 0; UPDATE t1 SET v = v + 7 WHERE case_id='r'; SELECT COUNT(*), COALESCE(SUM(v),0) FROM t1; ALTER TABLE t1 ADD COLUMN ddl_touch_1 INT DEFAULT 0; ``` 5. In the bad metadata state, restart only the affected SQL/API node, NodeId 7 / HostName ndb2: ```bash # Replace service/container names with the local deployment names. restart SQL/API node with NDB API NodeId 7 sleep 90 ndb_mgm -e SHOW tail -300 /var/log/mysql/mysqld.err ss -lntp | grep 3306 || true mysqladmin --protocol=tcp -uroot -h127.0.0.1 -P3306 --connect-timeout=3 ping || true ``` Observed result: The SQL/API node starts, connects to all NDB storage nodes, reaches "ready for connections", detects the schema UUID mismatch, and aborts: ```text Schema UUID from NDB '53a92a25-6f14-11f1-a447-9ef7d609cc81' != from DD Schema UUID '' ha_ndbcluster_binlog.cc:866: bool Ndb_binlog_setup::setup(Thd_ndb*): Assertion `false' failed. mysqld got signal 6 ``` Afterwards, port 3306 is not listening on that SQL/API node, `mysqladmin ping` returns connection refused, and `ndb_mgm SHOW` reports API NodeId 7 as not connected. Other SQL/API nodes remain usable. Expected result: The SQL/API node should not abort because of a schema UUID mismatch between NDB and the local DD. It should either resynchronize or repair the local DD metadata, or fail startup with a controlled diagnostic error and a clear recovery path. Suggested fix: Please check the schema UUID validation and error handling in `Ndb_binlog_setup::setup()`. A mismatch between the schema UUID stored in NDB and the local DD schema UUID should not lead to an unconditional assertion that terminates mysqld.