Description:
We are running MySQL Cluster which has 2 data nodes, and 1 management node.
And trying to add new 2 data nodes.
I found that all data nodes cannot boot after `ALTER TABLE ... REORGANIZE PARTITION`.
(data node log when booting.)
```
2022-03-11 14:14:05 [ndbd] INFO -- NDB start phase 1 completed
2022-03-11 14:14:05 [ndbd] INFO -- Start phase 2 completed
2022-03-11 14:14:05 [ndbd] INFO -- Phase 2 did more initialisations, master accepted our start, we started REDO log initialisations
2022-03-11 14:14:05 [ndbd] INFO -- Start NDB start phase 2
2022-03-11 14:14:06 [ndbd] INFO -- Request permission to start our node from master Starting
2022-03-11 14:14:06 [ndbd] INFO -- Request permission to start our node from master Completed
2022-03-11 14:14:06 [ndbd] INFO -- NDB start phase 2 completed
2022-03-11 14:14:06 [ndbd] INFO -- Start phase 3 completed
2022-03-11 14:14:06 [ndbd] INFO -- Phase 3 locked the data dictionary, performed local connecti on setups, we asked for permission to start our node
2022-03-11 14:14:06 [ndbd] INFO -- Start NDB start phase 3
2022-03-11 14:14:06 [ndbd] INFO -- NDB start phase 3 completed
2022-03-11 14:14:06 [ndbd] INFO -- Start phase 4 completed
2022-03-11 14:14:06 [ndbd] INFO -- Phase 4 continued preparations of the REDO log
2022-03-11 14:14:06 [ndbd] INFO -- Request copying of distribution and dictionary information f rom master Starting
2022-03-11 14:14:06 [ndbd] INFO -- LDM(1):Ready to start execute REDO log phase, prepare REDO l og phase completed
2022-03-11 14:14:06 [ndbd] INFO -- Receive arbitrator node 1 [ticket=486100011521732d]
2022-03-11 14:14:08 [ndbd] INFO -- Copying of dictionary information from master Starting
Found pending trans (0) - committing
commit alter 93
commit alter 184
commit alter 590
commit alter 688
Found pending trans (742) - aborting
Found pending trans (743) - aborting
Found pending trans (744) - aborting
Found pending trans (745) - aborting
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 1
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 741
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 699
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 700
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 701
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 702
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 703
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 704
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 705
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 706
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 707
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 708
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 709
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 710
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 711
2022-03-11 14:14:08 [ndbd] INFO -- Restart recreating table with id = 712
error: [ code: 21029 line: 27125 node: 8 count: 1 status: 0 key: 0 name: '' ]
2022-03-11 14:14:08 [ndbd] INFO -- Failed to recreate object 712 during restart, error 21029.
2022-03-11 14:14:08 [ndbd] INFO -- DBDICT (Line: 5259) 0x00000002
2022-03-11 14:14:08 [ndbd] INFO -- Error handler shutting down system
2022-03-11 14:14:08 [ndbd] ALERT -- Node 8: Forced node shutdown completed. Occured during start phase 5. Caused by error 2355: 'Failure to restore schema(Resource configuration error). Permanent error, external action needed'.
```
It seems to wrong with table id=712.
Whereas, I restored all data nodes and use `ALTER TABLE ... ALGORITHM=COPY, ORGANIZE PARTITION`, I can restart data nodes correctly.
How to repeat:
add new data nodes.
reference:
- https://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-online-add-node-example.html
1. setup MySQL Cluster with 2 data nodes
2. add 2 data nodes
3. create new nodegroup
4. reorganize partion
5. stop a data node (id=8)
6. start a data node (id=8)