Description:
When joining more than one instance simultaneously to a cluster using addInstance (via automation), I get this error if a clone is already running:
* Configuring ClusterSet managed replication channel...
** Changing replication source of 10.170.254.92:3306 to 10.162.254.74:3306
Monitoring recovery process of the new cluster member. Press ^C to stop monitoring and let it continue in background.
ESC[1mClone based state recovery is now in progress.ESC[0m
ESC[36mNOTE: ESC[0mA server restart is expected to happen as part of the clone process. If the
server does not support the RESTART command or does not come back after a
while, you may need to manually start it back.
* Waiting for clone to finish...
ESC[36mNOTE: ESC[0m10.170.254.92:3306 is being cloned from 10.170.255.16:3306
ESC[31mERROR: ESC[0mThe clone process has failed: Clone Donor Error: 3634 : Too many concurrent clone operations. Maximum allowed - 1.. (3862)
ESC[31mERROR: ESC[0mClone Donor Error: 3634 : Too many concurrent clone operations. Maximum allowed - 1..
It'd be helpful if addInstance had an option to wait for cloning to become available rather than just exiting.
How to repeat:
Start two instances at the same time on a cluster that needs a non-trivial amount of time to clone.
Suggested fix:
add a 'waitClone' option that takes a maximum number of seconds to wait for cloning to become available.
cluster.addInstance('new instance', {waitClone: 86400})