MySQL Bugs: #100521: Cancelling active GR auto-initialization

Bug #100521	Cancelling active GR auto-initialization
Submitted:	13 Aug 2020 16:21	Modified:	17 Aug 2020 14:51
Reporter:	Robert Azzopardi	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server	Severity:	S3 (Non-critical)
Version:	8.0.21	OS:	Windows (Windows 2016)
Assigned to:		CPU Architecture:	Any
Tags:	Cancelling active GR auto-initialization

Description:
Hi,

following an upgrade from MySQL 8.0.18 to 8.0.21 we noticed that when the cluster needs to be recovered from dba.rebootClusterFromCompleteOutage(), it reports that
following,

NOTE: Cancelling active GR auto-initialization at xxx.xxx.xxx.xxx:3306

After 20 to 30 minutes the Cluster will be successfully rebooted. We have replicated this on various test machines (Windows) and it result is always the same. 

can you please advise if this is a normal behavior or is it a bug as we never experienced such issues on version 8.0.18?

Thanks

Robert

How to repeat:
windows

Stop all cluster nodes.
Start all cluster nodes again.

Run dba.rebootClusterFromCompleteOutage() to scan/join for all nodes 

then you get the following message

NOTE: Cancelling active GR auto-initialization at xxx.xxx.xxx.xxx:3306

Eventually after 20-30 minutes cluster will recover.

Hi,

This looks like normal behavior to me. Can you please upload
 - all config files (from all nodes)
 - all log files (from all nodes)

thanks

Files uploaded. From the Err logs it looks that nodes cannot be reached over the network, however it is not the case. All nodes are reachable and tested. This issue occurred as soon as we upgraded MySQL version from 8.0.18 to 8.0.21 with same data size. 

thanks

Hi Mr. Azzopardi,

My esteemed colleague has asked not only for all logs, but also all configuration files for all configuration files as well.

Hence, beside configuration, we miss the relevant binary log files and a full description of the commands issued that led to this auto-initialisation problem.

We are waiting on your feedback.

Also, you have to send us some other info.

Like, how did you upgrade from  8.0.18 to 8.0.21, then what has caused the necessity that the cluster needs to be recovered from dba.rebootClusterFromCompleteOutage(). What happened to cause this event ??? If it is a hardware or OS error, it could lead to the corruption that can not be fixed. In that case, you have to start from the scratch on that node.

If the Cluster has successfully rebooted, then what is the bug that you are reporting actually ???

Hi,

can you please indicate what other files you need and their location so i send them over?

secondly we power off the VMs before we do Windows Update such that we take a snapshot will all the VMs powered off. Then we start the VMs and everytime we have to recover from the outage despite all VMs have been gracefully shutdown one at a time.  Is there any other way to shutdown the cluster without causing this issue?

With version 8.0.18, we used to do the same procedure but recover from outage used to take a few seconds but now its taking 30 mins..

Thanks

Hi,

If you are not shutting down MySQL server before powering off VM or the entire hardware, then this report can not be considered a bug.

Hence, if you confirm that you are not doing proper shutdown, we shall label this report as "Not a bug".

All nodes are being shut down gracefully one at a time by stopping the Mysql service. This was never an issue prior the upgrade using the same procedure. 

Thanks

Hi Mr. Azzopardi,

You have replied to us that all nodes are shut down gracefully.

This raises a question of what has caused the necessity that the cluster needs to be recovered from dba.rebootClusterFromCompleteOutage() ???? 

This remains to be answered properly.

To answer your last question ....

We also need your  configuration files for our server, then we need the relevant binary log files and a full description of the commands issued that led to this auto-initialisation problem.

Also,  how did you upgrade from  8.0.18 to 8.0.21 ???

Please, provide us with all info that we requested, as well as with answer to the questions in the previous comment.

How do i get the configuration and binary files?

Hi,

This is all described in our 8.0 Reference Manual, freely available on our dev.mysql.com "Documentation" pages.

This bug seems like a duplicate of https://bugs.mysql.com/bug.php?id=98948