| Bug #107057 | Innodb Cluster fails to spot when a database is put into offline mode manually | ||
|---|---|---|---|
| Submitted: | 19 Apr 2022 15:26 | Modified: | 21 May 2022 16:09 |
| Reporter: | IGG t | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | Shell AdminAPI InnoDB Cluster / ReplicaSet | Severity: | S2 (Serious) |
| Version: | 8.0.27 | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
[26 Apr 2022 12:02]
MySQL Verification Team
Hi, Not sure if this is a bug or design decision. Will verify and let GR team decide. BTW, did you try to rescan()? https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/classmysqlsh_1_1dba_1_1_cluster.h... thanks for the report
[21 May 2022 16:09]
Margaret Fisher
Posted by developer: Changelog entry added for MySQL 8.0.30: InnoDB Cluster’s Cluster.status() command did not check whether a server instance had been put into offline mode manually by setting the offline_mode system variable. The status was still reported as ONLINE, but some InnoDB Cluster operations failed. The command now checks for this situation and reports the status as OFFLINE if that has been set.
Description: I have a 5 node innodb cluster set up in single primary mode. If I run cluster.status() it correctly reports that there are 5 nodes, with one primary. If I log onto one of the secondary nodes, and "SET @@global.offline_mode = ON;" it puts this database into offline mode. This can be confirmed by trying to open a new connection to the database. However if you look at cluster.status() it still reports that all 5 nodes are still available. If I the move the primary instance to the offline node, it allows it to happen, and even claims it has succeeded. Trying to open a connection to the Primary Node (in this case using mysqlrouter for example) returns an error as the Primary is now offline. How to repeat: Set up a 5 node cluster with the primary on node1. on node1 using mysqlshell run cluster.status() to confirm the primary is node1. Open a connection direct to node2, and run SET @@global.offline_mode = ON; back on node1 using mysqlshell run cluster.status() to see that node2 is still showing as available. now run cluster.setPrimaryInstance('node2:3306') It should tell you that "The instance 'node2:3306' was successfully elected as primary." run cluster.status() to see that node2 is now the primary now try and open a connection to the Primary node (maybe using mysqlrouter to automatically re-direct to the current primary). It will report that the primary is offline, and therefore you can't connect to it. Suggested fix: Innodb Router needs to be aware when a database is moved into offline mode, as it can lead to the Primary being on a node that is "offline"