Bug #78746 mysqlrpladmin switchover does not change the read_only setting
Submitted: 8 Oct 2015 3:27 Modified: 22 Nov 2016 10:27
Reporter: monty solomon Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Utilities Severity:S1 (Critical)
Version:1.5.6 OS:Any
Assigned to: CPU Architecture:Any

[8 Oct 2015 3:27] monty solomon
Description:
After executing the mysqlrpladmin switchover command the new master is still running in read-only mode and the old master is still running in read-write mode.

mysqlrpladmin does not change the master to read_only before swapping roles.

How to repeat:
Create a master/slave cluster and check the read_only settings on the master and the slave to become the new master

% mysql -h lively-mode -sNe'select @@read_only'
0

% mysql -h bitter-unit -sNe'select @@read_only'
1

Execute mysqlrpladmin to switch the master

% mysqlrpladmin switchover --verbose --discover-slaves-login=.myrpladmin.cnf[lively-mode] --master=.myrpladmin.cnf[lively-mode] --new-master=.myrpladmin.cnf[bitter-unit] --rpl-user=SUSR_Repl:redacted --log=mysqlrpladmin.log --demote-master

Check the read_only settings on the new master and former master and observe that they were not changed

% mysql -h bitter-unit -sNe'select @@read_only'
1

% mysql -h lively-mode -sNe'select @@read_only'
0

Suggested fix:
Change the following line in connect() in server.py to set the value to True/False instead of to ON/OFF

	self.read_only = self.show_server_variable("READ_ONLY")[0][1]

or change the following line in set_read_only() in server.py to check for the value ON/OFF instead of True/False

	if not self.read_only:
[8 Oct 2015 4:24] monty solomon
The following statement in set_read_only() is missing parentheses

            return self.exec_query("SET @@GLOBAL.READ_ONLY = %s" %
                                   "ON" if on else "OFF")

It should read

            return self.exec_query("SET @@GLOBAL.READ_ONLY = %s" %
                                   ("ON" if on else "OFF"))
[8 Oct 2015 4:37] monty solomon
The code sets the master to be read-only then it stops all the slaves and then it erroneously sets the master to be read-write again. The master should stay blocked and read-only. Only if the master is demoted and connected as a slave to the promoted master should it be set to read-write again.

        lock_ftwrl = Lock(self.master, [], lock_options)
        self.master.set_read_only(True)

        ...

        # Stop all slaves
        self._report("# Stopping slaves.")
        self.run_cmd_on_slaves("stop", not self.verbose)

        # Unblock master
        self.master.set_read_only(False)
        lock_ftwrl.unlock()
[8 Oct 2015 4:41] monty solomon
The code to demote the master does not change the read-only setting on the demoted master. It may assume that demoted master was left in a read-only state.
[8 Oct 2015 5:08] monty solomon
The code does not set the promoted master to be read/write.

The failover() method should be checked to make sure it does not have similar issues.
[3 Nov 2016 10:53] Flex Gao
Still has this problem on 1.6.4
[22 Nov 2016 10:27] Chiranjeevi Battula
Hello monty solomon,

Thank you for the bug report.
This is most likely duplicate of Bug #76522, please see Bug #76522.

Thanks,
Chiranjeevi.