MySQL Bugs: #73254: Can mysql fabric automatically restart replication after failover?

Bug #73254	Can mysql fabric automatically restart replication after failover?
Submitted:	10 Jul 2014 9:00	Modified:	3 Nov 2014 15:11
Reporter:	george sibley	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Fabric	Severity:	S3 (Non-critical)
Version:	1.4.3	OS:	Linux (ubuntu 3.11.0-18-generic)
Assigned to:	Mats Kindahl	CPU Architecture:	Any

Description:
When a database server becomes 'faulty' in a fabric group and we re-introduce it after bringing it back up, we have to restart the replication processing to get lost changes. We seem to have to run a few commands on the slave to get the re-introduced slave to retrieve replication updates from the newly promoted 'Primary'.

Here's a list of what we did:-

1. Shutdown the 'Primary' server. The 'Secondary' is successfully promoted to 'Primary'

2. Perform some updates on the new 'Primary'

3. re-introduce the 'Faulty' slave back into the server group.

> mysqlfabric server set_status <database instance guid of 'faulty' server> SPARE
> mysqlfabric server set_status <database instance guid of 'faulty' server> SECONDARY

4. Check the status of the slave's replication.

> show slave status;

We see that the slave is 'Waiting for master to send events', which looks healthy.

5. We then have to enter the following to receive the lost updates from the newly promoted Master.

> stop slave;
> reset master;
> start slave;

The updates are then received successfully on the new slave.

I would've thought that the fabric processing would handle this last piece of processing logic for us? Are we missing some configuration or setup option?

How to repeat:
The issue can be repeated consistently using the instructions supplied in the description.

Hi George,

The old (i.e. faulty) master should automatically fetch new transactions, that were sent to the new master while it was down, after being brought back online and having it status set to SPARE or SECONDARY. 

I did manage to reproduce the bug. Do you mind pasting the execution of the following commands after step 3:

. mysqlfabric group health <group id>
. mysqlfabric group lookup_servers <group_id>
. show slave status

If there is any error reported by the previous commands, please, provide also some content from the logs (i.e. Fabric, MySQL Servers).

What is the MySQL Version you are using?

Hi George,

I meant to say that I did *not* manage to reproduce the bug.

Bug closed because it was not reproducible and there were no feedback that could help us to reproduce it. Please open the bug again if the issue re-surfaces.