Bug #68386 Master scans all binlogs when slave reconnects with auto positioning

• Submitted: 15 Feb 2013 1:03
• Modified: 3 Apr 2013 14:50
• Reporter: Santosh Praneeth Banda
• Status: Closed
• Category: MySQL Server: Replication
• Severity: S2 (Serious)
• Version: mysql 5.6.10
• OS: Any
• CPU Architecture: Any
• Tags: auto_positioning, GTID

[15 Feb 2013 1:03] Santosh Praneeth Banda

Description:
With both master and slave gtid enabled and slaves reconnects(in-case of I/O thread failure or in-case of change master) sending its Executed_Gtid_set and Retreived_Gtid_set. Master needs to scan all the binlogs before actually starting sending the Events to slave. There should be better way of handling this.

How to repeat:
Do a stop slave and start slave on Gtid enabled slave and log the events binlogs scanned by master.

Suggested fix:
There are PreviousGtidEvents stored in every binlog file. Lets say A is set of Gtids in that event. When slave sends the whole set of gtids it got (say set B), master can do binary search on the criteria that ' A should be a superset of B'.

[15 Feb 2013 1:08] Yoshinori Matsunobu

This is serious since usually tens or hundreds of GBs of binary logs exist on master, and most of older binary logs are not cached in filesystem cache. And i/o thread disconnect/reconnect is not uncommon. Reading 100+GB binlogs every i/o thread reconnects is painful.

[20 Feb 2013 13:49] Mark Callaghan

This should be more than a feature request. This probably prevents anyone from using gtid if they have one or both of:
1) many slaves per master
2) 100+ gb of binlogs per master

[20 Feb 2013 18:59] Santosh Praneeth Banda

Change the severity level.

[21 Feb 2013 9:15] Luis Soares

Thanks for bringing up this issue. We are aware of the problem and we are working on a fix.

[1 Mar 2013 14:34] MySQL Verification Team

A fix was pushed in the 5.6 tree, so we close this bug.

[3 Apr 2013 14:50] Jon Stephens

Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

[3 Apr 2013 14:51] Jon Stephens

Fixed in 5.6+. Documented as follows in the 5.6.11 and 5.7.1 changelogs:

        When using GTIDs and binary log auto-positioning, the master had
        to scan all binary logs whenever the slave reconnected (due to
        reasons such as I/O thread failure or a change of master) before
        it could send any events to slave. Now, the master starts from
        the oldest binary log that contains any GTID not found on the
        slave.
      

Closed.