Bug #75772 5.7.6->5.6 replication fails
Submitted: 4 Feb 2015 19:07 Modified: 1 Apr 2015 4:19
Reporter: Jorge Jordao Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Replication Severity:S1 (Critical)
Version:5.7.6 OS:Any
Assigned to: CPU Architecture:Any

[4 Feb 2015 19:07] Jorge Jordao
Description:
Basic replication from 5.7.6 master to 5.6.23 slave fails after first statement.

Pre-validation by replication developer Joao Gramacho:

The "Executed_Gtid_Set: 00000000-0000-0000-0000-000000000000:0" at the end of the "How to Repeat" section is not a good sign.
It seems that the slave is assuming the ANONYMOUS GTID as '00000000-0000-0000-0000-000000000000:0'. So it applied the 'CREATE TABLE' and logged this GTID for it. The next statement (the INSERT) is using the same GTID, so it produces no changes, and no errors, just like a transaction that was already executed.

How to repeat:
Started 5.7.6 server (master) with --port=10001 --log-bin=master-bin --server-id=1
Started 5.6.23 server (slave) with --port=10002 --log-bin=slave-bin --server-id=2 --log-slave-updates

[slave] change master to master_host='localhost',master_port=10001, master_user='root'

[slave] start slave

[slave] show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: localhost
                  Master_User: root
                  Master_Port: 10001
                Connect_Retry: 60
              Master_Log_File: master-bin.000001
          Read_Master_Log_Pos: 153
               Relay_Log_File: pid-relay-bin.000002
                Relay_Log_Pos: 334
        Relay_Master_Log_File: master-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 153
              Relay_Log_Space: 505
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 243f4ba5-ac96-11e4-ad99-a44e31b96590
             Master_Info_File: /home/jorge/Temp/57-56/slave/data/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0

[master] create table t1 (c INT)

[slave] show tables
+----------------+
| Tables_in_test |
+----------------+
| t1             |
+----------------+

[master] insert into t1 values(1)

[master] select * from t1
+------+
| c    |
+------+
|    1 |
+------+

[slave] select * from t1

[slave] select count(*) from t1
+----------+
| count(*) |
+----------+
|        0 |
+----------+

[slave] show binlog events
+------------------+-----+-------------+-----------+-------------+---------------------------------------+
| Log_name         | Pos | Event_type  | Server_id | End_log_pos | Info                                  |
+------------------+-----+-------------+-----------+-------------+---------------------------------------+
| slave-bin.000001 |   4 | Format_desc |         2 |         120 | Server ver: 5.6.23-log, Binlog ver: 4 |
| slave-bin.000001 | 120 | Query       |         1 |         217 | use `test`; create table t1 (c INT)   |
+------------------+-----+-------------+-----------+-------------+---------------------------------------+

[slave] show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: localhost
                  Master_User: root
                  Master_Port: 10001
                Connect_Retry: 60
              Master_Log_File: master-bin.000001
          Read_Master_Log_Pos: 588
               Relay_Log_File: pid-relay-bin.000002
                Relay_Log_Pos: 769
        Relay_Master_Log_File: master-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 588
              Relay_Log_Space: 940
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 243f4ba5-ac96-11e4-ad99-a44e31b96590
             Master_Info_File: /home/jorge/Temp/57-56/slave/data/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set: 00000000-0000-0000-0000-000000000000:0
                Auto_Position: 0
[1 Apr 2015 4:19] Erlend Dahl
[16 Mar 2015 9:52] David Moss

This is fixed in the 5.7.7/5.8.0 version:

When replicating from a 5.7.6 or later server to a 5.6.23 or earlier server,
if the older version applier thread encountered an Anonymous_gtid_log_event
it caused an assert. The fix ensures that these new log events added in MySQL
5.7.6 and later do not cause this problem with 5.6.24 and later slaves. If
gtid_mode is OFF and the applier thread encounters a Gtid_log_event, the
applier thread aborts with an error. If gtid_mode is ON and the applier
thread encounters a Anonymous_gtid_log_event, the applier thread aborts with
an error.