Bug #116191 Replication silently stops replicating when using replicate_wild_do_table
Submitted: 20 Sep 2024 20:37 Modified: 22 Sep 2024 19:49
Reporter: Michael Villegas Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Server: Replication Severity:S3 (Non-critical)
Version:8.0.38 OS:Linux
Assigned to: MySQL Verification Team CPU Architecture:x86
Tags: replicate-wild-do-table, replication filter

[20 Sep 2024 20:37] Michael Villegas
Description:
Hi,
We found a strange case where replication silently stops working when we use replication filter replicate_wild_do_table with two or more items and the replica instance restarts.

After we set the same filter dynamically, replication continues working.

If we only set one item on replicate_wild_do_table the bug does not happen, it has to be two or more items

We replicated the same bug on 8.0.36.

How to repeat:
Two servers, 8.0.38, async replication

# Source

mysql> \s
--------------
mysql  Ver 8.0.38 for Linux on x86_64 (MySQL Community Server - GPL)

Connection id:        9
Current database:
Current user:        root@localhost
SSL:            Not in use
Current pager:        stdout
Using outfile:        ''
Using delimiter:    ;
Server version:        8.0.38 MySQL Community Server - GPL
Protocol version:    10
Connection:        Localhost via UNIX socket
Server characterset:    utf8mb4
Db     characterset:    utf8mb4
Client characterset:    utf8mb4
Conn.  characterset:    utf8mb4
UNIX socket:        /var/lib/mysql/mysql.sock
Binary data as:        Hexadecimal
Uptime:            6 min 8 sec

Threads: 3  Questions: 68  Slow queries: 4  Opens: 138  Flush tables: 3  Open tables: 57  Queries per second avg: 0.184

# Replica

mysql> \s
--------------
mysql  Ver 8.0.38 for Linux on x86_64 (MySQL Community Server - GPL)

Connection id:        14
Current database:
Current user:        root@localhost
SSL:            Not in use
Current pager:        stdout
Using outfile:        ''
Using delimiter:    ;
Server version:        8.0.38 MySQL Community Server - GPL
Protocol version:    10
Connection:        Localhost via UNIX socket
Server characterset:    utf8mb4
Db     characterset:    utf8mb4
Client characterset:    utf8mb4
Conn.  characterset:    utf8mb4
UNIX socket:        /var/lib/mysql/mysql.sock
Binary data as:        Hexadecimal
Uptime:            3 min 2 sec

Threads: 8  Questions: 16  Slow queries: 0  Opens: 152  Flush tables: 3  Open tables: 71  Queries per second avg: 0.087
--------------

mysql> show slave status \G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 10.0.2.20
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: CENTOS9-1-bin.000003
          Read_Master_Log_Pos: 1811
               Relay_Log_File: CENTOS9-2-relay-bin.000034
                Relay_Log_Pos: 508
        Relay_Master_Log_File: CENTOS9-1-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table: LabTest.%,Support.%
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1811
              Relay_Log_Space: 1251
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 0567b4dd-42ae-11ef-abab-0800273637de
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.00 sec)

No user databases on either host.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.00 sec)

my.cnf of the source

[root@CENTOS9-1 ~]# cat /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/8.0/en/server-configuration-defaults.html

[mysqld]
server-id=1
log-bin
log-slave-updates
#gtid-mode=ON
#ienforce-gtid-consistency=true
binlog_checksum=NONE
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_transaction_dependency_tracking=WRITESET_SESSION
innodb_buffer_pool_size=500M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

my.cnf of the replica

[root@CENTOS9-2 ~]# cat /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/8.0/en/server-configuration-defaults.html

[mysqld]
server-id=2
log-bin
log-slave-updates
#gtid-mode=ON
#enforce-gtid-consistency=true
binlog_checksum=NONE
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_transaction_dependency_tracking=WRITESET_SESSION
innodb_buffer_pool_size=500M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
replicate_wild_do_table = LabTest.%,Support.%

Restart the replica

[root@CENTOS9-2 ~]# systemctl restart mysqld
[root@CENTOS9-2 ~]#

Create Support database on the source

mysql> create database Support;
Query OK, 1 row affected (0.01 sec)

mysql> show master status \G
*************************** 1. row ***************************
             File: CENTOS9-1-bin.000003
         Position: 2179
     Binlog_Do_DB:
 Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.00 sec)

Confirm replica is caught up, but Support database does not exist

mysql> show slave status \G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 10.0.2.20
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: CENTOS9-1-bin.000003
          Read_Master_Log_Pos: 2179
               Relay_Log_File: CENTOS9-2-relay-bin.000036
                Relay_Log_Pos: 508
        Relay_Master_Log_File: CENTOS9-1-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table: LabTest.%,Support.%
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 2179
              Relay_Log_Space: 1423
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 0567b4dd-42ae-11ef-abab-0800273637de
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.00 sec)

Drop the database from the source

mysql> drop database Support;
Query OK, 0 rows affected (0.01 sec)

Reset replication filters to the same setting, but do it dynamically

mysql> stop slave;
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> CHANGE REPLICATION FILTER REPLICATE_WILD_DO_TABLE = ('LabTest.%', 'Support.%');
Query OK, 0 rows affected (0.00 sec)

mysql> start slave;
Query OK, 0 rows affected, 1 warning (0.01 sec)

Create Support database on source

mysql> create database Support;
Query OK, 1 row affected (0.01 sec)

mysql> show master status \G
*************************** 1. row ***************************
             File: CENTOS9-1-bin.000003
         Position: 2547
     Binlog_Do_DB:
 Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.00 sec)

Confirm replica is caught up, Support database exists

mysql> show slave status \G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 10.0.2.20
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: CENTOS9-1-bin.000003
          Read_Master_Log_Pos: 2547
               Relay_Log_File: CENTOS9-2-relay-bin.000037
                Relay_Log_Pos: 508
        Relay_Master_Log_File: CENTOS9-1-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table: LabTest.%,Support.%
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 2547
              Relay_Log_Space: 1251
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 0567b4dd-42ae-11ef-abab-0800273637de
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| Support            |
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

Restart replica again

[root@CENTOS9-2 ~]# systemctl restart mysqld
[root@CENTOS9-2 ~]#

Create table on source in the Support database

mysql> use Support;
Database changed
mysql> create table t1 (c1 int primary key); insert into t1 values (1);
Query OK, 0 rows affected (0.01 sec)

Query OK, 1 row affected (0.00 sec)

mysql> show master status \G
*************************** 1. row ***************************
             File: CENTOS9-1-bin.000003
         Position: 3005
     Binlog_Do_DB:
 Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.00 sec)

Show that replica is up to date but table does not exist.

mysql> show slave status \G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 10.0.2.20
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: CENTOS9-1-bin.000003
          Read_Master_Log_Pos: 3005
               Relay_Log_File: CENTOS9-2-relay-bin.000039
                Relay_Log_Pos: 780
        Relay_Master_Log_File: CENTOS9-1-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table: LabTest.%,Support.%
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 3005
              Relay_Log_Space: 1513
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 0567b4dd-42ae-11ef-abab-0800273637de
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.00 sec)

mysql> use Support;
Database changed
mysql> show tables;
Empty set (0.00 sec)
[20 Sep 2024 22:47] Michael Villegas
Hello,
I wanted to provide an update on the bug reported.
If we add the filters one on each line in the config file, the replication works as expected.

Instead of:
replicate_wild_do_table = LabTest.%,Support.%

Change it to:
replicate_wild_do_table = LabTest.%
replicate_wild_do_table = Support.%

Still, I believe it needs to print an error message in the error log. Currently, it stops replicating silently.
[22 Sep 2024 19:49] MySQL Verification Team
Hi,

replicate_wild_do_table = LabTest.%,Support.%

This is not a supported syntax. If you look at documentation:
https://dev.mysql.com/doc/refman/8.0/en/replication-options-replica.html#option_mysqld_rep...

```
 To specify more than one table, use this option multiple times, once for each table
```

Thanks
[22 Sep 2024 19:51] MySQL Verification Team
> Still, I believe it needs to print an error message in the error log. Currently, it stops replicating silently.

Hi,

I see you found the proper syntax. Great.

As for the "report error message", possibly, but in reality it's a "string" so what you wrote is a valid string, it just don't do what you expect.

Thanks for using MySQL Server