Description:
When MySQL complains that the disk is full, the output of SELECT mysqlbackup_page_track_get_start_lsn() goes back to 0 and Xtrabackup is not able to perform the backup with page tracking. Since you do not support Xtrabackup, please test with MEB.
How to repeat:
To reproduce, ensure that the replica has less disk space than the master. A quick way of setting this up is to make a small disk image and mount it in /var/lib/mysql:
dd if=/dev/zero of=/mysql.img bs=1024k count=2000
mkfs -t ext4 /mysql.img
mkdir /var/lib/mysql
chown mysql:mysql /var/lib/mysql
mount -t ext4 -o loop /mysql.img /var/lib/mysql
Then setup asynchronous replication between primary and replica. On replica, enable page tracking:
mysql> INSTALL COMPONENT "file://component_mysqlbackup";
mysql> SELECT mysqlbackup_page_track_set(true);
Then on primary, run sysbench. Eg:
MYSQL_HOST=192.168.1.231
MYSQL_PORT=3306
MYSQL_USER=sbtest
MYSQL_PASSWORD=sbtest
MYSQL_DB=sbtest
MYSQL_THREADS=1
MYSQL_TIME=3600
MYSQL_TABLE_SIZE=1000000
MYSQL_TABLE_COUNT=10
mysql -h $MYSQL_HOST -P $MYSQL_PORT -u $MYSQL_USER -p$MYSQL_PASSWORD -e 'DROP DATABASE IF EXISTS sbtest; CREATE DATABASE IF NOT EXISTS sbtest';
On replica, monitor page tracking:
while [ 1 ]; do mysql -e 'SELECT mysqlbackup_page_track_get_start_lsn();'; sleep 0.1;date;done
Sun Nov 24 04:15:34 AM UTC 2024
+----------------------------------------+
| mysqlbackup_page_track_get_start_lsn() |
+----------------------------------------+
| 142563815 |
+----------------------------------------+
Sun Nov 24 04:15:34 AM UTC 2024
+----------------------------------------+
| mysqlbackup_page_track_get_start_lsn() |
+----------------------------------------+
| 0 |
+----------------------------------------+
Replica MySQL error log:
2024-11-24T04:15:32.638040Z 256 [ERROR] [MY-000035] [Server] Disk is full writing './node-el9-3-relay-bin.000020' (OS errno 28 - No space left on device). Waiting for someone to free space... Retry in 60 secs. Message reprinted in 600 secs.
2024-11-24T04:15:32.638032Z 258 [ERROR] [MY-000035] [Server] Disk is full writing './mysql-bin.000008' (OS errno 28 - No space left on device). Waiting for someone to free space... Retry in 60 secs. Message reprinted in 600 secs.
Even after reclaiming disk space, page tracking output is still 0. If you run Xtrabackup with page tracking, you are not able to perform the backup with error “[Xtrabackup] failed to fetch query result SELECT mysqlbackup_page_track_set(1) : Query execution was interrupted”
Please test with MEB incremental backup with page tracking on your end.
Thank you.
