Bug #34766 No error message for Full disk while performing network or local backup
Submitted: 22 Feb 2008 20:44 Modified: 6 Nov 2008 12:06
Reporter: Hema Sridharan Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: Backup Severity:S1 (Critical)
Version:mysql-6.0-backup OS:Linux
Assigned to: Øystein Grøvlen CPU Architecture:Any

[22 Feb 2008 20:44] Hema Sridharan
Description:
1) Create a database and create tables in it. 
2) Add lot of entries (using TPCB database).
3) Backup the database to a network file.
4) Check the DISK space availability. (use df command)
5) Repeat the backup till the available size in the partition is zero (Use% is 100)
6) Now  repeat  again the backup operation after the size becomes zero to check the full disk test. File size grows for sometime and then becomes zero.
7) BACKUP command completes without any error.

BACKUP command should have errored out instead it completes and indicates a successful backup. Actual file location indicates that the BACKUP file size is zero, means nothing got BACKED UP.

How to repeat:
1) Create a database / table and add lot of entries ( Using TPCB database )

2) Backup the database to a file (network)
(repeat this step few times, rename the file and ensure that the partition is full)

mysql> backup database TPCB TO '/tmp/backup_nw1/Big_TPCB.dmp';
+-----------+
| backup_id |
+-----------+
| 52        |
+-----------+
1 row in set (4 min 43.45 sec)

mysql> backup database TPCB TO '/tmp/backup_nw1/Big_TPCB1.dmp';
+-----------+
| backup_id |
+-----------+
| 53        |
+-----------+
1 row in set (4 min 14.52 sec)

mysql> backup database TPCB TO '/tmp/backup_nw1/Big_TPCB2.dmp';
+-----------+
| backup_id |
+-----------+
| 54        |
+-----------+
1 row in set (4 min 27.76 sec)

mysql> backup database TPCB TO '/tmp/backup_nw1/Big_TPCB3.dmp';
+-----------+
| backup_id |
+-----------+
| 55        |
+-----------+
1 row in set (4 min 15.07 sec)

3) Verify the file size and disk usage after each backup.

ndbdev@ndb16:/data2/hema/backup_dmp> df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p2     9.2G  7.6G  1.2G  88% /
/dev/cciss/c0d0p1     190M   24M  156M  14% /boot
none                  3.8G     0  3.8G   0% /dev/shm
/dev/cciss/c0d1p1      67G   25G   40G  39% /space
/dev/cciss/c0d2p1      67G   84M   64G   1% /data0
/dev/cciss/c0d3p1      67G   84M   64G   1% /data1
/dev/cciss/c0d4p1      67G   28G   36G  44% /data2
df: `/tmp1/test1': Permission denied
ndb17:/tmp/backup_test
                      9.2G  7.6G  1.2G  87% /tmp/backup_nw1

ndbdev@ndb16:/data2/hema/backup_dmp> df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p2     9.2G  7.6G  1.2G  88% /
/dev/cciss/c0d0p1     190M   24M  156M  14% /boot
none                  3.8G     0  3.8G   0% /dev/shm
/dev/cciss/c0d1p1      67G   25G   40G  39% /space
/dev/cciss/c0d2p1      67G   84M   64G   1% /data0
/dev/cciss/c0d3p1      67G   84M   64G   1% /data1
/dev/cciss/c0d4p1      67G   28G   36G  44% /data2
df: `/tmp1/test1': Permission denied
ndb17:/tmp/backup_test
                      9.2G  8.1G  661M  93% /tmp/backup_nw1

ndbdev@ndb16:/data2/hema/backup_dmp> cd /tmp/backup_nw1/

ndbdev@ndb16:/tmp/backup_nw1> ls -lrt
total 4712092
-rw-rw----  1 ndbdev ndbdev       3896 Feb 19 03:40 test_nw
-rw-rw----  1 ndbdev ndbdev 1776053989 Feb 22 06:20 Big_TPCB.dmp
-rw-rw----  1 ndbdev ndbdev 1776054173 Feb 22 06:26 Big_TPCB1.dmp
-rw-rw----  1 ndbdev ndbdev 1268342784 Feb 22 06:37 Big_TPCB2.dmp

ndbdev@ndb16:/tmp/backup_nw1> df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p2     9.2G  7.6G  1.2G  88% /
/dev/cciss/c0d0p1     190M   24M  156M  14% /boot
none                  3.8G     0  3.8G   0% /dev/shm
/dev/cciss/c0d1p1      67G   25G   40G  39% /space
/dev/cciss/c0d2p1      67G   84M   64G   1% /data0
/dev/cciss/c0d3p1      67G   84M   64G   1% /data1
/dev/cciss/c0d4p1      67G   28G   36G  44% /data2
df: `/tmp1/test1': Permission denied
ndb17:/tmp/backup_test
                      9.2G  8.8G     0 100% /tmp/backup_nw1

ndbdev@ndb16:/tmp/backup_nw1> ls -lrt
total 4712092
-rw-rw----  1 ndbdev ndbdev       3896 Feb 19 03:40 test_nw
-rw-rw----  1 ndbdev ndbdev 1776053989 Feb 22 06:20 Big_TPCB.dmp
-rw-rw----  1 ndbdev ndbdev 1776054173 Feb 22 06:26 Big_TPCB1.dmp
-rw-rw----  1 ndbdev ndbdev 1268342784 Feb 22 06:37 Big_TPCB2.dmp

ndbdev@ndb16:/tmp/backup_nw1> df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p2     9.2G  7.6G  1.2G  88% /
/dev/cciss/c0d0p1     190M   24M  156M  14% /boot
none                  3.8G     0  3.8G   0% /dev/shm
/dev/cciss/c0d1p1      67G   25G   40G  39% /space
/dev/cciss/c0d2p1      67G   84M   64G   1% /data0
/dev/cciss/c0d3p1      67G   84M   64G   1% /data1
/dev/cciss/c0d4p1      67G   28G   36G  44% /data2
df: `/tmp1/test1': Permission denied
ndb17:/tmp/backup_test
                      9.2G  8.8G     0 100% /tmp/backup_nw1

ndbdev@ndb16:/tmp/backup_nw1> ls -lrt
total 4712092
-rw-rw----  1 ndbdev ndbdev       3896 Feb 19 03:40 test_nw
-rw-rw----  1 ndbdev ndbdev 1776053989 Feb 22 06:20 Big_TPCB.dmp
-rw-rw----  1 ndbdev ndbdev 1776054173 Feb 22 06:26 Big_TPCB1.dmp
-rw-rw----  1 ndbdev ndbdev 1268342784 Feb 22 06:37 Big_TPCB2.dmp
-rw-rw----  1 ndbdev ndbdev 1238794240 Feb 22 06:49 Big_TPCB3.dmp

ndbdev@ndb16:/tmp/backup_nw1> df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p2     9.2G  7.6G  1.2G  88% /
/dev/cciss/c0d0p1     190M   24M  156M  14% /boot
none                  3.8G     0  3.8G   0% /dev/shm
/dev/cciss/c0d1p1      67G   25G   40G  39% /space
/dev/cciss/c0d2p1      67G   84M   64G   1% /data0
/dev/cciss/c0d3p1      67G   84M   64G   1% /data1
/dev/cciss/c0d4p1      67G   28G   36G  44% /data2
df: `/tmp1/test1': Permission denied

ndb17:/tmp/backup_test
                      9.2G  8.8G     0 100% /tmp/backup_nw1

ndbdev@ndb16:/tmp/backup_nw1> ls -lrt
total 4712092
-rw-rw----  1 ndbdev ndbdev       3896 Feb 19 03:40 test_nw
-rw-rw----  1 ndbdev ndbdev 1776053989 Feb 22 06:20 Big_TPCB.dmp
-rw-rw----  1 ndbdev ndbdev 1776054173 Feb 22 06:26 Big_TPCB1.dmp
-rw-rw----  1 ndbdev ndbdev 1268342784 Feb 22 06:37 Big_TPCB2.dmp
-rw-rw----  1 ndbdev ndbdev 1557069824 Feb 22 06:49 Big_TPCB3.dmp

ndbdev@ndb16:/tmp/backup_nw1> ls -lrt
total 4712092
-rw-rw----  1 ndbdev ndbdev       3896 Feb 19 03:40 test_nw
-rw-rw----  1 ndbdev ndbdev 1776053989 Feb 22 06:20 Big_TPCB.dmp
-rw-rw----  1 ndbdev ndbdev 1776054173 Feb 22 06:26 Big_TPCB1.dmp
-rw-rw----  1 ndbdev ndbdev 1268342784 Feb 22 06:37 Big_TPCB2.dmp
-rw-rw----  1 ndbdev ndbdev          0 Feb 22 06:50 Big_TPCB3.dmp
ndbdev@ndb16:/tmp/backup_nw1>

If you notice, Big_TPCB2.dmp didn't had complete data BACKUP. For subsequent BACKUP Big_TPCB3, size grew and finally became 0.

BACKUP command showed successful completion on both occasions though, actual BACKUP didn't happen.

Suggested fix:
Back command should error out for full disk destination.
[25 Feb 2008 15:26] Jonathan Miller
This affects both network and local backups
[6 May 2008 21:01] MySQL Verification Team
Bug: http://bugs.mysql.com/bug.php?id=36529 has been marked as duplicate
of this one.
[7 May 2008 7:28] Rafal Somla
Fixing backup system so that it correctly reports all errors is the subject of WL#4384.
[6 Nov 2008 12:06] Øystein Grøvlen
I cannot repeat this behavior.  
I get an error message when disk goes full, though not very informative.
The issue of misleading error message is covered by BUG#37249.