Bug #17844 | DD: INFORMATION_SCHEMA.FILES not showing correct free space information | ||
---|---|---|---|
Submitted: | 1 Mar 2006 22:19 | Modified: | 8 Mar 2006 14:05 |
Reporter: | Jonathan Miller | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Cluster: Cluster (NDB) storage engine | Severity: | S2 (Serious) |
Version: | 5.1.8 | OS: | Linux (Linux 32 Bit OS) |
Assigned to: | CPU Architecture: | Any |
[1 Mar 2006 22:19]
Jonathan Miller
[1 Mar 2006 23:34]
Jonathan Miller
More examples (5MB): + DROP TABLE IF EXISTS t1; + DROP DATABASE IF EXISTS mysqltest; + CREATE DATABASE mysqltest; + **** Begin Duplicate Statement Testing **** + CREATE LOGFILE GROUP lg1 + ADD UNDOFILE 'undofile.dat' + INITIAL_SIZE 16M + UNDO_BUFFER_SIZE = 1M + ENGINE=NDB; + ALTER LOGFILE GROUP lg1 + ADD UNDOFILE 'undofile02.dat' + INITIAL_SIZE 4M ENGINE NDB; + CREATE TABLESPACE ts1 + ADD DATAFILE 'datafile.dat' + USE LOGFILE GROUP lg1 + INITIAL_SIZE 8M + ENGINE NDB; + CREATE TABLE mysqltest.t1 + (pk1 MEDIUMINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + c2 CHAR(50) NOT NULL, c3 INT NOT NULL, c4 BIT NOT NULL) + TABLESPACE ts1 STORAGE DISK + ENGINE=NDB; + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', + (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', + (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) + AS '% Free Space' FROM INFORMATION_SCHEMA.FILES + WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 8388608 8388608 100.0000 + datafile.dat 8388608 8388608 100.0000 + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 8388608 6291456 75.0000 + datafile.dat 8388608 6291456 75.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 5 + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 8388608 6291456 75.0000 + datafile.dat 8388608 6291456 75.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 55 + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 8388608 6291456 75.0000 + datafile.dat 8388608 6291456 75.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 555 + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 8388608 4194304 50.0000 + datafile.dat 8388608 4194304 50.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 50555 + DROP TABLE mysqltest.t1; + ALTER TABLESPACE ts1 + DROP DATAFILE 'datafile.dat' + ENGINE=NDB; + DROP TABLESPACE ts1 + ENGINE=NDB; + DROP LOGFILE GROUP lg1 + ENGINE=NDB; + DROP DATABASE IF EXISTS mysqltest;
[1 Mar 2006 23:36]
Jonathan Miller
More examples (20MB): INITIAL_SIZE 20M + ENGINE NDB; + CREATE TABLE mysqltest.t1 + (pk1 MEDIUMINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + c2 CHAR(50) NOT NULL, c3 INT NOT NULL, c4 BIT NOT NULL) + TABLESPACE ts1 STORAGE DISK + ENGINE=NDB; + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', + (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', + (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) + AS '% Free Space' FROM INFORMATION_SCHEMA.FILES + WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 20971520 20971520 100.0000 + datafile.dat 20971520 20971520 100.0000 + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 20971520 18874368 90.0000 + datafile.dat 20971520 18874368 90.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 5
[1 Mar 2006 23:37]
Jonathan Miller
More examples (200MB): + INITIAL_SIZE 200M + ENGINE NDB; + CREATE TABLE mysqltest.t1 + (pk1 MEDIUMINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + c2 CHAR(50) NOT NULL, c3 INT NOT NULL, c4 BIT NOT NULL) + TABLESPACE ts1 STORAGE DISK + ENGINE=NDB; + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', + (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', + (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) + AS '% Free Space' FROM INFORMATION_SCHEMA.FILES + WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 209715200 209715200 100.0000 + datafile.dat 209715200 209715200 100.0000 + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 209715200 207618048 99.0000 + datafile.dat 209715200 207618048 99.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 5 + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 209715200 207618048 99.0000 + datafile.dat 209715200 207618048 99.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 55 + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile.dat 209715200 205520896 98.0000 + datafile.dat 209715200 205520896 98.0000 + SELECT COUNT(*) FROM mysqltest.t1; + COUNT(*) + 50555
[2 Mar 2006 3:44]
Jonathan Miller
I am still not sure this shows correct information even with lager files sizes. What the following shows is that we have 33% left for datafile2 and 100% for datafile with a record count of 203000. Then we add an additional 140,000 records that sould take 33% and it does. So datafile2 is now 0% free space and datafile is still 100% Then we add an additional 140,000 records which I would expect to eat up 33% or less from datafile depending on if all the space had really been used from datafile2, but what I find is that 75% of the free space from datafile is now showing as used. So I don't think we can trust this. If I am wrong, please show where I am making my mistake. + TABLESPACE_NAME FILE_NAME Total Extent Size Total Free In Bytes % Free Space + ts1 datafile2.dat 12582912 4194304 33.3333 + ts1 datafile2.dat 12582912 4194304 33.3333 + ts1 datafile.dat 8388608 8388608 100.0000 + ts1 datafile.dat 8388608 8388608 100.0000 + **** Show Record Count t2 **** + SELECT COUNT(*) FROM t2; + COUNT(*) + 203000 + **** Add more rows **** + **** Show free space **** + TABLESPACE_NAME FILE_NAME Total Extent Size Total Free In Bytes % Free Space + ts1 datafile2.dat 12582912 0 0.0000 + ts1 datafile2.dat 12582912 0 0.0000 + ts1 datafile.dat 8388608 8388608 100.0000 + ts1 datafile.dat 8388608 8388608 100.0000 + **** Show Record Count t2 **** + SELECT COUNT(*) FROM t2; + COUNT(*) + 343000 + **** Add more rows **** + **** Show free space **** + SELECT FILE_NAME, (TOTAL_EXTENTS * EXTENT_SIZE) AS 'Total Extent Size', (FREE_EXTENTS * EXTENT_SIZE) AS 'Total Free In Bytes', (((FREE_EXTENTS * EXTENT_SIZE)*100)/(TOTAL_EXTENTS * EXTENT_SIZE)) AS '% Free Space' FROM INFORMATION_SCHEMA.FILES WHERE ENGINE="ndbcluster" and FILE_TYPE = 'DATAFILE'; + FILE_NAME Total Extent Size Total Free In Bytes % Free Space + datafile2.dat 12582912 0 0.0000 + datafile2.dat 12582912 0 0.0000 + datafile.dat 8388608 2097152 25.0000 + datafile.dat 8388608 2097152 25.0000 + SELECT COUNT(*) FROM t2; + COUNT(*) + 483000
[2 Mar 2006 7:15]
Jonas Oreland
It shows free extents, not free space in extents. Example (like yours): 2 node, 2 replicas -> 2 partitions (fragments) extent size 1Mb datafile 2Mb Insert into partition 1, will allocate 1 extent in primary and 1 in backup replicas. Hence #free extents is now 1, as 1 is allocated to partition 1. There is however still free space in the extent allocated... Insert into partition 2, will allocate 1 extent in primary and 1 in backup replicas. Hence #free extens is now 0. one extend allocated to partition 1 and one to partition two. There is however still free space in both partitions. But you can not allocate a new extent to any partition. So #free extents, shows #extents that can be allocated to any partition. Remeber a tablespace can be shared between many tables (partitions).
[7 Mar 2006 19:29]
Omer Barnir
Another example how free extents do not reflect free space: Furthermore, the above does not reflect any space that is freed from extents as people remove data from tables. After a while you can easily find yourself in a situation where all the extents are 50% empty (because rows have been removed) but we will still show 100% full (since there is no unallocated extent.
[7 Mar 2006 22:21]
Omer Barnir
This bug includes alot of confusing information as a result of wrong information in WL3087 about the free space information available as part of INFORMATION_SCHEMA.FILES To make it clear that the problem reported is about the ability to get free space information and not about information available in INFORMATION_SCHEMA.FILES , the issue is being tracked in bug 18051 http://bugs.mysql.com/bug.php?id=18051 and this bug is closed (set to documentation.
[8 Mar 2006 14:05]
Jon Stephens
Thank you for your bug report. This issue has been addressed in the documentation. The updated documentation will appear on our website shortly, and will be included in the next release of the relevant product(s). Additional info: Recent additions to the Manual (see http://dev.mysql.com/doc/refman/5.1/en/files-table.html and http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html) make clear the usage statistics in I_S.FILES relate to extents and not bytes, and that usage estimates are just that - estimates only.