Bug #49709 free_extents shown wrong if extent_size is not a power of 2?
Submitted: 15 Dec 2009 9:53 Modified: 10 Feb 2010 13:43
Reporter: Hartmut Holzgraefe Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Disk Data Severity:S3 (Non-critical)
Version:mysql-cluster-6.3.25 OS:Linux
Assigned to: Pekka Nousiainen CPU Architecture:Any

[15 Dec 2009 9:53] Hartmut Holzgraefe
Description:
When creating a tablespace with an extent size that is not a power of two the number of free_extents in I_S.FILES is lower than the number of total_extents.

How to repeat:
test code:

CREATE LOGFILE GROUP lg_1     ADD UNDOFILE 'undo_1.log'     INITIAL_SIZE 16M     UNDO_BUFFER_SIZE 2M     ENGINE NDBCLUSTER;

CREATE TABLESPACE ts_1 ADD DATAFILE 'foo' use logfile group lg_1 INITIAL_SIZE 10M EXTENT_SIZE 32K engine=ndb;
SHOW WARNINGS;

SELECT FILE_NAME, EXTRA, INITIAL_SIZE, EXTENT_SIZE, TOTAL_EXTENTS, FREE_EXTENTS  FROM INFORMATION_SCHEMA.FILES WHERE TABLESPACE_NAME IS NOT NULL\G

ALTER TABLESPACE ts_1 DROP DATAFILE 'foo' ENGINE=ndb;
DROP TABLESPACE ts_1 ENGINE=ndb;

CREATE TABLESPACE ts_1 ADD DATAFILE 'foo' use logfile group lg_1 INITIAL_SIZE 10000000 EXTENT_SIZE 32000 engine=ndb;
SHOW WARNINGS;

SELECT FILE_NAME, EXTRA, INITIAL_SIZE, EXTENT_SIZE, TOTAL_EXTENTS, FREE_EXTENTS  FROM INFORMATION_SCHEMA.FILES WHERE TABLESPACE_NAME IS NOT NULL\G

ALTER TABLESPACE ts_1 DROP DATAFILE 'foo' ENGINE=ndb;
DROP TABLESPACE ts_1 ENGINE=ndb;

CREATE TABLESPACE ts_1 ADD DATAFILE 'foo' use logfile group lg_1 INITIAL_SIZE 10000000 EXTENT_SIZE 10000 engine=ndb;
SHOW WARNINGS;

SELECT FILE_NAME, EXTRA, INITIAL_SIZE, EXTENT_SIZE, TOTAL_EXTENTS, FREE_EXTENTS  FROM INFORMATION_SCHEMA.FILES WHERE TABLESPACE_NAME IS NOT NULL\G

ALTER TABLESPACE ts_1 DROP DATAFILE 'foo' ENGINE=ndb;
DROP TABLESPACE ts_1 ENGINE=ndb;

DROP LOGFILE GROUP lg_1 ENGINE=ndb;

expected result: free_extents==local_extents in all 3 cases

actual result:

*************************** 1. row ***************************
    FILE_NAME: foo
        EXTRA: CLUSTER_NODE=2
 INITIAL_SIZE: 10485760
  EXTENT_SIZE: 32768
TOTAL_EXTENTS: 320
 FREE_EXTENTS: 320
*************************** 2. row ***************************
    FILE_NAME: foo
        EXTRA: CLUSTER_NODE=3
 INITIAL_SIZE: 10485760
  EXTENT_SIZE: 32768
TOTAL_EXTENTS: 320
 FREE_EXTENTS: 320

*************************** 1. row ***************************
    FILE_NAME: foo
        EXTRA: CLUSTER_NODE=2
 INITIAL_SIZE: 10000000
  EXTENT_SIZE: 32000
TOTAL_EXTENTS: 312
 FREE_EXTENTS: 305
*************************** 2. row ***************************
    FILE_NAME: foo
        EXTRA: CLUSTER_NODE=3
 INITIAL_SIZE: 10000000
  EXTENT_SIZE: 32000
TOTAL_EXTENTS: 312
 FREE_EXTENTS: 305

*************************** 1. row ***************************
    FILE_NAME: foo
        EXTRA: CLUSTER_NODE=2
 INITIAL_SIZE: 10000000
  EXTENT_SIZE: 10000
TOTAL_EXTENTS: 1000
 FREE_EXTENTS: 305
*************************** 2. row ***************************
    FILE_NAME: foo
        EXTRA: CLUSTER_NODE=3
 INITIAL_SIZE: 10000000
  EXTENT_SIZE: 10000
TOTAL_EXTENTS: 1000
 FREE_EXTENTS: 305

Suggested fix:
not sure if this is just an error in the code generating the I_S table 
( as it was in bug #28642)
or if there are indeed less free extends ...
[7 Feb 2010 16:54] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99562

3075 Pekka Nousiainen	2010-02-07
      bug#49709 03_bug49709.diff
      Make DBDICT round DD sizes in same way as LGMAN/TSMAN.
      This avoids discrepancy in information_schema.files.
      Return warnings if rounded.
[8 Feb 2010 15:38] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99608

3076 Pekka Nousiainen	2010-02-08
      bug#49709 ifdef some ndbouts
[9 Feb 2010 10:35] Bugs System
Pushed into 5.1.41-ndb-7.0.13 (revid:pekka@mysql.com-20100209052415-xie31hvmbn1yhzr1) (version source revid:pekka@mysql.com-20100209052415-xie31hvmbn1yhzr1) (merge vers: 5.1.41-ndb-7.0.13) (pib:16)
[9 Feb 2010 10:38] Bugs System
Pushed into 5.1.41-ndb-6.3.32 (revid:pekka@mysql.com-20100208163827-jjvujexbm0gyquno) (version source revid:pekka@mysql.com-20100208163827-jjvujexbm0gyquno) (merge vers: 5.1.41-ndb-6.3.32) (pib:16)
[9 Feb 2010 10:41] Bugs System
Pushed into 5.1.41-ndb-6.2.19 (revid:pekka@mysql.com-20100208153743-1rr33j9x70a70qaa) (version source revid:pekka@mysql.com-20100208153743-1rr33j9x70a70qaa) (merge vers: 5.1.41-ndb-6.2.19) (pib:16)
[10 Feb 2010 11:30] Pekka Nousiainen
Pushed to:

mysql-5.1.41 ndb-6.2.19
mysql-5.1.41 ndb-6.3.32
mysql-5.1.41 ndb-7.0.13
mysql-5.1.41 ndb-7.1.2

Set to documenting:

The sizes given for following disk objects are now rounded
explicitly in stored kernel metadata.  An implicit rounding
with same effects has always taken place.

1) undo_buffer_size: rounded up to multiple of page size (32k)
2) undofile: rounded down to multiple of page size (32k)
3) extent_size: rounded up to multiple of page size (32k)
4) datafile: rounded down to multiple of page size (32k)
             and then up to multiple of extent size

For each object, if any rounding took place, a warning is
returned to mysql.

NOTE 1: Actual file size in 4) adds a header (of a few pages).

NOTE 2: It is recommended to give sizes in correct multiples
instead of using this complicated rounding behaviour.
[10 Feb 2010 13:43] Jon Stephens
Documented bugfix in the NDB-6.2.19, 6.3.32, 7.0.13, and 7.1.2 changelogs, as follows:

        For a Disk Data tablespace having an extent size that was not an
        integer power of two, the value of the FREE_EXTENTS column in
        the INFORMATION_SCHEMA.FILES table was smaller than the value of
        TOTAL_EXTENTS.

        As part of this fix, the implicit rounding of INITIAL_SIZE, 
        EXTENT_SIZE, and UNDO_BUFFER_SIZE performed by NDBCLUSTER 
        (see "CREATE TABLESPACE Syntax") is now done explicitly, and the 
        rounded values are used for calculating INFORMATION_SCHEMA.FILES 
        column values and other purposes.

Also updated indicated section of Manual with info about rounding algorithm.

Closed.