Bug #30046 Datamemory Usage is not accurate
Submitted: 25 Jul 2007 17:16 Modified: 22 Aug 2007 13:22
Reporter: ws lee Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:5.0.45 OS:Solaris
Assigned to: Hartmut Holzgraefe CPU Architecture:Any

[25 Jul 2007 17:16] ws lee
Description:
this is config.ini of my actual system.
-----------------
[NDBD DEFAULT]
NoOfReplicas=2    
DataMemory=500M   
IndexMemory=500M 
[NDB_MGMD]
hostname=192.160.30.5           
datadir=/var/lib/mysql-cluster  
ID=1

[NDB_MGMD]
hostname=192.160.30.6            
datadir=/var/lib/mysql-cluster 
ID=2

[NDBD]
hostname=192.160.30.15                   
datadir=/usr/local/mysql/data   

[NDBD]
hostname=192.160.30.16          
datadir=/usr/local/mysql/data  

[MYSQLD]
hostname=192.160.30.5           
hostname=192.160.30.6     
-----------------

and table desc is below.
--------------------
CREATE TABLE `bbs` (
  `bbsid` int(11) NOT NULL,
  `dataid` int(11) NOT NULL,
  `name` mediumtext,
  `mail` mediumtext,
  `url` mediumtext,
  `subject` mediumtext,
  `data` mediumtext,
  `file` mediumtext,
  `size_w` smallint(6) default NULL,
  `size_h` smallint(6) default NULL,
  `size_kt_w` smallint(6) default NULL,
  `size_kt_h` smallint(6) default NULL,
  `color` mediumtext,
  `host` mediumtext,
  `ua` mediumtext,
  `key` varchar(100) default NULL,
  `timecreated` datetime default NULL,
  PRIMARY KEY  (`dataid`),
  UNIQUE KEY `unique_bbs_data_key` USING BTREE (`key`),
  KEY `Index_bbsid` (`bbsid`),
  KEY `Index_timecreated` (`timecreated`)
) ENGINE=ndbcluster DEFAULT CHARSET=utf8;
--------------------

How to repeat:

I inserted bbs table to data.(Inserted text data is japanese character.)
but stopped 259.2M 
Error is below
----------------
ERROR 1114 (HY000): The table 'bbs' is full.
----------------

but, really bbs table size is 271745024 byte.
mysql> show table status;
+-----------------+------------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+-----------------------+
| Name            | Engine     | Version | Row_format | Rows   | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time         | Check_time | Collation       | Checksum | Create_options | Comment               |
+-----------------+------------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+-----------------------+
| bbs             | ndbcluster |      10 | Dynamic    |  90568 |           2728 |   271745024 |               0 |            0 |         0 |           NULL | NULL                | NULL                | NULL       | utf8_general_ci |     NULL |                | number_of_replicas: 2 | 
+-----------------+------------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+-----------------------+

and so, i confirmed top command
-----------------------------
ast pid:  9299;  load averages:  0.01,  0.01,  0.01                                                                       02:01:38
36 processes:  35 sleeping, 1 on cpu
CPU states: 99.6% idle,  0.2% user,  0.2% kernel,  0.0% iowait,  0.0% swap
Memory: 2047M real, 934M free, 1330M swap in use, 4241M swap free

   PID USERNAME LWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
  5889 mysql     31  59    0 1284M  769M sleep   12:21  0.16% ndbd
-----------------------------

memory is enough.

why not use 500M of datamemory?
this is mysql cluster bug?

p.s)
i doubt (medium)text type is not handled japanese character with utf-8 enviroment.
[22 Aug 2007 13:22] Hartmut Holzgraefe
Thank you for taking the time to write to us, but this is not a bug. Please double-check the documentation available at http://dev.mysql.com/doc/ and the instructions on
how to report a bug at http://bugs.mysql.com/how-to-report.php

See http://dev.mysql.com/doc/refman/5.0/en/show-table-status.html

  "Beginning with MySQL 5.0.3, for NDB Cluster  tables, the output of this statement shows appropriate values for the Avg_row_length and Data_length columns, with the exception that BLOB columns are not taken into account."

Cluster stores BLOB and TEXT fields in seperate hidden helper tables,
only the first 256(?) bytes of a BLOB or TEXT are stored in the actual
table row itself, and SHOW TABLE STATUS only shows the Data_length
for the actual table, not the helper table(s) storing the BLOB and
TEXT extends.