Bug #7593 Cannot Create A Large NDB Data Warehouse
Submitted: 30 Dec 2004 18:43 Modified: 31 Dec 2004 9:50
Reporter: Dan Gruhn Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S1 (Critical)
Version:4.1.8 (Max) OS:Linux (Linux Fedora Core 1)
Assigned to: Mikael Ronström CPU Architecture:Any

[30 Dec 2004 18:43] Dan Gruhn
Description:
Mikael Ronstrom says this looks like a bug and asked me to file this report.

I'm having a problem getting 4.1.8 to create a data warehouse that I've used just fine for a long time in a non-cluster environment. I've downloaded the tar/gz file mysql-max-4.1.8-pc-linux-i686.tar.gz and installed it.

Here are the commands that I run and their output:

> mysqladmin -f drop dgruhn_dw
Database "dgruhn_dw" dropped
> mysqladmin create dgruhn_dw
> mysql dgruhn_dw <dw_schema

ERROR 1005 at line 352: Can't create table './dgruhn_dw/#sql-117c_2d.frm' (errno: 4009)

The schema file I'm using is failing at line 352, which is the following:

ALTER TABLE AirAssetConfiguration ADD UNIQUE (ConfigurationName, SignatureName, FuelConfigurationName);

We've found it is faster to create tables and then add indexes later, so this point in the schema file is just adding new indexes to tables. We have about 188 tables and the rest of the almost 5000 lines of the schema file are adding indexes and inserting values to the tables (we fail before getting to the insertions).

When I look at the NDB error log, I see the following:

Date/Time: Thursday 30 December 2004 - 12:01:34
Type of error: error
Message: Internal program error (failed ndbrequire)
Fault ID: 2341
Problem data: Suma.cpp
Object of reference: SUMA (Line: 1355) 0x0000000c
ProgramName: /usr/local/mysql/bin/ndbd
ProcessID: 11773
TraceFile: /var/lib/mysql-cluster/ndb_2_trace.log.1
***EOM***

The line in Suma.cpp being referenced is the following:
...
// The attribute buffer
LocalDataBuffer<15> attrBuf(c_dataBufferPool, tabPtr.p->m_attributes);

// Temporary buffer
DataBuffer<15> theRest(c_dataBufferPool);

if(!attrBuf.seize(noAttribs)){
ndbrequire(false); <-----------------------------------------------
return false;
}
...

It seems that I'm having a problem with not enough attributes being allocated. However, I've played with a number of parameters and can't seem to get beyond this point. Here is my config.ini file:

[NDBD DEFAULT]
NoOfReplicas = 1

[MYSQLD DEFAULT]
[NDB_MGMD DEFAULT]
[TCP DEFAULT]

[NDB_MGMD]
HostName = analyst04-lx.group-w-inc.com

[NDBD]
HostName = analyst04-lx.group-w-inc.com
DataDir = /var/lib/mysql-cluster
MaxNoOfTables = 1600
DataMemory = 512MB
IndexMemory = 512MB
MaxNoOfAttributes = 32767
MaxNoOfOrderedIndexes = 2048
MaxNoOfUniqueHashIndexes = 2048
MaxNoOfTriggers = 4096
Diskless = Y
StopOnError = N

[MYSQLD]
...

I've got about 2 GB available on the analyst04 machine, which it seems like should be more than enough to at least create the warehouse. That machine is running the manager and the NDB, but no server. My local machine is running a server.

How to repeat:
Set up an empty server with the configurations shown in the description and run the given commands.  The dw_schema file is almost 5K lines (394K bytes,  37KB gzipped) and I would be happy to email it to someone if I could get an email address.

Suggested fix:
If this is a bug, please fix it.  If not:

1) How can I check the values of various parameters to see if I'm setting them correctly?

2) Are there some better values I could use for the parameters I'm setting?

3) Are there some other parameters I should be modifying as well?

4) Does anyone have any other ideas what I could try or where I could look?
[30 Dec 2004 19:52] Dan Gruhn
Data warehouse schema mysql commands file

Attachment: dw_schema.gz (application/x-gzip, text), 36.27 KiB.

[30 Dec 2004 19:53] Dan Gruhn
I've added the dw_schema file to this bug report.
[31 Dec 2004 1:06] Mikael Ronström
Bug fixed, bug causes a limit of 64 unique indexes in the cluster, bug fix removes this
limit (caused by memory leak)
[31 Dec 2004 9:50] Mikael Ronström
Fix pushed to 4.1 and 5.0
[3 Jan 2005 13:27] Dan Gruhn
Mikael  & Dean,

Thanks for your quick resolution for this.  How can I get the fix that you folks have come up with?

Dan