Bug #47170 out of stringmemory (773) with stringmemory=100

• Submitted: 7 Sep 2009 11:16
• Modified: 9 Sep 2009 16:25
• Reporter: Johan Andersson
• Status: Closed
• Category: MySQL Cluster: Cluster (NDB) storage engine
• Severity: S3 (Non-critical)
• Version: mysql-5.1-telco-6.2
• OS: Any
• Assigned to: Jonas Oreland
• CPU Architecture: Any
• Tags: 7.0.7, StringMemory

[7 Sep 2009 11:16] Johan Andersson

Description:
Created 2027 tables each having 128 columns
After 2027 I got error 773 (NDB error code 773: Out of string memory, please modify StringMemory config parameter: Permanent error: Schema error)

see how to repeat

How to repeat:
Run the script create_tabs.sh  (config.ini (ndbd default) below):

./create_tabs.sh 20320 128

cat create_tabs.sh

#!/bin/bash

MYSQL_BIN=/usr/local/mysql/mysql/bin
MYSQL_LIBS=/usr/local/mysql/mysql/lib:/usr/local/mysql/mysql/lib/mysql

attrs=$2
tables=$1

i=0
while [ $i -lt $tables ]; 
do
cols=""
header="CREATE TABLE t$i ("
j=0
while [ $j -lt $attrs ]; 
do
   cols="$cols a$j INTEGER NOT NULL,"
   j=`expr $j + 1`
done
footer="PRIMARY KEY USING HASH(a0))ENGINE=NDB;"

sql="$header $cols $footer"
LD_LIBRARY_PATH=$MYSQL_LIBS $MYSQL_BIN/mysql -uroot test -S/tmp/mysql.sock --verbose --execute="$sql"
i=`expr $i + 1`

done

config.ini settings below:

[NDBD DEFAULT]
NoOfReplicas=2
Datadir=/data1/mysqlcluster/
FileSystemPathDD=/data1/mysqlcluster/
#FileSystemPathUndoFiles=/data1/mysqlcluster/
#FileSystemPathDataFiles=/data1/mysqlcluster/
DataMemory=2048M
IndexMemory=256M
LockPagesInMainMemory=0

MaxNoOfConcurrentOperations=100000

StringMemory=100
MaxNoOfTables=20320
MaxNoOfOrderedIndexes=2048
MaxNoOfUniqueHashIndexes=512
MaxNoOfAttributes=2861056
DiskCheckpointSpeedInRestart=100M
FragmentLogFileSize=256M
InitFragmentLogFiles=FULL
NoOfFragmentLogFiles=12
RedoBuffer=32M

TimeBetweenLocalCheckpoints=20
TimeBetweenGlobalCheckpoints=1000
TimeBetweenEpochs=100

MemReportFrequency=30
BackupReportFrequency=10

### Params for setting logging 
LogLevelStartup=15
LogLevelShutdown=15
LogLevelCheckpoint=8
LogLevelNodeRestart=15

### Params for increasing Disk throughput 
BackupMaxWriteSize=1M
BackupDataBufferSize=16M
BackupLogBufferSize=4M
BackupMemory=20M
#Reports indicates that odirect=1 can cause io errors (os err code 5) on some systems. You must test.
#ODirect=1

### Watchdog 
TimeBetweenWatchdogCheckInitial=60000

### TransactionInactiveTimeout  - should be enabled in Production 
#TransactionInactiveTimeout=30000
### CGE 6.3 - REALTIME EXTENSIONS 
#RealTimeScheduler=1
#SchedulerExecutionTimer=80
#SchedulerSpinTimer=40

### DISK DATA 
#SharedGlobalMemory=384M
#read my blog how to set this:
#DiskPageBufferMemory=3072M

### Multithreading 
MaxNoOfExecutionThreads=8
BatchSizePerLocalScan=512

[8 Sep 2009 8:45] Sveta Smirnova

Thank you for the report.

Verified as described with settings provided.

[9 Sep 2009 8:53] Jonas Oreland

Problem was that for really big values of MaxNoOfTables + MaxNoOfAttributes,
the calculation could overflow.

Changed to Uint64 for intermediate calculations

[9 Sep 2009 8:55] Jonas Oreland

Furthermore triage should be
D2 : You can't create all the tables that you need

[9 Sep 2009 8:58] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/82777

2990 Jonas Oreland	2009-09-09
      ndb - bug#47170
        Calculation of StringMemory could overflow for big values 
          of MaxNoOfTables/Attributes

[9 Sep 2009 11:04] Bugs System

Pushed into 5.1.35-ndb-7.1.0 (revid:[e-mail]-20090909110427-4vsmv05b2i6svff9) (version source revid:[e-mail]-20090909110427-4vsmv05b2i6svff9) (merge vers: 5.1.35-ndb-7.1.0) (pib:11)

[9 Sep 2009 11:05] Bugs System

Pushed into 5.1.37-ndb-6.3.27 (revid:[e-mail]-20090909090834-znkdnl02929etrky) (version source revid:[e-mail]-20090909090834-znkdnl02929etrky) (merge vers: 5.1.37-ndb-6.3.27) (pib:11)

[9 Sep 2009 11:06] Bugs System

Pushed into 5.1.37-ndb-7.0.8 (revid:[e-mail]-20090909110150-kdvot0410na89hk5) (version source revid:[e-mail]-20090909091751-4somugxyxyue3gzo) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)

[9 Sep 2009 12:00] Jonas Oreland

also pushed to 6.2.19

[9 Sep 2009 16:25] Jon Stephens

Documented bugfix in the NDB-6.2.19, 6.3.27, and 7.0.8 changelogs, as follows:

      For very large values of MaxNoOfTables + MaxNoOfAttributes,
      the calculation for StringMemory could overflow when creating large
      numbers of tables, leading to NDB error 773 (Out of string memory, please
      modify StringMemory config parameter), even when StringMemory was set to
      100 (the maximum).

Closed.