Description:
when trying to run a batch insert to fill a table with data in MySQL cluster database 7.2.10 ,suddenly all data nodes in the cluster are forcing shutdown, and I get the error below(error repeated several times):
------------------------------
Forced node shutdown completed. Caused by error 1217 ,No message slogan found (please report a bug if you get this error code) (Unknown)
-----------------------------------
I have been monitoring the dataMemory and indexMemory and it were fine, last reading were:
-------------------------
Node 3: Data usage is 6%
Node 3: Index usage is 4%
Node 4: Data usage is 6%
Node 4: Index usage is 4%
-----------------------------------
Here's a part of my management log:
-------------------------------------
2013-04-30 09:00:29 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1785 started. Keep GCI = 2924750 oldest restorable GCI = 2924751
2013-04-30 09:02:10 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1785 completed
2013-04-30 09:02:11 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1786 started. Keep GCI = 2924799 oldest restorable GCI = 2924800
2013-04-30 09:03:52 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1786 completed
2013-04-30 09:03:53 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1787 started. Keep GCI = 2924848 oldest restorable GCI = 2924849
2013-04-30 09:05:34 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1787 completed
2013-04-30 09:05:35 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1788 started. Keep GCI = 2924897 oldest restorable GCI = 2924898
2013-04-30 09:07:15 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1788 completed
2013-04-30 09:07:16 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1789 started. Keep GCI = 2924946 oldest restorable GCI = 2924947
2013-04-30 09:08:57 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1789 completed
2013-04-30 09:08:58 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1790 started. Keep GCI = 2924995 oldest restorable GCI = 2924996
2013-04-30 09:10:39 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1790 completed
2013-04-30 09:10:40 [MgmtSrvr] INFO -- Node 4: Local checkpoint 1791 started. Keep GCI = 2925044 oldest restorable GCI = 2925045
2013-04-30 09:11:29 [MgmtSrvr] ALERT -- Node 1: Node 3 Disconnected
2013-04-30 09:11:29 [MgmtSrvr] ALERT -- Node 1: Node 4 Disconnected
2013-04-30 09:11:29 [MgmtSrvr] ALERT -- Node 4: Forced node shutdown completed. Caused by error 1217: 'No message slogan found (please report a bug if you get this error code)(Unknown). Unknown'.
2013-04-30 09:11:29 [MgmtSrvr] ALERT -- Node 3: Forced node shutdown completed. Caused by error 1217: 'No message slogan found (please report a bug if you get this error code)(Unknown). Unknown'.
-------------------------------------
and Here's my config.ini file:
---------------------------------
[ndbd default]
# Options affecting ndbd processes on all data nodes:
NoOfReplicas=2 # Number of replicas
DataDir=C:/mysql/bin/cluster-data # Directory for each data node's data files
# Forward slashes used in directory path,
# rather than backslashes. This is correct;
# see Important note in text
DataMemory=7G # Memory allocated to data storage
IndexMemory=400M # Memory allocated to index storage
# For DataMemory and IndexMemory, we have used the
# default values. Since the "world" database takes up
# only about 500KB, this should be more than enough for
# this example Cluster setup.
MaxNoOfAttributes = 10000
MaxNoOfOrderedIndexes = 1024
MaxNoOfConcurrentTransactions = 500000
MaxNoOfConcurrentOperations = 500000
MaxNoOfLocalOperations = 550000
[ndb_mgmd]
# Management process options:
nodeId=1
HostName=10.245.2.10 # Hostname or IP address of management node
DataDir=C:/mysql/bin/cluster-logs # Directory for management node log files
[ndb_mgmd]
# Management process options:
nodeId=2
HostName=10.245.2.11 # Hostname or IP address of management node
DataDir=C:/mysql/bin/cluster-logs # Directory for management node log files
[ndbd]
# Options for data node "A":
HostName=10.245.2.14 # Hostname or IP address
[ndbd]
# Options for data node "B":
HostName=10.245.2.130 # Hostname or IP address
[mysqld]
# SQL node options:
HostName=10.245.2.10 # Hostname or IP address
[mysqld]
# SQL node options:
HostName=10.245.2.11 # Hostname or IP address
---------------------------------
Thanks in advance.
How to repeat:
1- create a new database .."hpm"
2- create a new table.In my case table name is "structres" with the following DDL:
------------------------------------------------
CREATE TABLE `structures` (
`STRUCTURE_ID` bigint(20) NOT NULL AUTO_INCREMENT,
`PARENT_STRUCTURE_ID` bigint(20) DEFAULT NULL,
`NAME_UK` varchar(250) DEFAULT NULL,
`NAME_DK` varchar(250) DEFAULT NULL,
`NAME_PL` varchar(250) DEFAULT NULL,
`NAME_SV` varchar(250) DEFAULT NULL,
`NAME_NO` varchar(250) DEFAULT NULL,
`NAME_FI` varchar(250) DEFAULT NULL,
`NAME_LT` varchar(250) DEFAULT NULL,
`DESCRIPTION_UK` text,
`DESCRIPTION_DK` text,
`DESCRIPTION_PL` text,
`DESCRIPTION_SV` text,
`DESCRIPTION_NO` text,
`DESCRIPTION_FI` text,
`DESCRIPTION_LT` text,
`IMAGE1` text,
`IMAGE2` text,
`SEQUENCE` bigint(20) DEFAULT NULL,
`CREATED_DATE` datetime DEFAULT NULL,
`CHANGED_DATE` datetime DEFAULT NULL,
`STATUS` varchar(10) DEFAULT 'INSERT',
`STRUCTURE` varchar(100) DEFAULT NULL,
`TYPE` varchar(1) DEFAULT NULL,
`ROOT_STRUCTURE_ID` bigint(20) DEFAULT NULL,
`TARGET` varchar(50) DEFAULT 'WEB',
PRIMARY KEY (`STRUCTURE_ID`)
) ENGINE=ndbcluster AUTO_INCREMENT=1363161223449 DEFAULT CHARSET=utf8
-------------------------------------------------
3- running a lot of inserts on hpm.structures table,where batch inserts 50 rows at the time.
4- after some time(around 20 minutes) you will get the error and all data nodes will force shutdown.