Bug #31328 Cluster restart fails after creating a lot of datafiles

• Submitted: 2 Oct 2007 1:25
• Modified: 17 Mar 2008 15:29
• Reporter: Shigenori Yatsuhashi
• Status: No Feedback
• Category: MySQL Cluster: Disk Data
• Severity: S1 (Critical)
• Version: 5.1.21
• OS: Linux (RHE Linux ES 4(Update 5))
• CPU Architecture: Any
• Tags: cluster, Datafile

[2 Oct 2007 1:25] Shigenori Yatsuhashi

Description:
The CPU usage of ndbd became 100% when adding 45th datafile(each datafile's initial_size is 512MB, and extent_size is 32KB) to a tablespace and there is no responce for query 'ALTER TABLESPACE...'.
I killed the query, but the cpu usage of ndbd kept 100% at all of 4 data nodes.
After shutdown cluster by ndb_mgm and start ndb_mgmd and ndbd, ndb_mgm showed following message.

Node 5: Forced node shutdown completed. Occured during startphase 4. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.
Node 6: Forced node shutdown completed. Occured during startphase 4. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.
Node 4: Forced node shutdown completed. Occured during startphase 4. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.
Node 3: Forced node shutdown completed. Occured during startphase 4. Caused by error 2341: 'Internal program error (failed ndbrequire)(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.

Similar case occured when adding 25th datafile(initial_size:1GB, extent_size:32KB).But adding 28 datafiles(initial_size:1GB, exten_size:1MB) succeeded.

How to repeat:
Add 45 datafiles(datafile_size: 512MB, extent_size: 32KB).

[2 Oct 2007 1:43] Shigenori Yatsuhashi

trace.log

Attachment: ndb_3_trace.log.5.gz (application/x-gzip, text), 48.86 KiB.

[2 Oct 2007 1:52] Shigenori Yatsuhashi

I missed number of datafiles. 46 is correct.
Sorry.

[17 Feb 2008 15:29] Valeriy Kravchuk

Thank you for a problem report. Please, try to repeat with a newer version, 5.1.23-rc, and inform about the results.

[18 Mar 2008 0:02] Bugs System

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".