Bug #31371 ndbd restart killed by signal 11

• Submitted: 3 Oct 2007 14:07
• Modified: 1 Mar 2008 9:22
• Reporter: Simon Tretter
• Status: No Feedback
• Category: MySQL Cluster: Cluster (NDB) storage engine
• Severity: S2 (Serious)
• Version: 5.1.22
• OS: Linux
• CPU Architecture: Any
• Tags: ndbd, restart, Signal 11

[3 Oct 2007 14:07] Simon Tretter

Description:
if i restart ndbd, the ndbd process on one server (everytime the same one, two total nodes) stops in phase 4 with signal 11.
2007-10-03 15:52:20 [ndbd] ALERT    -- Node 11: Forced node shutdown completed. Initiated by signal 11.
some milliseconds later the second node goes down, cause of the failure of the first node.

also strange is that no trace file is created?

last log lines from node 10:
RESTORE table: 377 561827 rows applied
RESTORE table: 362 2 rows applied
RESTORE table: 365 2 rows applied
RESTORE table: 366 0 rows applied
RESTORE table: 368 0 rows applied
RESTORE table: 370 11 rows applied
RESTORE table: 373 0 rows applied
RESTORE table: 377 561827 rows applied
2007-10-03 15:52:27 [ndbd] INFO     -- Node 11 disconnected
2007-10-03 15:52:27 [ndbd] INFO     -- QMGR (Line: 2648) 0x0000000a
2007-10-03 15:52:27 [ndbd] INFO     -- Error handler startup shutting down system
2007-10-03 15:52:27 [ndbd] INFO     -- Error handler shutdown completed - exiting
2007-10-03 15:52:27 [ndbd] INFO     -- Angel received ndbd startup failure count 1.
2007-10-03 15:52:27 [ndbd] ALERT    -- Node 10: Forced node shutdown completed. Occured during startphase 4. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.

last log lines from node 11:
RESTORE table: 347 9 rows applied
RESTORE table: 349 10 rows applied
RESTORE table: 351 0 rows applied
RESTORE table: 353 0 rows applied
RESTORE table: 357 0 rows applied
RESTORE table: 359 0 rows applied
RESTORE table: 362 2 rows applied
RESTORE table: 365 2 rows applied
RESTORE table: 366 0 rows applied
RESTORE table: 368 0 rows applied
RESTORE table: 370 14 rows applied
RESTORE table: 373 0 rows applied
RESTORE table: 377 562383 rows applied
2007-10-03 15:52:20 [ndbd] ALERT    -- Node 11: Forced node shutdown completed. Initiated by signal 11.

How to repeat:
i can't say how to repeat it, if i do --initial setup again, the cluster starts up again.

[3 Oct 2007 16:54] Simon Tretter

additional infos:
- ndb nodes: 2x 8gig memory, dual core
- apis are on the nodes with minimal config
- 3th server with managment node
- 142 tables, between 20 - 25 000 000 entrys per table
- total size of ~6 gig in ndb

[1 Feb 2008 9:22] Valeriy Kravchuk

Please, upload the logs from all the nodes.

[2 Mar 2008 0:00] Bugs System

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".