MySQL Bugs: #45119: Segmentation Fault on update of indexed column in mixed disk-memory column table

Bug #45119	Segmentation Fault on update of indexed column in mixed disk-memory column table
Submitted:	27 May 2009 8:44	Modified:	21 Oct 2009 9:30
Reporter:	Nathan Thera	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	MySQL Cluster: Disk Data	Severity:	S3 (Non-critical)
Version:	mysql-5.1-telco-7.06	OS:	Linux
Assigned to:	Assigned Account	CPU Architecture:	Any
Tags:	mysql-5.1.32 ndb-7.0.6, segmentation fault ndb disk error 6000 ndbd.cpp

Description:
Hi 

I am updating the columns of ndb mixed disk-memory tables. When I run a query that changes the value of an indexed column one of the ndb data nodes will die.
The queries are in the format of:
UPDATE table SET indexedColumn = "value" WHERE id = "idnum"
This error did not happen when all columns of the tables are in memory.

I am running mysql-5.1.32 ndb-7.0.5b freshly inserted ndb cluster with 4 nodes in a 2 replica setup.

Please let me know if you require any additional logs or test situations.

Below is some output from the logs and the trace file attached in the next comment.

Nathan

From out log:
2009-05-26 14:00:12 [ndbd] INFO     -- Received signal 11. Running error handler.
2009-05-26 14:00:12 [ndbd] INFO     -- Signal 11 received; Segmentation fault
2009-05-26 14:00:12 [ndbd] INFO     -- ndbd.cpp
2009-05-26 14:00:12 [ndbd] INFO     -- Error handler signal restarting system
2009-05-26 14:00:12 [ndbd] INFO     -- Error handler shutdown completed - exiting
2009-05-26 14:00:14 [ndbd] ALERT    -- Node 4: Forced node shutdown completed, restarting. Initiated by signal 11. Caused by error 6000: 'Error OS signal received(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.
2009-05-26 14:00:14 [ndbd] INFO     -- Ndb has terminated (pid 25720) restarting
2009-05-26 14:00:18 [ndbd] INFO     -- Configuration fetched from '172.29.71.119:1186', generation: 4
2009-05-26 14:00:18 [ndbd] INFO     -- Angel pid: 25719 ndb pid: 10306
NDBMT: non-mt
2009-05-26 14:00:18 [ndbd] INFO     -- NDB Cluster -- DB node 4
2009-05-26 14:00:18 [ndbd] INFO     -- mysql-5.1.32 ndb-7.0.5-beta --
2009-05-26 14:00:18 [ndbd] INFO     -- WatchDog timer is set to 6000 ms
2009-05-26 14:00:18 [ndbd] INFO     -- Ndbd_mem_manager::init(1) min: 6148Mb initial: 6532Mb
Adding 6532Mb to ZONE_LO (1,209023)
2009-05-26 14:00:20 [ndbd] INFO     -- Start initiated (mysql-5.1.32 ndb-7.0.5)

From error log: 
Time: Tuesday 26 May 2009 - 14:00:12
Status: Temporary error, restart node
Message: Error OS signal received (Internal error, programming error or missing error message, please report a bug)
Error: 6000
Error data: Signal 11 received; Segmentation fault
Error object: ndbd.cpp
Program: ndbd
Pid: 25720
Trace: /var/lib/mysql-cluster/ndb_4_trace.log.15
Version: mysql-5.1.32 ndb-7.0.5-beta
***EOM***

How to repeat:
1) Create ndb table with mixed disk and memory columns.
2) Create an index for one of the columns (memory column; did not test disk column)
3) Populate table with rows.
4) Change the value of the column with index.

One ndb data node dies at this point. Repeat until the cluster runs out of nodes to fully crash the nodes. The data node assigned Master is usually the one that dies.

Suggested fix:
-Only use memory columns for the table.
-Do not update columns with an index.
-Add new row with updated column value and remove the previous one.

trace log of the node that crashed

Attachment: ndb_4_trace.log.zip (application/x-zip-compressed, text), 57.12 KiB.

Thank you for the report.

Please provide output of SHOW CREATE TABLE table

Hi

I have updated trace logs and structure of the same issue running on 7.0.6.
This is from a freshly inserted NDB cluster.

Nathan

Out_log
2009-07-03 22:50:42 [ndbd] INFO     -- Received signal 11. Running error handler.
2009-07-03 22:50:43 [ndbd] INFO     -- Signal 11 received; Segmentation fault
2009-07-03 22:50:43 [ndbd] INFO     -- ndbd.cpp
2009-07-03 22:50:43 [ndbd] INFO     -- Error handler signal restarting system
2009-07-03 22:50:43 [ndbd] INFO     -- Error handler shutdown completed - exiting
2009-07-03 22:50:43 [ndbd] ALERT    -- Node 6: Forced node shutdown completed, restarting. Initiated by sign
al 11. Caused by error 6000: 'Error OS signal received(Internal error, programming error or missing error me
ssage, please report a bug). Temporary error, restart node'.
2009-07-03 22:50:43 [ndbd] INFO     -- Ndb has terminated (pid 11271) restarting
2009-07-03 22:50:54 [ndbd] INFO     -- Configuration fetched from '172.29.71.97:1186', generation: 1
2009-07-03 22:50:54 [ndbd] INFO     -- Angel pid: 22541 ndb pid: 14577
NDBMT: non-mt
2009-07-03 22:50:54 [ndbd] INFO     -- NDB Cluster -- DB node 6
2009-07-03 22:50:54 [ndbd] INFO     -- mysql-5.1.34 ndb-7.0.6 --
2009-07-03 22:50:54 [ndbd] INFO     -- WatchDog timer is set to 6000 ms
2009-07-03 22:50:54 [ndbd] INFO     -- Ndbd_mem_manager::init(1) min: 7172Mb initial: 7556Mb
Adding 7556Mb to ZONE_LO (1,241791)
2009-07-03 22:50:56 [ndbd] INFO     -- Start initiated (mysql-5.1.34 ndb-7.0.6)

Error_log
Time: Friday 3 July 2009 - 22:50:42
Status: Temporary error, restart node
Message: Error OS signal received (Internal error, programming error or missing error message, please report a bug)
Error: 6000
Error data: Signal 11 received; Segmentation fault
Error object: ndbd.cpp
Program: ndbd
Pid: 11271
Trace: /var/lib/mysql-cluster/ndb_6_trace.log.8
Version: mysql-5.1.34 ndb-7.0.6
***EOM***

Trace file of the 7.0.6 crash

Attachment: ndb_6_trace.log.zip (application/x-zip-compressed, text), 57.84 KiB.

Hi,

I tried with your table, and did not manage to reproduce.
Could you provide a dump of the table in question ?

/Jonas

Unfortunately the table is much too large to send.

I will see if I can create a smaller data set.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".