Bug #85569 Forced node shutdown completed, restarting.
Submitted: 21 Mar 2017 16:41 Modified: 26 Apr 2017 18:57
Reporter: Danilo Escolin Email Updates:
Status: No Feedback Impact on me:
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:5.7.16-ndb-7.5.4-cluster-gpl OS:Red Hat
Assigned to: Bogdan Kecman CPU Architecture:Any
Tags: cluster, MySQL, Node failure caused abort of transaction, temporary error 4010

[21 Mar 2017 16:41] Danilo Escolin
#uname -a
Linux srv-db1 3.10.0-514.6.2.el7.x86_64 #1 SMP Thu Feb 23 03:04:39 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

#cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"

I have two hosts with 1 ndb_mgmd, 2 ndbd and 1 mysqld running in each host. The commands to start is: 

./ndb_mgmd --initial --ndb-nodeid=49 --config-dir=/home/mysql/cluster/admin/ --config-file=/home/mysql/cluster/admin/config.ini

./ndbd --ndb-nodeid=1 --ndb-connectstring=,,

./mysqld --defaults-file=/home/mysql/cluster/server/my.cnf

The problem is when i delete more than 7000 rows, on tables with references(i attach an example) the node restart and in log file shows: Forced node shutdown completed, restarting. Initiated by signal 6. Caused by error 6000: 'Error OS signal received(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.

How to repeat:
On table VALUE_LIST with more than 7000 rows, when delete all data, the node restart.
[21 Mar 2017 16:44] Danilo Escolin
cluster config

Attachment: config.ini (application/octet-stream, text), 1.90 KiB.

[21 Mar 2017 16:45] Danilo Escolin
mysqld config

Attachment: my.cnf (application/octet-stream, text), 712 bytes.

[21 Mar 2017 16:46] Danilo Escolin
test case

Attachment: test.sql (application/octet-stream, text), 2.83 KiB.

[22 Mar 2017 16:44] Bogdan Kecman

Please generate (with ndb_error_reporter) and upload full log of the crash.

[22 Mar 2017 17:06] Danilo Escolin

I copied the file to sftp.oracle.com /support/incoming/
[22 Mar 2017 17:14] Bogdan Kecman

Many files there :) can you please tell me the name of the file you uploaded :)

best regards
[22 Mar 2017 17:18] Danilo Escolin
Sorry, the file name is: ndb_error_report_85569.tar.bz2
[26 Mar 2017 18:57] Bogdan Kecman
Hi Danilo,

[root@localhost O]# tar jtvf ndb_error_report_85569.tar.bz2
drwxr-xr-x root/root         0 2017-03-20 21:45 ndb_error_report_20170320144534/
-rwxr-xr-x root/mysql 10891298 2017-03-02 17:34 ndb_error_report_20170320144534/mysql

The file you uploaded is archive of mysqld binary. What I need you to do is to use ndb_error_reporter to collect and archive all log files. No binaries needed. Not sure how this happened, ndb_error_reporter does not pack binaries??!

best regards
[27 Apr 2017 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".