Bug #52677 Cluster shuts down when stopping one node
Submitted: 8 Apr 2010 5:48 Modified: 6 Jun 2010 13:55
Reporter: vineet daniel Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:mysql-5.1-telco-7.0 OS:Linux (CentOS 5)
Assigned to: Assigned Account CPU Architecture:Any

[8 Apr 2010 5:48] vineet daniel
Description:
We were performing a rolling restart of our cluster and when I stopped one ndb node the whole cluster went down. We restarted the cluster as per the procedure and tried stopping the node again. Same thing happened. Further, We have just around 1 GB of data and the cluster takes almost hour and half to start, gets stuck in phase 5. 

We have two mgmd nodes on two seperate machines
Two ndbd nodes and mysqld nodes. 
Ndbd node and mysqld node reside on same machine.
Total 4 machines. 
2 working as mgmd nodes and others as ndbd and mysqld nodes.

How to repeat:
Just stop one of the ndbd nodes and the cluster will go down.
[8 Apr 2010 6:11] Sveta Smirnova
Thank you for the report.

Please provide all Cluster log files. Use  ndb_error_reporter  to generate the report.
[8 Apr 2010 6:22] vineet daniel
Hi 
Ive been trying to upload log files but I'm unable to do so, as ftp link given is broken and the size of log files is much more than specified. Kindly provide alternate ftp location.

Thanks
[8 Apr 2010 6:30] Sveta Smirnova
Thank you for the feedback.

I just checked: FTP link works. It is just not listable, you need to use FTP client to upload the file:

$touch bug52677

$ftp ftp://ftp.mysql.com/pub/mysql/upload/
Connected to ftp.mysql.com.
220 Welcome to MySQL AB's FTP service
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Switching to Binary mode.
250 Directory successfully changed.
250 Directory successfully changed.
250 Directory successfully changed.
ftp> put bug52677
local: bug52677 remote: bug52677
229 Entering Extended Passive Mode (|||11019|)
150 Ok to send data.
     0       0.00 KB/s 
226 File receive OK.
ftp> bye
221 Goodbye.
[8 Apr 2010 9:56] vineet daniel
log files have been uploaded on the ftp specified. Kindly check.
[8 Apr 2010 12:06] Sveta Smirnova
Thank you for the feedback.

What is the file name?
[8 Apr 2010 13:22] vineet daniel
the file name is
ndb_error_report_20100407215602.tar.bz2
[6 May 2010 12:27] Jonas Oreland
Hi,

I'm looking in cluster logs found in uploaded file,
and find no restart of any kind.

Did I miss anything, or which restart are you referring too ?

/Jonas
[6 May 2010 13:55] Jonas Oreland
Hi again,

sure, I understood the description. The problem is that
the logs does not seems to cover the time when this behavior
was observed.

I.e
1) *when* did you do this
2) and if (as I think) the cluster-logs does not cover this period, can you upload logs that does

/Jonas
[6 Jun 2010 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".