Bug #43026 ndb_error_reporter fails to collect NDB management daemon info
Submitted: 19 Feb 2009 15:22 Modified: 30 Jul 2009 14:40
Reporter: Guido Ostkamp Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:mysql-5.1-telco-7.0 OS:Any
Assigned to: CPU Architecture:Any
Tags: 5.1-telco-6.4 -> 6.4.3

[19 Feb 2009 15:22] Guido Ostkamp
Description:
The tool 'ndb_error_reporter' fails to collect info for node 1, which is the management node with exception of 'config.ini', e.g. files like ndb_1_cluster.log* are not included.

Tests were run on Solaris 10 Sparc compiled with Sun Studio 12.
Version used is bazaar revision 
jonas@mysql.com-20090218220511-bgnaexvwjjfq2g6w dated Wed 2009-02-18 23:05:11
on Solaris 10 Sparc, compiled with
CC=cc CXX=CC CFLAGS="-xO5 -fast -g -mt -m64" CXXFLAGS="-xO5 -fast -g -mt -m64" ./configure
--prefix=/export/home/wsch/6.4_2009_01_29 --with-plugins=all --without-docs --without-man

******
root@eibe# ./ndb_error_reporter ./config.ini

 Copying data from node 2

root@clusternode1-priv's password: 
ndb_2.pid                                                                                100%    6     0.0KB/s   00:00    
ndb_2_error.log                                                                          100% 8053     7.9KB/s   00:00    
scp: /global/TspArchiveLog1/mysql-cluster/ndb_2_fs: not a regular file
ndb_2_out.log                                                                            100% 2227KB   2.2MB/s   00:01    
ndb_2_trace.log.15                                                                       100%  914KB 913.7KB/s   00:00    
ndb_2_trace.log.15_t1                                                                    100%  862KB 862.1KB/s   00:00    
ndb_2_trace.log.15_t2                                                                    100%  881KB 880.7KB/s   00:00    
ndb_2_trace.log.15_t3                                                                    100%  881KB 880.7KB/s   00:00    
ndb_2_trace.log.15_t4                                                                    100%  881KB 880.7KB/s   00:00    
ndb_2_trace.log.15_t5                                                                    100%  881KB 880.7KB/s   00:00    
ndb_2_trace.log.15_t6                                                                    100%  953KB 953.3KB/s   00:00    
ndb_2_trace.log.next                                                                     100%    2     0.0KB/s   00:00    

 Copying data from node 3

root@clusternode2-priv's password: 
ndb_3.pid                                                                                100%    5     0.0KB/s   00:00    
ndb_3_error.log                                                                          100%   12KB  12.3KB/s   00:00    
scp: /global/TspArchiveLog2/mysql-cluster/ndb_3_fs: not a regular file
ndb_3_out.log                                                                            100% 5969KB   5.8MB/s   00:01    
ndb_3_trace.log.24                                                                       100%  914KB 914.5KB/s   00:00    
ndb_3_trace.log.24_t1                                                                    100%  862KB 862.1KB/s   00:00    
ndb_3_trace.log.24_t2                                                                    100%  889KB 888.7KB/s   00:00    
ndb_3_trace.log.24_t3                                                                    100%  889KB 888.7KB/s   00:00    
ndb_3_trace.log.24_t4                                                                    100%  889KB 888.7KB/s   00:00    
ndb_3_trace.log.24_t5                                                                    100%  889KB 888.7KB/s   00:01    
ndb_3_trace.log.24_t6                                                                    100%  985KB 985.4KB/s   00:00    
ndb_3_trace.log.next                                                                     100%    2     0.0KB/s   00:00    

 Copying data from node 1

root@eibe's password: 
scp: /ndb_1*: No such file or directory

 Copying configuration file...

        ./config.ini

Please attach ndb_error_report_20090219160357.tar.bz2 to your error report
******

How to repeat:
Run ndb_error_reporter on management node in Solaris Cluster with 2 data nodes.
[19 Feb 2009 15:44] Stewart Smith
What is the DataDir for the management server? and log destination (if any).
[20 Feb 2009 9:37] Guido Ostkamp
The ndb_mgmd was started with command line

./ndb_mgmd --configdir=. -f config.ini --initial --reload

current dir at time of startup was /var/lib/mysql-cluster (also when ndb_error_reporter was called later).

Our config.ini:

[NDBD DEFAULT]
NoOfReplicas=2
DataMemory=2048M
IndexMemory=512M
MaxNoOfConcurrentOperations=262144
MaxNoOfConcurrentScans=500
MaxNoOfLocalScans=4096
BatchSizePerLocalScan=256
TransactionDeadlockDetectionTimeout=30000
LockPagesInMainMemory=1
FragmentLogFileSize=256M
NoOfFragmentLogFiles=12
RedoBuffer=32M
Diskcheckpointspeed=2M
Diskcheckpointspeedinrestart=100M
TimeBetweenGlobalCheckpoints=1000
TimeBetweenLocalCheckpoints=20
MaxNoOfExecutionThreads=8

[MYSQLD DEFAULT]
BatchSize=992
MaxScanBatchSize=4M

[NDB_MGMD DEFAULT]

[TCP DEFAULT]
SendBufferMemory=2M
ReceiveBufferMemory=2M

# Managment Server

[NDB_MGMD]
HostName=eibe                   # the IP of THIS SERVER

# Storage Engines

[NDBD]
Id=2
HostName=clusternode1-priv      # the IP of the FIRST SERVER
DataDir=/global/TspArchiveLog1/mysql-cluster

[NDBD]
Id=3
HostName=clusternode2-priv      # the IP of the SECOND SERVER
DataDir=/global/TspArchiveLog2/mysql-cluster

# 2 MySQL Clients
# I personally leave this blank to allow rapid changes of the mysql clients;
# you can enter the hostnames of the above two servers here. I suggest you dont.

[MYSQLD]
[MYSQLD]
...

Regards

Guido Ostkamp
[10 Jun 2009 12:48] Jonathan Miller
For tracing you can use ndb api tracing.
[30 Jul 2009 14:40] Geert Vanderkelen
Can't be repeated using MySQL Cluster 7.0.6 (which 'replaced' 6.4).

If you can, please run for both management nodes:
 ndb_config --config-file=config.ini --id=1 --query=datadir

This is used for copying, at apparently it was missing in your case.
 scp: /ndb_1*: No such file or directory

Cheers,
Geert