090513 11:57:58 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
090513 11:57:59  InnoDB: Started; log sequence number 0 46439
090513 11:57:59 [Note] NDB: NodeID is 5, management server '146.18.237.48:1186'
090513 11:57:59 [Note] NDB[0]: NodeID: 5, all storage nodes connected
090513 11:57:59 [Warning] NDB: server id set to zero will cause any other mysqld with bin log to log with wrong server id
090513 11:57:59 [Note] Starting Cluster Binlog Thread
090513 11:57:59 [Note] Event Scheduler: Loaded 0 events
090513 11:57:59 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.30-ndb-6.3.20-cluster-gpl'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Cluster Server (GPL)
090513 11:58:00 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$mysql/ndb_schema
090513 11:58:00 [Note] NDB Binlog: logging ./mysql/ndb_schema (UPDATED,USE_WRITE)
090513 11:58:00 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$mysql/ndb_apply_status
090513 11:58:00 [Note] NDB Binlog: logging ./mysql/ndb_apply_status (UPDATED,USE_WRITE)
INVALID SUB_GCP_COMPLETE_REP
gci_hi: 2523429
gci_lo: 5
sender: 1010004
count: 5
bucket count: 3
nodes: 3
/usr/sbin/mysqld(my_print_stacktrace+0x33)[0x852bee6]
/usr/sbin/mysqld(handle_segfault+0x358)[0x8218684]
/lib/tls/libpthread.so.0[0xcf59c8]
/lib/tls/libc.so.6(abort+0xe9)[0xba8209]
/usr/sbin/mysqld(_ZN14NdbEventBuffer24execSUB_GCP_COMPLETE_REPEPK17SubGcpCompleteRepj+0x22a)[0x850defe]
/usr/sbin/mysqld(_ZN3Ndb20handleReceivedSignalEP12NdbApiSignalP16LinearSectionPtr+0x254)[0x84f45de]
/usr/sbin/mysqld(_ZN3Ndb14executeMessageEPvP12NdbApiSignalP16LinearSectionPtr+0x2b)[0x84f53bf]
/usr/sbin/mysqld(_ZN17TransporterFacade8for_eachEP12NdbApiSignalP16LinearSectionPtr+0x86)[0x84fb76a]
/usr/sbin/mysqld(_Z7executePvP12SignalHeaderhPjP16LinearSectionPtr+0x5e4)[0x84fc67e]
/usr/sbin/mysqld(_ZN19TransporterRegistry6unpackEPjjt7IOState+0x1ba)[0x856fcfa]
/usr/sbin/mysqld(_ZN19TransporterRegistry12get_tcp_dataEP15TCP_Transporter+0xde)[0x853851c]
/usr/sbin/mysqld(_ZN19TransporterRegistry14performReceiveEv+0x80)[0x85386ec]
/usr/sbin/mysqld(_ZN17TransporterFacade13external_pollEj+0x5c)[0x84fb156]
/usr/sbin/mysqld(_ZN9PollGuard14wait_for_inputEi+0x98)[0x84fd174]
/usr/sbin/mysqld(_ZN3Ndb25waitCompletedTransactionsEiiP9PollGuard+0x64)[0x84f3fb0]
/usr/sbin/mysqld(_ZN3Ndb10poll_transEiiP9PollGuard+0x51)[0x84f40a5]
/usr/sbin/mysqld(_ZN3Ndb11sendPollNdbEiii+0x6b)[0x84f415f]
/usr/sbin/mysqld(_ZN14NdbTransaction14executeNoBlobsENS_8ExecTypeEN12NdbOperation11AbortOptionEi+0x86)[0x8503482]
/usr/sbin/mysqld(_ZN7NdbBlob23executePendingBlobReadsEv+0x4d)[0x84f01ef]
/usr/sbin/mysqld(_ZN7NdbBlob15readDataPrivateEPcRj+0x1e5)[0x84f0d5f]
/usr/sbin/mysqld(_ZN7NdbBlob18atNextResultCommonEv+0xa1)[0x84f3071]
/usr/sbin/mysqld(_ZN7NdbBlob21atNextResultNdbRecordEPKcj+0x7d)[0x84f31b9]
/usr/sbin/mysqld(_ZN16NdbScanOperation10nextResultEPPKcbb+0x11d)[0x84c9a65]
/usr/sbin/mysqld(_ZN16NdbScanOperation10nextResultEbb+0x43)[0x84c9b13]
/usr/sbin/mysqld(_Z36ndbcluster_setup_binlog_table_sharesP3THD+0x328)[0x8397efc]
/usr/sbin/mysqld(ndb_util_thread_func+0x374)[0x838ebcc]
/lib/tls/libpthread.so.0[0xcef3cc]
/lib/tls/libc.so.6(__clone+0x5e)[0xc47c3e]
090513 11:58:00 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 337730 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x8d6b898
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xb0bc236c thread_stack 0x30000
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at (nil) is an invalid pointer
thd->thread_id=0
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
090513 11:58:00 mysqld_safe Number of processes running now: 0
090513 11:58:00 mysqld_safe mysqld restarted
090513 11:58:01  InnoDB: Started; log sequence number 0 46439
090513 11:58:01 [Note] NDB: NodeID is 5, management server '146.18.237.48:1186'
090513 11:58:02 [Note] NDB[0]: NodeID: 5, no storage nodes connected (timed out)
090513 11:58:02 [Warning] NDB: server id set to zero will cause any other mysqld with bin log to log with wrong server id
090513 11:58:02 [Note] Starting Cluster Binlog Thread
090513 11:58:02 [Note] Event Scheduler: Loaded 0 events
090513 11:58:02 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.30-ndb-6.3.20-cluster-gpl'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Cluster Server (GPL)
090513 11:58:05 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$mysql/ndb_schema
INVALID SUB_GCP_COMPLETE_REP
gci_hi: 2523433
gci_lo: 7
sender: 1010004
count: 5
bucket count: 3
nodes: 3
/usr/sbin/mysqld(my_print_stacktrace+0x33)[0x852bee6]
/usr/sbin/mysqld(handle_segfault+0x358)[0x8218684]
/lib/tls/libpthread.so.0[0xcf59c8]
/lib/tls/libc.so.6(abort+0xe9)[0xba8209]
/usr/sbin/mysqld(_ZN14NdbEventBuffer24execSUB_GCP_COMPLETE_REPEPK17SubGcpCompleteRepj+0x22a)[0x850defe]
/usr/sbin/mysqld(_ZN3Ndb20handleReceivedSignalEP12NdbApiSignalP16LinearSectionPtr+0x254)[0x84f45de]
/usr/sbin/mysqld(_ZN3Ndb14executeMessageEPvP12NdbApiSignalP16LinearSectionPtr+0x2b)[0x84f53bf]
/usr/sbin/mysqld(_ZN17TransporterFacade8for_eachEP12NdbApiSignalP16LinearSectionPtr+0x86)[0x84fb76a]
/usr/sbin/mysqld(_Z7executePvP12SignalHeaderhPjP16LinearSectionPtr+0x5e4)[0x84fc67e]
/usr/sbin/mysqld(_ZN19TransporterRegistry6unpackEPjjt7IOState+0x1ba)[0x856fcfa]
/usr/sbin/mysqld(_ZN19TransporterRegistry12get_tcp_dataEP15TCP_Transporter+0xde)[0x853851c]
/usr/sbin/mysqld(_ZN19TransporterRegistry14performReceiveEv+0x80)[0x85386ec]
/usr/sbin/mysqld(_ZN17TransporterFacade13external_pollEj+0x5c)[0x84fb156]
/usr/sbin/mysqld(_ZN9PollGuard14wait_for_inputEi+0x98)[0x84fd174]
/usr/sbin/mysqld(_ZN9PollGuard22wait_for_input_in_loopEib+0x70)[0x84fd1ec]
/usr/sbin/mysqld(_ZN9PollGuard13wait_n_unlockEitjb+0x3d)[0x84fd2c9]
/usr/sbin/mysqld(_ZN16NdbDictInterface10dictSignalEP12NdbApiSignalP16LinearSectionPtrii14WaitSignalTypeijPKii+0x174)[0x84d2640]
/usr/sbin/mysqld(_ZN16NdbDictInterface21executeSubscribeEventER3NdbR21NdbEventOperationImpl+0xf7)[0x84d48ab]
/usr/sbin/mysqld(_ZN17NdbDictionaryImpl21executeSubscribeEventER21NdbEventOperationImpl+0x34)[0x84d48fc]
/usr/sbin/mysqld(_ZN21NdbEventOperationImpl14execute_nolockEv+0x6b)[0x850e18f]
/usr/sbin/mysqld(_ZN21NdbEventOperationImpl14execute_nolockEv+0x86)[0x850e1aa]
/usr/sbin/mysqld(_ZN21NdbEventOperationImpl7executeEv+0x33)[0x850e271]
/usr/sbin/mysqld(_ZN17NdbEventOperation7executeEv+0x1f)[0x84bde21]
/usr/sbin/mysqld(_Z27ndbcluster_create_event_opsP3THDP19st_ndbcluster_sharePKN13NdbDictionary5TableEPKc+0x4a0)[0x839b488]
/usr/sbin/mysqld(_Z30ndbcluster_create_binlog_setupP3THDP3NdbPKcjS4_S4_c+0x48d)[0x839cfdf]
/usr/sbin/mysqld(_ZN13ha_ndbcluster6createEPKcP8st_tableP24st_ha_create_information+0x301)[0x8390141]
/usr/sbin/mysqld(_Z27ha_create_table_from_engineP3THDPKcS2_+0x1cf)[0x82f23c5]
/usr/sbin/mysqld(_Z28ndb_create_table_from_engineP3THDPKcS2_+0x5c)[0x83853e4]
/usr/sbin/mysqld(_Z36ndbcluster_setup_binlog_table_sharesP3THD+0x4f2)[0x83980c6]
/usr/sbin/mysqld(ndb_util_thread_func+0x7d2)[0x838f02a]
/lib/tls/libpthread.so.0[0xcef3cc]
/lib/tls/libc.so.6(__clone+0x5e)[0xc47c3e]
090513 11:58:05 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 337730 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x94aa9f0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xb0bb836c thread_stack 0x30000
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at (nil) is an invalid pointer
thd->thread_id=0
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
090513 11:58:05 mysqld_safe Number of processes running now: 0
090513 11:58:05 mysqld_safe mysqld restarted
090513 11:58:05  InnoDB: Started; log sequence number 0 46439
090513 11:58:05 [Note] NDB: NodeID is 5, management server '146.18.237.48:1186'
090513 11:58:06 [Note] NDB[0]: NodeID: 5, no storage nodes connected (timed out)
090513 11:58:06 [Warning] NDB: server id set to zero will cause any other mysqld with bin log to log with wrong server id
090513 11:58:06 [Note] Starting Cluster Binlog Thread
090513 11:58:06 [Note] Event Scheduler: Loaded 0 events
090513 11:58:06 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.30-ndb-6.3.20-cluster-gpl'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Cluster Server (GPL)
090513 11:58:09 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$mysql/ndb_schema
090513 11:58:09 [Note] NDB Binlog: logging ./mysql/ndb_schema (UPDATED,USE_WRITE)
INVALID SUB_GCP_COMPLETE_REP
gci_hi: 2523437
gci_lo: 9
sender: 1010004
count: 5
bucket count: 3
nodes: 3
/usr/sbin/mysqld(my_print_stacktrace+0x33)[0x852bee6]
/usr/sbin/mysqld(handle_segfault+0x358)[0x8218684]
/lib/tls/libpthread.so.0[0xcf59c8]
/lib/tls/libc.so.6(abort+0xe9)[0xba8209]
/usr/sbin/mysqld(_ZN14NdbEventBuffer24execSUB_GCP_COMPLETE_REPEPK17SubGcpCompleteRepj+0x22a)[0x850defe]
/usr/sbin/mysqld(_ZN3Ndb20handleReceivedSignalEP12NdbApiSignalP16LinearSectionPtr+0x254)[0x84f45de]
/usr/sbin/mysqld(_ZN3Ndb14executeMessageEPvP12NdbApiSignalP16LinearSectionPtr+0x2b)[0x84f53bf]
/usr/sbin/mysqld(_ZN17TransporterFacade8for_eachEP12NdbApiSignalP16LinearSectionPtr+0x86)[0x84fb76a]
/usr/sbin/mysqld(_Z7executePvP12SignalHeaderhPjP16LinearSectionPtr+0x5e4)[0x84fc67e]
/usr/sbin/mysqld(_ZN19TransporterRegistry6unpackEPjjt7IOState+0x1ba)[0x856fcfa]
/usr/sbin/mysqld(_ZN19TransporterRegistry12get_tcp_dataEP15TCP_Transporter+0xde)[0x853851c]
/usr/sbin/mysqld(_ZN19TransporterRegistry14performReceiveEv+0x80)[0x85386ec]
/usr/sbin/mysqld(_ZN17TransporterFacade13external_pollEj+0x5c)[0x84fb156]
/usr/sbin/mysqld(_ZN9PollGuard14wait_for_inputEi+0x98)[0x84fd174]
/usr/sbin/mysqld(_ZN9PollGuard22wait_for_input_in_loopEib+0x70)[0x84fd1ec]
/usr/sbin/mysqld(_ZN9PollGuard13wait_n_unlockEitjb+0x3d)[0x84fd2c9]
/usr/sbin/mysqld(_ZN16NdbDictInterface10dictSignalEP12NdbApiSignalP16LinearSectionPtrii14WaitSignalTypeijPKii+0x174)[0x84d2640]
/usr/sbin/mysqld(_ZN16NdbDictInterface8getTableEP12NdbApiSignalP16LinearSectionPtrjb+0x78)[0x84de5e4]
/usr/sbin/mysqld(_ZN16NdbDictInterface8getTableERK10BaseStringb+0x1d3)[0x84de845]
/usr/sbin/mysqld(_ZN17NdbDictionaryImpl23fetchGlobalTableImplRefERK21GlobalCacheInitObject+0xf8)[0x84ded04]
/usr/sbin/mysqld(_ZNK13NdbDictionary10Dictionary14getTableGlobalEPKc+0x65)[0x84c2595]
/usr/sbin/mysqld(_Z34ndbcluster_read_binlog_replicationP3THDP3NdbP19st_ndbcluster_sharePKN13NdbDictionary5TableEjP8st_tableb+0x9d)[0x8399803]
/usr/sbin/mysqld(_Z30ndbcluster_create_binlog_setupP3THDP3NdbPKcjS4_S4_c+0x311)[0x839ce63]
/usr/sbin/mysqld(_ZN13ha_ndbcluster6createEPKcP8st_tableP24st_ha_create_information+0x301)[0x8390141]
/usr/sbin/mysqld(_Z27ha_create_table_from_engineP3THDPKcS2_+0x1cf)[0x82f23c5]
/usr/sbin/mysqld(_Z28ndb_create_table_from_engineP3THDPKcS2_+0x5c)[0x83853e4]
/usr/sbin/mysqld(_Z36ndbcluster_setup_binlog_table_sharesP3THD+0x879)[0x839844d]
/usr/sbin/mysqld(ndb_util_thread_func+0x7d2)[0x838f02a]
/lib/tls/libpthread.so.0[0xcef3cc]
/lib/tls/libc.so.6(__clone+0x5e)[0xc47c3e]
090513 11:58:09 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 337730 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0xa5caa00
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xb0b9836c thread_stack 0x30000
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at (nil) is an invalid pointer
thd->thread_id=0
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
090513 11:58:09 mysqld_safe Number of processes running now: 0
090513 11:58:09 mysqld_safe mysqld restarted
090513 11:58:09  InnoDB: Started; log sequence number 0 46439
090513 11:58:09 [Note] NDB: NodeID is 5, management server '146.18.237.48:1186'
090513 11:58:10 [Note] NDB[0]: NodeID: 5, no storage nodes connected (timed out)
090513 11:58:10 [Warning] NDB: server id set to zero will cause any other mysqld with bin log to log with wrong server id
090513 11:58:10 [Note] Starting Cluster Binlog Thread
090513 11:58:11 [Note] Event Scheduler: Loaded 0 events
090513 11:58:11 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.30-ndb-6.3.20-cluster-gpl'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Cluster Server (GPL)
090513 11:58:11 [Note] /usr/sbin/mysqld: Normal shutdown

090513 11:58:11 [Note] Event Scheduler: Purging the queue. 0 events
090513 11:58:11 [Note] Stopping Cluster Utility thread
090513 11:58:12 [Note] Stopping Cluster Binlog
090513 11:58:13  InnoDB: Starting shutdown...
090513 11:58:14  InnoDB: Shutdown completed; log sequence number 0 46439
090513 11:58:14 [Note] /usr/sbin/mysqld: Shutdown complete

090513 11:58:14 mysqld_safe mysqld from pid file /var/lib/mysql/ast1.nts.fedex.com.pid ended