Description:
mysql client stopped reacting after a show tables in a database.
Closer inspection showed the following
061120 16:18:58 mysqld started
061120 16:18:59 InnoDB: Started; log sequence number 0 46403
061120 16:19:09 [Note] Starting MySQL Cluster Binlog Thread
061120 16:19:09 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.11-beta-log' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)
061120 16:19:09 [Note] SCHEDULER: Manager thread booting
061120 16:19:09 [Note] SCHEDULER: Loaded 0 events
061120 16:19:09 [Note] SCHEDULER: Suspending operations
061121 6:18:32 [ERROR] Invalid (old?) table or database name 'lost+found'
061121 6:18:32 [ERROR] Invalid (old?) table or database name 'mysql-cluster'
061121 8:31:24 [ERROR] Got error 4350 when reading table './pmt/terminalderivedmetric'
061121 8:32:02 [ERROR] Got error 4350 when reading table './pmt/terminalderivedmetric'
061121 8:32:32 [ERROR] Got error 4006 when reading table './ccmt/isplistforipaddresspool'
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
key_buffer_size=8388600
read_buffer_size=1044480
max_used_connections=35
max_connections=100
threads_connected=32
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 212591 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
thd=0xdb9dfff0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x30661c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x81d9210
0x8c88b8
(nil)
0x828c51e
0x828c322
0x81f015f
0x81f52b6
0x81f9abb
0x81f07e3
0x81f032d
0x81ef85f
0x8c3080
0x1f423e
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how
to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x94620d8 = commit
thd->thread_id=9830
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
Number of processes running now: 0
061121 08:33:00 mysqld restarted
061121 8:33:01 InnoDB: Started; log sequence number 0 46403
061121 8:33:01 [Note] Starting MySQL Cluster Binlog Thread
/usr/sbin/mysqld: Table 'general_log' is marked as crashed and should be repaired
/usr/sbin/mysqld: Table 'slow_log' is marked as crashed and should be repaired
061121 8:33:10 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.11-beta-log' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)
061121 8:33:10 [Note] SCHEDULER: Manager thread booting
061121 8:33:10 [Note] SCHEDULER: Loaded 0 events
061121 8:33:10 [Note] SCHEDULER: Suspending operations
061121 8:34:49 [ERROR] Invalid (old?) table or database name 'lost+found'
061121 8:34:49 [ERROR] Invalid (old?) table or database name 'mysql-cluster'
061121 8:39:02 [ERROR] Invalid (old?) table or database name 'lost+found'
061121 8:39:02 [ERROR] Invalid (old?) table or database name 'mysql-cluster'
061121 9:25:19 [ERROR] Invalid (old?) table or database name 'lost+found'
061121 9:25:19 [ERROR] Invalid (old?) table or database name 'mysql-cluster'
061121 9:35:07 [Note] /usr/sbin/mysqld: Normal shutdown
061121 9:35:07 [Note] SCHEDULER: Killing manager thread 2
061121 9:35:07 [Note] SCHEDULER: Waiting the manager thread to reply
061121 9:35:07 [Note] SCHEDULER: Resuming operations
061121 9:35:07 [Note] SCHEDULER: Manager thread started with id 2
061121 9:35:07 [Note] SCHEDULER: Shutting down
061121 9:35:07 [Note] SCHEDULER: Waiting for worker threads to finish
061121 9:35:07 [Note] SCHEDULER: Emptying the queue
061121 9:35:07 [Note] SCHEDULER: Stopped
061121 9:35:09 [Warning] /usr/sbin/mysqld: Forcing close of thread 33 user: 'ccmt'
061121 9:35:09 [Warning] /usr/sbin/mysqld: Forcing close of thread 32 user: 'ccmt'
061121 9:35:09 [Warning] /usr/sbin/mysqld: Forcing close of thread 31 user: 'ccmt'
061121 9:35:09 [Warning] /usr/sbin/mysqld: Forcing close of thread 30 user: 'pmt'
Multiple warning , I eventually had to kill mysqld myselve.
Looks like Bug #18279 and #19176
Also the ndb nodes seem to stay up and running. Connectinos from the other node keep working.
Trace :
thd=0xdb9dfff0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x30661c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x81d9210 handle_segfault + 416
0x8c88b8 (?)
(nil)
0x828c51e _Z19ha_commit_one_phaseP3THDb + 110
0x828c322 _Z15ha_commit_transP3THDb + 386
0x81f015f _Z9end_transP3THD25enum_mysql_completiontype + 95
0x81f52b6 _Z21mysql_execute_commandP3THD + 13382
0x81f9abb _Z11mysql_parseP3THDPcj + 347
0x81f07e3 _Z16dispatch_command19enum_server_commandP3THDPcj + 1123
0x81f032d _Z10do_commandP3THD + 141
0x81ef85f handle_one_connection + 655
0x8c3080 (?)
0x1f423e (?)
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how
to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
How to repeat:
not yet repeatable , only similar bugs