MySQL Bugs: #10340: Stress: MySQL daemon NOT restarting if server constantly hit by many clients

Bug #10340	Stress: MySQL daemon NOT restarting if server constantly hit by many clients
Submitted:	3 May 2005 15:58	Modified:	13 Jul 2005 12:30
Reporter:	Disha	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	MySQL Server	Severity:	S2 (Serious)
Version:	5.0.4 Beta	OS:	Linux (Linux)
Assigned to:		CPU Architecture:	Any

Description:
MySQL daemon fails to start after crashing if the server is being continously hit by clients. When we try to relaunch the daemon, it dies out with an error "mysqld got signal 11"

How to repeat:
Start the MySQL server daemon and have ~100 clients querying the server continously, kill the MySQL daemon and then try to relaunch it while the clients are trying to send the queries. Observe that the MySQL daemon starts and then again dies with an unexpected error.

Expected Results : The MySQL daemon should start even if the clients are sending queries.

Actual Results : The MySQL daemon fails to start and the following dump is produced:

050503 13:52:36 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
050503 13:52:36 InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 11 69914687.
InnoDB: Doing recovery: scanned up to log sequence number 11 69914687
InnoDB: Last MySQL binlog file position 0 0, file name
050503 13:52:36 InnoDB: Started; log sequence number 11 69914687
050503 13:52:36 InnoDB: Starting recovery for XA transactions...
050503 13:52:36 InnoDB: 0 transactions in prepared state after recovery
050503 13:52:36 [Note] ./mysqld: ready for connections.
Version: '5.0.4-beta-standard' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Edition - Standard (GPL)
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=10
max_connections=100
threads_connected=10
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Hi!

Can you please send us a test-case that shows this behaviour ? .. maybe a perl or c app will do the job.

Thanks.

Disha,

can you compile mysqld yourself:

CXXFLAGS="-O3 -g" CFLAGS="-O3 -g" ./configure
make

and run it inside gdb:

gdb mysqld

run

When it chrashes, do

bt full

and post the stack trace.

Regards,

Heikki

Hi, we have a debug build... we will take stack traces and get it accross to you.

Stack trace

Attachment: Trace_logs.zip (application/x-zip-compressed, text), 71.16 KiB.

Stack trace attached. Although I am not sure if these traces points out the exact issue.

Thank you for this bug report. To properly diagnose the problem, we
need a backtrace to see what is happening behind the scenes. To
find out how to generate a backtrace, please read
http://www.mysql.com/doc/en/Making_trace_files.html

Once you have generated a backtrace, please submit it to this bug
report and change the status back to 'Open'. Thank you for helping
us make our products better.

Additional info:

Disha,
we need resolved stack traces. If you can run mysqld inside gdb, type
"bt full" when it crashes. Otherwise, please resolve the addresses reported after "Stack range sanity check OK, backtrace follows". You can use the script mysql-test/resolve-stack for that.

I'm sorry for sending an inappropriate pointer to the manual. The InnoDB storage engine does not make use of the DBUG package. What we need is a resolved stack trace.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

Disha,
can you please check if this bug still occurs with 5.0.9, and post a stack trace (not MySQL debug trace, but a real stack trace from gdb)?