MySQL Bugs: #8996: MySQL regularly crashes with 4.1.11 latest bktree

Bug #8996	MySQL regularly crashes with 4.1.11 latest bktree
Submitted:	6 Mar 2005 16:43	Modified:	7 Mar 2005 15:40
Reporter:	jocelyn fournier (Silver Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S1 (Critical)
Version:	4.1.11 bktree	OS:	Linux (linux)
Assigned to:	Sergei Golubchik	CPU Architecture:	Any

Description:
Hi,

I'm using the latest 4.1.11 bktree, and MySQL seems to crash randomly every now and then, with always the same stack :

0x80c9c7d handle_segfault + 653
0x835a4ca __pthread_sighandler + 122
0x832170f alloc_root + 15
0x8095835 sql_alloc(unsigned int) + 37
0x8125ea1 push_warning(THD*, MYSQL_ERROR::enum_warning_level, unsigned int, char const*) + 209
0x80c08d3 send_error(THD*, unsigned int, char const*) + 211
0x80cb266 handle_connections_sockets + 1206
0x80ce37b main + 1563
0x8360d5c __libc_start_main + 652
0x8048121 _start + 33

Regards,
  Jocelyn

How to repeat:
Unfortunatelly, due to the random character of the crash, I've no clue on how to repeat this issue.

However, push_warning wasn't called at all in send_error in 4.1.10 code.

Hi!

Gluh has modified the code recently:

http://lists.mysql.com/internals/22361

I am assigning this critical bug to Gluh.

Thank you,

Heikki

Yes, push_warning call was added to to  send_error by Gluh.

But I fail to see how send_error can be called from handle_connections_sockets.

Jocelyn, it is optimized or debug build ?

Can you attach to the bugreport the output of 

  (gdb) disassemble handle_connections_sockets

?

(or 'objdump -ld mysqld.o' and remove everything but handle_connections_sockets function)

handle_connections_sockets disassembly

Attachment: handle_connections_sockets.asm (application/octet-stream, text), 28.56 KiB.

Hi Sergei,

It's an optimised build.
I've attached the disassembly of handle_connections_sockets.
However, I agree the stack looks weird, I would  have expect close_connection for example to appear in the stack.

  Jocelyn

Sorry, this disassembly is useless - it's full of unresolved references and I cannot see which function is called in each call. Could you repeat the same objdump... but for mysqld binary ?

Disassembly with proper symbol resolution

Attachment: handle_connections_socket.S (application/octet-stream, text), 28.44 KiB.

Hi Sergei,

Sorry about this, I've now uploaded the handle_connections_socket from the mysqld.S file, which has proper symbols resolution (you can see the call to send_error at  80cb261).

  Jocelyn

Hi,

After taken a quick look at the disassembly, it looks like the close_connection function has been inlined in handle_connections_sockets. (from  80cb22b to 80cb275)

  Jocelyn

Right, gcc inlined create_new_thread and close_connection.
Now I can repeat the crash in debug non-optimized build too - the trick is to set max_connections and try to open more connections than allowed

Sergey,
for what it is worth, 5.0.3 has been crashing on me when I exceed the maximum number of connections, since November or December 2004. Maybe it's the same bug?

Marko - I just tried, 5.0.3 did not crash.
In any case it would be a different bug as the code is different there.
But please, do submit (separately) it if you see the crash once again.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/22726

Hi,

I just wonder if the same kind of crash couldn't occur in net_printf (which is called for example for a "can't create new thread" error in create_new_thread) since push_warning has been added in this function as well (adding the test on thd->killed should fix the problem since thd->killed is properly set in create_new_thread in this case).

  Jocelyn

Sergei,
5.0 still has the bug, which I have now reported as Bug #9022.

Jocelyn - yes, it probably could. I now fixed it there too, thanks.

Though it would be much more difficult it - it requires pthread_create to fail, which is not as easy to achieve as going over --max-connections.

Anyway, both fixed in 4.1.11