Bug #95893 A client process may be blocked permanently on network congestion
Submitted: 20 Jun 9:29 Modified: 20 Jun 9:34
Reporter: Wei Zhao (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Connection Handling Severity:S2 (Serious)
Version:5.6,5.7,8.0 OS:Any
Assigned to: CPU Architecture:Any
Tags: Contribution

[20 Jun 9:29] Wei Zhao
Description:
A mysql client process can be blocked permanently if the session net_write_timeout is short(e.g. 1 second) and the query result takes a lot of packets to transmit and there is a network congestion during the transmission from mysqld to the client process.

How to repeat:
At server side prepare a table my_big_table with a lot of data(e.g. 32MB), and set global and session net_write_timeout=1. And to imitate a network congestion that surely happens, we have to use gdb to block the execution of mysqld and client mysql at the right place. Use gdb to attach to the mysqld process and set a breakpoint at function vio_write() and vio_io_wait().Then in the client, issue a ‘select * from my_big_table’, and almost immediately use gdb to attach to the client mysql process, then you will most probably be blocked at such a callstack, and keep it blocked.

Then in the gdb attached to mysqld process, you will meet many breakpoint hits in vio_write() (server sending result packets to client) and then finally you will see vio_io_wait() (send() would block because OS socket write buffer is full) is called and in vio_io_wait(), this statement is executed:

errno= SOCKET_ETIMEDOUT;

Then the statement execution at server side finishes successfully, but only partial results are sent to the client.At client side, however, the query statement blocks forever even if you quit the attached gdb.

Suggested fix:
The cause of this issue is that if the poll() called in vio_io_wait() times out, net_write_raw_loop() return error and the execution of the sql statement completes. The timeout error is simply ignored, and this is wrong! Server side should have disconnected the socket connection so that client side can return from blocked recv() syscall.
[20 Jun 9:31] Wei Zhao
this patch fixes the bug

(*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.

Contribution: client-permanent-block-fix.diff (application/octet-stream, text), 1.06 KiB.

[20 Jun 9:34] Umesh Shastry
Hello Wei Zhao,

Thank you for the report and contribution.

regards,
Umesh