Bug #105612 Server hangs - Flags for USE_PPOLL_IN_VIO not propagated when client uses SSL
Submitted: 17 Nov 2021 11:35 Modified: 17 Nov 2021 11:54
Reporter: Kamil Holubicki (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Connection Handling Severity:S3 (Non-critical)
Version:8.0.27, 5.7.36 OS:Any
Assigned to: CPU Architecture:Any
Tags: Contribution

[17 Nov 2021 11:35] Kamil Holubicki
Description:
This issue is hard to reproduce, and I don't have a reliable way to do it.

How to repeat:
I was able to trigger it with long, parallel execution of mysql_client_test MTR test:
./mysql-test/mtr --parallel=12 mysql_client_test{,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,}
but it is rare.
It looks like it occurs randomly, and the common denominator is having the following log in the error file:
Forcefully disconnecting 0 remaining clients

So the mechanism is that we have opened the client's SSL connection and we are shutting down the server from yet another client session.

Suggested fix:
Code analysis leads to the conclusion that there is missing propagation of thread_id, signal_mask and poll_shutdown_flag in vio.cc::vio_reset() which is called during SSL handshake. These flags are used for shutting down the client's connection socket.

Please see the attached patch for the fix proposal.
[17 Nov 2021 11:36] Kamil Holubicki
proposed fix

(*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.

Contribution: use_ppoll_in_vio.patch (text/x-patch), 552 bytes.

[17 Nov 2021 11:38] Kamil Holubicki
Note:
Forcefully disconnecting 0 remaining clients

in the original description should be:

Forcefully disconnecting 1 remaining clients
[17 Nov 2021 11:54] MySQL Verification Team
Hello Kamil Holubicki ,

Thank you for the report and feedback.
Verified as described with 8.0.27 build.

regards,
Umesh
[17 Nov 2021 11:58] MySQL Verification Team
-- 8.0.27
 ./mysql-test/mtr --parallel=50 mysql_client_test{,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,}
-bash: ./mysql-test/mtr: No such file or directory
[umshastr@support-cluster03:/export/home/tmp/ushastry/mysql-8.0.27/mysql-test]$ ./mtr --parallel=50 mysql_client_test{,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,}
Logging: ./mtr  --parallel=50 mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test mysql_client_test
MySQL Version 8.0.27
Checking supported features
Using 'all' suites
Collecting tests
Checking leftover processes
Removing old var directory
Creating var directory '/export/home/tmp/ushastry/mysql-8.0.27/mysql-test/var'
Installing system database
Using parallel: 32

==============================================================================
                  TEST NAME                   WORKER RESULT  TIME (ms) COMMENT
------------------------------------------------------------------------------
[  3%] main.mysql_client_test                   w24 [ pass ]  63949
[  6%] main.mysql_client_test                   w20 [ pass ]  65492
[  9%] main.mysql_client_test                   w4  [ pass ]  65828
.
.

## 
 find ./var/ -type f -exec grep -H 'Forcefully disconnecting 1 remaining clients' {} \;
./var/5/log/mysqld.1.err:2021-11-17T11:53:52.330646Z 0 [Note] [MY-010118] [Server] Forcefully disconnecting 1 remaining clients
./var/23/log/mysqld.1.err:2021-11-17T11:53:52.465721Z 0 [Note] [MY-010118] [Server] Forcefully disconnecting 1 remaining clients
[26 Jan 2022 8:43] Frederic Descamps
This contribution has been processed by our development team and has been accepted.

It will be included into a future release.

Thank you again.