MySQL Bugs: #109011: MySQL spawn hundreds of threads and hang on too many connections

Bug #109011	MySQL spawn hundreds of threads and hang on too many connections
Submitted:	4 Nov 2022 21:47	Modified:	7 Nov 2022 13:44
Reporter:	Milutin Kubik	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	MySQL Server	Severity:	S3 (Non-critical)
Version:	8.0.31	OS:	CentOS (CloudLinux 7)
Assigned to:		CPU Architecture:	x86

Description:
Our high traffic server started hanging after updating to 8.0.31. Symptoms were hundreds of threads spawned (as many as max_connections) and then will remain locked with Too many connections error. 

After receiving SHUTDOWN by SystemD it won't stop and has to be killed by SIGTERM. No other option. 

No errors in log. Happened randomly sometimes after hours sometimes few times in 10 minutes. No specific queries were detected causing this.

How to repeat:
Traffic around 8-10K queries per second. Total DB size of 200GB. After a while sometimes several hours it will spawn many processes and hang up on Too many connections and never cleared up such connections.

Hi Mr. Kubik,

Thank you for your bug report.

However, it is not a bug.

Our server does not create connections at it's own will. Connections are created by various applications, users, proxies, replicated servers , routers etc .....

Connections are closed once that users have disconnected, except in the case that server is misconfigured. Misconfiguration is the problem for DBA, but it is not a bug in our Server.

Not a bug.

No other changes were introduced to system other than unattended update to 8.0.31. Immediately after migrating back to 8.0.30 issue is no longer present.

Hi Mr. Kubik,

We only have one proper test case for a bug that shows similar behaviour. However, unlike your report, that bug report comes with a fully repeatable test case. It is not enough just to spawn couple of thousands connections as we can do that easily with mysqlslap. It takes some configuration setup.

Hence, check for the Release Notes for 8.0.32, once when it is out. It is not yet known when will that happen. Even internally, it becomes known only couple of days before it is published.

Sorry, we are experiencing a similar bug to the one reported by this user after upgrading multiples servers to 8.0.31, (we have detected the same bug after upgrading from 5.7.39 to 5.7.40).

I would like to know if you can tell me the URL of the bug that another user reported with similar symptoms and that has a test case, to see if we can get some more information about this problem, or if we can contribute something new.

Thanks.

Hi Mr. Neto,

If you have read our comments, you would have learned that a proper test case is provided in the internal bug. Internal bugs are filed in an internal server, which is not connected to the Internet.

Hence, once that 8.0.32, read the Release Notes and see if there is a bug fixed that is related to the hanging connection. Release 8.0.32 should be published in the couple of months ....