MySQL Bugs: #105101: MySQL DB Server continuous memory and swap consumption increase

Bug #105101	MySQL DB Server continuous memory and swap consumption increase
Submitted:	1 Oct 2021 15:29	Modified:	6 Oct 2021 3:16
Reporter:	Cristiano Silva	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	MySQL Server	Severity:	S3 (Non-critical)
Version:	5.7.29	OS:	Red Hat
Assigned to:		CPU Architecture:	Any

Description:
The issue has happened with two different servers so far. The only solution found for both cases is to restart the server to free swap and memory.

From July to the 1st of October, memory consumption went from 40.74G to 57.96G averaging an 11% increase per month. Swap consumption was more impressive, from 84M to 2.36G in the same period. The increases were respectful of 90%, 77%, and 38%, comparing the memory use at the start of each month.

It seems like an indication of a memory leak. Still, this server does not have any other applications besides the MySQL DB, which replicates to two different servers. The only further indication of swap or memory consumption that we found is information schema seems to be holding into 55 temp tables.

How to repeat:
Start the server, monitor memory, and swap consumption with a service such as Datadog, etc., until it reaches the max threshold memory and swap available on the server.

Suggested fix:
At this moment, restart MySQL Server.

Hi Mr. Silva,

Thank you for your bug report.

However, it is not a bug.

First of all, you are using an old release of 5.7. Between 5.7.29 and 5.7.35 there are a number of bugs fixed that are dealing with some small memory leaks, that might accumulate over the time.

Next, we can not process any bug report without a proper proof. Usually, we require a test case, but in your case, the output from the reports by Valgrind or GNU malloc (in diagnostic mode), Memwatch or similar. We would also accept the analysis of our code that proves that memory is not appropriately returned to the OS.

Next, Linux malloc libraries have a tendency of keeping memory, that is deallocated, to the process that consumes most memory, which in your case is MySQL server. You can easily reclaim the memory by writing a short C program that would just calloc() some amount of memory that you have seen increasing.

It is also expected behaviour that memory increases after starting MySQL, for the reasons described in the previous paragraph, but also due to the larger number of connections.

It is also true that our I_S schema uses mostly memory-based tables for the output, but those are deallocated as soon as the query finishes.

Not a bug.

Hi,

Thanks for the feedback. I'm scheduling an update immediately for this. We have performed a restart of the server and for 10 hours it was good until it spiked again. Memory is a slow spike climb and swap is a huge spike.

Right now the MySQL update is our next hope.