MySQL Bugs: #111657: Memory not released on drop table

Bug #111657	Memory not released on drop table
Submitted:	4 Jul 2023 12:48	Modified:	4 Jul 2023 13:16
Reporter:	Emmanuel Duverney	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	MySQL Server	Severity:	S2 (Serious)
Version:	8.033	OS:	Windows (Server 2022)
Assigned to:		CPU Architecture:	Any
Tags:	drop table, Memory Engine, myisam

Description:
Memory not released when dropping MyISAM tables with MEMORY engine. This happened on two Windows Server 2022 machines with MySQL 8.0.33. Replacing MySQL 8.0.33 with MySQL 8.0.24 solved the problem. No test done with other versions.

How to repeat:
Install MySQL 8.0.33 on Windows Server 2022. Create a schema and add a table with a MEMORY engine to it. Fill the table with enough data to see the memory consumption with the process explorer of Windows. Run drop table statement on the table. Execute a select count(*) query on the table: an error occurs because the table is no longer available. Check the memory consumption with the process explorer to see that the memory is as high as it was before dropping the table.

Thank you for your bug report.

However, it is not a bug.

Malloc libraries on most of the operating systems tend to leave the memory chunks as part of the process that takes most of the memory. Hence, free() makes that pointer inaccessible to the program, but when new allocation is performed, malloc library does not have to make kernel calls in order to allocate memory. It simply uses this (hidden) cache attached to the process to make it available to that process again.

This is a very well known phenomenon on many operating systems , including Windows , which we use as well.

There is a way to reduce memory that is bound, but not allocated, to some process. See how much free memory is left on OS and then write and execute a simple C program then will calloc() and free() a memory, whose size  is larger then what OS reports as the available memory.

You will see that mysqld process size will be reduced. However, speed of execution will be reduced too, since then, for memory allocation, OS kernel will have to do the complex work of memory allocation.

This is a very well known phenomena on many operating systems.

Not a bug.

I did find this description of memory management (freed but still kept for future use), but it isn't the case here unfortunately. In the tests I did on the two customers installations, the application did create MEMORY tables using in total over 16 GB of the available 24 GB. So I started the application a first time, watched the memory usage raised, then stopped the application, waited for a while and tried to access all the memory tables to make sure that all had been really dropped (which was the case). Then I resarted the application and could watch the memory increasing till I had a crash of MySQL and had to reboot the server. On top of it, as I wrote in the description, once I did replace MySQL 8.0.33 with the version 8.0.24 the problem did not appear anymore.

Hi Mr. Duverney,

Thank you for your reply. 

However, we have not heard how did calloc() / free() work in your case with 8.0.33.

Regarding 8.0.33 using more memory, that is expected behaviour, since we added many more features.

If you consider that we have a memory leak, please provide us with the output from the corresponding tool, like ASAN or similar.

Do understand that this is a forum for reports with a fully repeatable test case and we have not got anything like that from you. We created and dropped large MEMORY tables without any problems.

Not a bug.