MySQL Bugs: #93711: connection_cext has a memory leak in the python mysql-connector

Bug #93711	connection_cext has a memory leak in the python mysql-connector
Submitted:	21 Dec 2018 5:34	Modified:	27 Mar 2021 22:16
Reporter:	Alexander Stachniak	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	Connector / Python	Severity:	S1 (Critical)
Version:	8.0.11	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	memory leak

Description:
The "prepare_for_mysql" function in connection_cext.py builds a strong-reffed list or dict. These references are never deleted, creating a memory leak that persists even after closing and deleting the connection. In situations where large amounts of data need to be inserted in an iterative process, the memory leak can quickly fill the entire available RAM leading to system crash.

How to repeat:
Easy to repeat: create random data, loop through insert statements, and profile the memory allocation assigned to the connector class. Example: https://github.com/zstachniak/Memory-Leak-in-MySQL-Connector/blob/master/memory_leak.ipynb

Suggested fix:
Refactor prepare_for_mysql to use weak references, or perform cleanup after executing.

I stumbled across this bug yesterday and I can confirm the memory leak.

Also confirming this severe bug. We have a long running server and it increases memory until it crashes by OOM. tracemalloc diff snapshots shows lines:
connection_cext.py:536
connection_cext.py:532
connection_cext.py:153
connection_cext.py:276
connection_cext.py:289

Seemed to me like some self._cmysql call bug, so I have tried the connection with use_pure = True and the problem has gone away.

AFAIK such severe bugs should be fixed soon.

Thank you for the bug report. Please check version 8.0.23. Thanks.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

I am seeing a memory leak in version 8.4.0

/usr/local/lib/python3.9/dist-packages/mysql/connector/connection_cext.py:506: size=170 KiB, count=2650, average=66 B

/usr/local/lib/python3.9/dist-packages/mysql/connector/connection_cext.py:506: size=192 KiB, count=3081, average=64 B

/usr/local/lib/python3.9/dist-packages/mysql/connector/connection_cext.py:506: size=214 KiB, count=3511, average=62 B