MySQL Bugs: #91643: mysql-connector-python memory leaks

Bug #91643	mysql-connector-python memory leaks
Submitted:	15 Jul 2018 19:26	Modified:	26 Aug 2018 9:55
Reporter:	Ross Qin	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	Connector / Python	Severity:	S2 (Serious)
Version:	8.0.11	OS:	MacOS (Mac OS X Sierra & High Sierra)
Assigned to:	Assigned Account	CPU Architecture:	x86 (x86_64bit Mac)
Tags:	memory leak, mysql-connector-python

Description:
I have 100 CSV files with 50000 rows in each file and I want to insert them into MySQL using Python 3.6.x. I commit the insertions after parsing each file and expect the memory usage to decrease after the committing since the content of the file will not be used in the future. However, it doesn't. The memory usage keeps growing and will finally store all 100CSV files in the memory and blow up. So there seems to be a memory leak in the connection or the insertion.

How to repeat:
Database only has 1 table with 2 columns:

CREATE TABLE `table` (
  `Id` int(11) NOT NULL AUTO_INCREMENT,
  `Value` int(11) NOT NULL,
  PRIMARY KEY (`Id`)
)

Here is the python code:

import mysql.connector    
insert_sql = ("INSERT INTO table (Value) VALUES (%s)")    
for i in range(100):
    cnx = mysql.connector.connect(user='root', password='password', host='127.0.0.1', database='database')
    cursor = cnx.cursor()
    params = [(j,) for j in range(50000)]
    # If I don't excute the following insertion, the memory usage is stable.
    cnx.executemany(insert_sql, params)
    cnx.commit()
    cursor.close()
    del cursor
    cnx.close()
    del cnx
    print('Finished processing one file')    
print('All done')

The execute() function has the same problem too:

import mysql.connector
insert_sql = ("INSERT INTO table (Value) VALUES (%s)")
for i in range(100):
    cnx = mysql.connector.connect(user='root', password='password', host='127.0.0.1', database='database')
    cursor = cnx.cursor()
    # Insert 50000 rows here
    for j in range(50000):
        cursor.execute(insert_sql, (j,))
    cnx.commit()
    cursor.close()
    cnx.close()
    print('Finished processing one file')
print('All done')

Hello Ross Qin,

Thank you for the bug report.
I could not repeat the issue at our end using with Connector / Python 8.0.11 with MySQL 8.0.11 version.
Could you please provide repeatable steps (exact steps, using pure python implementation or the c-extension, create table statement , sample CVS file, stack trace etc. - please make it as private if you prefer) to confirm this issue at our end?

Thanks,
Chiranjeevi.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".