Bug #953 memory leak calling libmysql and dep. lib. (ODBC , ADO)
Submitted: 30 Jul 2003 5:06 Modified: 4 Aug 2003 18:39
Reporter: khoa do Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:4.0.13-nt OS:Windows (Windows 2000)
Assigned to: MySQL Verification Team CPU Architecture:Any

[30 Jul 2003 5:06] khoa do
Description:
I have had memory leaks calling libmysql and other dependent libraries.  As a result, my COM and ASP applications which are dependent on a MySQL backend do not function properly after a few days because increased memory usage crashes the ASP runtime engine.  

How to repeat:
dbrule.zip (SQL scripts for the db)

1. ADO+MySQL ODBC driver 3.51 and C++
     - noticed increased memory usage, handle counts, great increase in page faults in clients that made repeated "select" calls
     - when I switched to MSSQL OLEDB and Oracle ODBC drivers the memory usage, handle counts, and page faults stabilized.
ex. rulecom.zip (com component), testrulecom.zip (test client)

2. ODBC+MySQL ODBC driver 3.51 and C++
    - noticed increased memory usage, great increase in page faults in clients that made repeated "select" calls
ex. dbupdate1.zip (ODBC test client)

3. Native+libmysql and C
    - noticed great increased memory usage, great increase in page faults in clients that made repeated "select" calls
    - I thought that using native calls would fix my memory leak issue but it turned out that I may have been experiencing the memory leak right in the libmysql client library
ex. libmysqltest.zip (a slight modification of libmysqltest to make repeated calls)
   

Suggested fix:
I have looked at the source for libmysql and without looking too much further and without understanding the architecture I have made the following observations:
1. "free_old_query" cannot properly free "mysql->field_alloc" because "mysql->fields == NULL" while "unpack_fields" seems to be responsible for the memory allocation.  Again, I don't understand enough of the architecture to understand what this means.
[30 Jul 2003 5:06] khoa do
sql scripts

Attachment: dbrule.zip (application/x-zip-compressed, text), 3.20 KiB.

[30 Jul 2003 5:11] khoa do
rulecom com component, ADO component data layer

Attachment: rulecom.zip (application/x-zip-compressed, text), 105.75 KiB.

[30 Jul 2003 5:11] khoa do
test client for rulecom component

Attachment: testrulecom.zip (application/x-zip-compressed, text), 3.98 KiB.

[30 Jul 2003 5:12] khoa do
ODBC client

Attachment: dbupdate1.zip (application/x-zip-compressed, text), 5.67 KiB.

[30 Jul 2003 5:14] khoa do
libmysqltest test client with slight modifications

Attachment: libmysqltest.zip (application/x-zip-compressed, text), 22.75 KiB.

[30 Jul 2003 5:44] khoa do
All projects files opened and save with VS.NET 2003.
[30 Jul 2003 9:07] MySQL Verification Team
The *zip files uploaded I got them corrupted. Could you please
download them again at:

ftp://support.mysql.com:/pub/mysql/secret
[30 Jul 2003 12:15] khoa do
At Miguel Solorzano request, I have uploaded the files to ftp://support.mysql.com:/pub/mysql/secret
[4 Aug 2003 6:39] MySQL Verification Team
Thank you, now I was able for to get the files and begin
the tests.
[4 Aug 2003 10:31] MySQL Verification Team
Could you please verify if the below support article can be applied
in your Win2k machine:

280790 - Memory Manager Allocates Paged Pool Before it Is Needed 
http://support.microsoft.com/?kbid=280790
[4 Aug 2003 11:29] khoa do
Yes, the fix http://support.microsoft.com/?kbid=280790 is on the server.  The server has service pack 3 which contains the hotfix.  However, I will apply service pack 4 to check.  Please stand by.
[4 Aug 2003 15:03] khoa do
I have applied servicepack 4 which should have all the latest hotfixes and including the one described earlier and I am still getting the memory leak. :-(
[4 Aug 2003 18:39] MySQL Verification Team
For practical reasons I began to test for a while your third case reported:

*******************************************************************************
3. Native+libmysql and C - noticed great increased memory usage, great increase in page faults in clients that made repeated "select" calls - I thought that using native calls would fix my memory leak issue but it turned out that I may have been experiencing the memory leak right in the libmysql client library
******************************************************************************

I modified your code of MyTest for to match the connection with my environment, performed two tests with the MySQL dynamic and static libraries.

Create a Purify session and create a Performance log with the Windows System
Performance Monitor with the below counters:

Memory -> Available bytes

Process -> Handle Count, Pool nonpaged bytes, pool paged bytes and private
bytes.

At the end of my test, Purify has reported:

[I] Summary of all memory leaks... {0 bytes, 0 blocks}
[I] Summary of new memory leaks... {0 bytes, 0 blocks}
[I] Program terminated at 04/08/2003 21:51:26

and the counters's behavior on the Performance Monitor are confirmed
the result I got from Purify.

So I am assuming that some environment issues (hardware, applications
installed, code of your application, etc) are the cause of your problem.