MySQL Bugs: #69082: Connector/NET introduces memory leak after an XmlSerializer has been instanced

Bug #69082	Connector/NET introduces memory leak after an XmlSerializer has been instanced
Submitted:	26 Apr 2013 11:36	Modified:	3 Jun 2013 6:17
Reporter:	Juergen Steinblock	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	Connector / NET	Severity:	S3 (Non-critical)
Version:	6.6.5.0	OS:	Windows (7 x64 / 2008 x64)
Assigned to:	Francisco Alberto Tirado Zavala	CPU Architecture:	Any
Tags:	xmlserializer memory-leak

Description:
Today I investigated down an Memory Leak in a console application that reads xml files and stores the result in a mysql database.

Both Windbg and Ants Memory Profiler showed large amounts of MySqlConnection / MySqlCommand / MySqlParameter objects.

I figured out that, after I create a XmlSerializer object, every query to the database leaks a little bit of memory, even if the XmlSerializer and MySql-Queries are totally unrelated.

Things get a little bit more wired. The same code with VB.NET has this issue but with C# not

How to repeat:
- Create a new Solution with Visual Studio 2010
- Add a new VB.Net Console application
- Modify Module1.vb to look like this

Imports System.Xml.Serialization
Imports MySql.Data.MySqlClient

Public Class Pool
End Class
Module Module1
    Sub Main()
        Dim serializer As New XmlSerializer(GetType(Pool))
        Do
            Dim result As Object = MySqlHelper.ExecuteScalar( _
                "Server=localhost;Port=13306;uid=root", "SELECT NOW()", Nothing)
            Console.WriteLine("{0} {1,10}", result, Environment.WorkingSet)
        Loop
    End Sub
End Module

- run the app
- this baby eats ~1MB Memory per second (at least in my configuration)
- comment out the line "Dim serializer As New XmlSerializer(GetType(Pool))"
- the app runs fine without a significant change in memory usage.

- Create another Console app, this time with C#
- modify Program.cs to look like this

using System;
using System.Xml.Serialization;
using MySql.Data.MySqlClient;
namespace ConsoleApplication1
{
    public class Pool { }
    class Program
    {
        static void Main(string[] args)
        {
            var serializer = new XmlSerializer(typeof(Pool));
            while(true)
            {
                var result = MySqlHelper.ExecuteScalar(
                    "Server=localhost;Port=13306;uid=root", "SELECT NOW()", null);
                Console.WriteLine("{0} {1,10}", result, Environment.WorkingSet);
            };
        }
    }
}

- run this app, this time I don't have a significant change in memory usage even with XmlSerializer

can be reproduced with
- MySql 5.1.54 on Windows 7 x64 and MySql 5.1.54 on Sever 2008 x64
- MySql.Data.dll 6.6.5 (via nuget) and 6.1.3
- Visual Studio 2010 / .NET 4.0
- the executed query does not matter, any other will do
- This is a very basic example. In the original setup the entry assembly is a VB.NET console application but the class that makes the XML deserialisation is written in C# and holds a static XmlSerializer instance
- If I change the code to execute 10000 queries, create a XmlSerializer and execute another 10000 queries memory starts leaking not until I created the serializer

Suggested fix:
Figure out why this happens and fix the memory leak

Hello.

The issue that you reported is not related with Connector/Net but it is a XmlSerializer bug related with the class constructor.
You can read something about it in the following link:
http://msdn.microsoft.com/en-us/magazine/cc163491.aspx#S4
You can find the info in the "Leaking Unmanaged Heap Memory" section of the link.

I did a little change to the VB code you posted (pass a XmlRootAttribute to XmlSerializer constructor), and the issue dissapear:

Imports System.Xml.Serialization
Imports MySql.Data.MySqlClient

Public Class Pool
End Class

Module MemoryLeakRevision

    Sub Main()
        Dim serializer As _
            New XmlSerializer(GetType(Pool), New XmlRootAttribute("Pool"))

        Dim counter As Integer = 1
        Do
            Dim result As Object = MySqlHelper.ExecuteScalar("Server=localhost;Port=3305;uid=root", "SELECT NOW()", Nothing)
            Console.WriteLine("{0} {1,10} | Counter:{2}", result, Environment.WorkingSet, counter)
            counter += 1
        Loop
    End Sub

End Module

Please see an image that I attached on files section where you can see the behaviour after the change I did. In the image you can appreciate that the counter loop is over 560k and the memory issue is not present.

Saludos and happy coding
Francisco Tirado

MySQL-VB Memory Leak Revision

Attachment: MySQL-VB Memory Leak Revision.jpg (image/jpeg, text), 293.13 KiB.

Hello Francisco,

I was aware about XmlSerializer containing a memory leak because I read some discussions about it before posting here.
But I thought they wouldn't apply because the are all like "XmlSerializer will create a dynamic assembly at runtime and load it in your app domain everytime you create a new instance" and I only create one instance and never really use the serializer at runtime.

I verified that the code change (use the overload) works and no memory is leaking anymore.

According to the link you provided, this seems to be the most plausible cause (Somewhere a finalizer does not complete and prevents other finalizers from running their cleanup code).

> If that misbehaving finalizer erroneously tries over and over again
> to access the database, never returning, the "well-behaved" finalizer will
> never get a chance to run.

Basis on the documentation some constructors has the problem with the memory leak but not all, the issue will depends on the constructor used besides the class attributes configuration. 

After some research and try to get more information about XmlSerializer best practices or known issues, I found the following information for troubleshooting:
http://msdn.microsoft.com/en-us/library/aa302290.aspx

I tried to figure out if there is a relation with connections to DB's and XmlSerializer, but I had no luck with that.