MySQL Bugs: #106886: ODBC driver memory leak

Bug #106886	ODBC driver memory leak
Submitted:	1 Apr 2022 9:00	Modified:	9 Jun 2022 21:40
Reporter:	c t	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Connector / ODBC	Severity:	S3 (Non-critical)
Version:	8.0.28	OS:	Windows (Windows 10 Enterprise Edition 64Bit )
Assigned to:		CPU Architecture:	Any
Tags:	ODBC 8

Description:
the ODBC driver dll leaks memory each time when a ODBC data source gets connected and disconnected (and no other ODBC connection is connected, so the Driver dll gets loaded during connect and unloaded during disconnect)

already existing bug which is not processed anymore (i guess cause of status)
https://bugs.mysql.com/bug.php?id=93593

ODBC Driver used:
mysql-connector-odbc-noinstall-8.0.28-win32

OS:
Edition	Windows 10 Enterprise
Version	20H2
Installed on	‎25.‎06.‎2021
OS build	19042.1586
Experience	Windows Feature Experience Pack 120.2212.4170.0

How to repeat:
// NOTE: build using VS2017, Use Multi-Byte Character Set

// tested with latest ODBC driver
// see output for unload/load
// 'memleak_test.exe' (Win32) : Unloaded 'C:\mysql-connector-odbc-noinstall-8.0.28-win32\lib\myodbc8w.dll'
// 'memleak_test.exe' (Win32) : Loaded 'C:\mysql-connector-odbc-noinstall-8.0.28-win32\lib\myodbc8w.dll'.Symbols loaded.

// REDUCED_OUTPUT can be used to prevent non error output of results of various odbc functions
#define REDUCED_OUTPUT 0
// ODBC_DATASOURCE is the ODBC connection used, configured to MySQL ODBC Unicode driver
#define ODBC_DATASOURCE "test_mycustomer"

#include <iostream>
#include <windows.h>
#include <sqltypes.h>
#include <sqlext.h>
#include <psapi.h>

#pragma comment(lib, "odbc32.lib")

class MemleakTest
{
public:
	MemleakTest(const std::string &ds)
		: odbcDataSource(ds)
	{}

	~MemleakTest()
	{
		disconnect();
	}

	bool connect()
	{
		const auto envAllocResult = SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &hEnv);
#if !REDUCED_OUTPUT
		std::cout << "envAllocResult: " << envAllocResult << std::endl;
#endif
		if (SQL_SUCCEEDED(envAllocResult))
		{
			const auto setEnvAttrResult = SQLSetEnvAttr(hEnv, SQL_ATTR_ODBC_VERSION, (void*)SQL_OV_ODBC3, 0);
#if !REDUCED_OUTPUT
			std::cout << "setEnvAttrResult: " << setEnvAttrResult << std::endl;
#endif
			if (SQL_SUCCEEDED(envAllocResult))
			{
				const auto hdbcAllocResult = SQLAllocHandle(SQL_HANDLE_DBC, hEnv, &hDbc);
#if !REDUCED_OUTPUT
				std::cout << "hdbcAllocResult: " << hdbcAllocResult << std::endl;
#endif
				if (SQL_SUCCEEDED(hdbcAllocResult))
				{
					const auto setConnAttrResult = SQLSetConnectAttr(hDbc, SQL_LOGIN_TIMEOUT, (void*)5, 0);
#if !REDUCED_OUTPUT
					std::cout << "setConnAttrResult: " << setConnAttrResult << std::endl;
#endif
					if (SQL_SUCCEEDED(setConnAttrResult))
					{
						// Connect to data source //
						const auto connectResult = SQLConnect(hDbc, (SQLCHAR*)odbcDataSource.c_str(), SQL_NTS, (SQLCHAR*) "", SQL_NTS, (SQLCHAR*) "", SQL_NTS);
#if !REDUCED_OUTPUT
						std::cout << "connectResult: " << connectResult << std::endl;
#endif
						if (SQL_SUCCEEDED(connectResult))
						{
#if !REDUCED_OUTPUT
							std::cout << "CONNECTED" << std::endl;
#endif
							return true;
						}
						else
						{
							std::cerr << "SQLConnect failed: " << connectResult << std::endl;
						}
					}
					else
					{
						std::cerr << "SQLSetConnectAttr failed: " << setConnAttrResult << std::endl;
					}

					FreeHandle(SQL_HANDLE_DBC, hDbc);
				}
				else
				{
					std::cerr << "SQLAllocHandle hDbc failed: " << hdbcAllocResult << std::endl;
				}
			}
			else
			{
				std::cerr << "SQLSetEnvAttr failed: " << setEnvAttrResult << std::endl;
			}

			FreeHandle(SQL_HANDLE_ENV, hEnv);
		}
		else
		{
			std::cerr << "SQLAllocHandle HENV failed: " << envAllocResult << std::endl;
		}

		return false;
	}

	void disconnect()
	{
		if (hDbc)
		{
			const auto disconnectResult = SQLDisconnect(hDbc);
#if !REDUCED_OUTPUT
			std::cout << "disconnectResult: " << disconnectResult << std::endl;
#endif
			if (!SQL_SUCCEEDED(disconnectResult))
			{
				std::cerr << "SQLDisconnect failed: " << disconnectResult << std::endl;
			}

			FreeHandle(SQL_HANDLE_DBC, hDbc);
		}
		if (hEnv)
		{
			FreeHandle(SQL_HANDLE_ENV, hEnv);
		}

#if !REDUCED_OUTPUT
		std::cout << "DISCONNECTED" << std::endl;
#endif
	}

private:
	std::string odbcDataSource;
	SQLHANDLE hEnv = nullptr;
	SQLHANDLE hDbc = nullptr;

	bool FreeHandle(SQLSMALLINT type, SQLHANDLE &handle)
	{
		if (nullptr == handle)
		{
			std::cerr << "invalid handle given" << std::endl;
			return false;
		}

		const auto freeHandleResult = SQLFreeHandle(type, handle);
		if (!SQL_SUCCEEDED(freeHandleResult))
		{
			std::cerr << "SQLFreeHandle(type " << type << ", handle " << handle << ") failed: " << freeHandleResult << std::endl;
			return false;
		}
		handle = nullptr;
		return true;
	}
};

void printMemory()
{
	const HANDLE hProcess = GetCurrentProcess();
	PROCESS_MEMORY_COUNTERS pmc;

	if (GetProcessMemoryInfo(hProcess, &pmc, sizeof(pmc)))
	{
		printf("\tWorkingSetSize: 0x%08X\n", pmc.WorkingSetSize);
	}

	CloseHandle(hProcess);
}

int main()
{
	std::cout << "***START ";
	printMemory();

	for (uint32_t counter = 0; counter < 500; ++counter)
	{
		MemleakTest memleakTest(ODBC_DATASOURCE);
		memleakTest.connect();
		memleakTest.disconnect();

		std::cout << "#" << counter;
		printMemory();

		Sleep(500);		// dont let firewall/server think its DDoS attack
	}

	std::cout << "***END ";
	printMemory();

	return 0;
}

Verified with the version 8.0.29.

Each new Connect/Disconnect cycle adds to the working set size about 0x1000.
This can only be observed when on each iteration ENV handle is allocated/freed.
If ENV is allocated once and reused for each new connection the working set size stabilizes after a few iterations.

verified, holding ENV handle alive will cause no memory leak. after ~15 iteration.
result of keeping ENV alive is that the driver is not unloaded, thus not loaded anymore and therefore not allocating (additional) memory.

possible WorkAround is clear: keeping ENV handle somehow statically, for ex. as singleton and reuse when creating connections).

unfortunately we have a wrapper class for the whole ODBC DB access, this includes also SQLAllocHandle/SQLFreeHandle the ENV handle. (very old legacy code)

will there be a fix for the memory consumption in case SQLAllocHandle/SQLFreeHandle for ENV is allocated each time a connection is created?

Posted by developer:
 
The memory leak was caused by wrong initialization counter, which did not allow calling of mysql_library_end() upon DLL_PROCESS_DETACH event.

Posted by developer:
 
The fix for the problem with mysqlclient library uninitialization is pushed to the source tree.

Posted by developer:
 
Another problem spotted when investigating this bug.
It was reported as a separate ticket and will have a separate fix:

https://bugs.mysql.com/bug.php?id=107328

Posted by developer:
 
Fixed as of the upcoming MySQL Connector/ODBC 8.0.30 release, and here's the proposed changelog entry from the documentation team:

Fixed memory leak caused by ODBC data source reconnects; now
mysql_library_end() is called upon the DLL_PROCESS_DETCH event.

The workaround is to reuse ENV for each new connection.

Thank you for the bug report.