MySQL Bugs: #93019: heap allocation leak in myodbc8w

Bug #93019	heap allocation leak in myodbc8w
Submitted:	30 Oct 2018 20:05	Modified:	28 May 2020 9:16
Reporter:	Gary Brookman	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	Connector / ODBC	Severity:	S1 (Critical)
Version:	8.0.13	OS:	Windows
Assigned to:		CPU Architecture:	Any
Tags:	heap, ODBC

Description:
two critical sections are handled incorrectly and there is a heap corruption.

How to repeat:
create a 32bit Windows console app
add these 2 lines:
	HWND hWndParent = GetConsoleWindow ();
	BOOL ret = SQLManageDataSources (hWndParent);
set Appverif to the EXE using default settings
run the EXE from WinDBG
from the GUI pick a system data source using the MySQL ODBC 8.0 Unicode driver
click the drop down arrow in the Database control.
  * You'll see two critical sections left active:
   - myodbc8w!THR_LOCK_charset+0x0 
   - myodbc8w!LOCK_load_client_plugin+0x0 
click the Cancel button.
  * you'll see many instances of "A heap allocation was leaked."

very repeatable.
In this sample app the corruption doesn't cause a problem if run outside a debugger. In my company's app the heap corruption causes an access violation.
I assigned a severity of S1 as the only workaround is to use a different version of the driver. That is outside of my control.

Thank you for the bug report. Please attach using the Files tab the client code file test case. Thanks.

Hi Gary,

The scenario you have described might be valid, but I find it confusing that the problem manifesting itself as a leak causes the corruption.

Leak and corruption are two very different things. A leak is a failure to free a resource, so it stays unavailable even after it is not needed anymore. A corruption is writing something in the wrong area of memory (like buffer overrun, etc).
Your debugger showed a leak, not a corruption.

We are aware of the issue with de-initializing of libmysqlclient, which keeps internal allocations for the whole time when ODBC driver is loaded. This might be the case even after ODBC connection is closed (ODBC driver still might be loaded).

Unfortunately, for ODBC application is impossible to tell when exactly libmysql internals must be deallocated. Due to libmysql design, if it is uninitialized the second initialization is not possible and any attempts to do it again cause the crash. Therefore, the driver makes no attempts to perform unititialization and keeps libmysql active at all times. The debuggers or leak detection tools usually spot this situation, but it does not cause much harm except the scenario when application is running for a very long time and internal allocations accumulate to a very large amout, which very rarely happens.

After the ODBC driver is unloaded the resources are forcefully freed by OS (not a very graceful way of doing it, but that's the only available alternative at the moment). Server developers are working on improving libmysql design to avoid the need to keep it inited at all times.

Now, about the corruption. So far I could not see how the above leak could cause the memory corruption. We would appreciate you providing more relevant information of how the corruption happens (perhaps stack trace, memory dump), details of how your application is operating (such as invoking the ODBC GUI prompt to let user enter the connection credentials, running multiple threads that share an ODBC connection or a statement structures). Most helpful would be the test program, which repeats the corruption problem (not only the leak).

We appreciate your cooperation and help.

Bogdan,

Thanks for the quick response. Sorry, my bad, this is a mature app that's been running against many OBDC drivers, including older versions of yours so when I saw the leak I jumped to conclusions. I can't upload the crash dump, it's too large, so I provided the results of a !analyze -v from the dump. 
Did the driver change so that it prematurely unloads? Looks like there's a pointer to the unloaded myodbc8a.dll.
BUCKET_ID:  APPLICATION_FAULT_NULL_CLASS_PTR_WRITE_INVALID_POINTER_WRITE_myodbc8a!SQLGetTypeInfo+53bc7

Note: fails with ANSI or Unicode driver.

This is repeatable. Our product is available with a 2 hour demo and the install is easy. I can crash it a couple of minutes. I can provide directions via text or a call.

If the DLL is unloading too soon I wonder if this article applies: 
https://blogs.msmvps.com/vandooren/2006/10/09/preventing-a-dll-from-being-unloaded-by-the-...

Thanks for looking,
Gary

Hi Gary,

Thank you for providing the additional details.
The crash occurs in SQLGetTypeInfo() function, but I doubt it has any relation to the leaks mentioned before.

I need to conduct some more tests before coming back with the results.

Hi Gary,

Unfortunately, we were not able to reproduce the crash on the test machines.
However, the crash was still possible as accumulation from a particular sequence of calls that leads to buffer overflow in SQLGetTypeInfo() function.

This has been fixed, but as I already mentioned there is no way to verify if the original problem is actually addressed.
I can attach a debug build of the patched ODBC driver for you to try. So, we will know for sure whether the patch fixes the original problem or not.

If I understand things correctly, you were using the 32-bit UNICODE version of MySQL ODBC Driver 8.0.13.
Is that correct? Please confirm.

One more thing: the debug build I am going to provide is only for the testing purposes. It is not intended to be used in a production environment.

Hi Bogdan,

Yes 32 bit, but Unicode and ANSI failed.

I can test the debug build. It readily and repeatedly crashes in my tests.

Thanks,
Gary

ODBC Driver Test build

Attachment: mysql-connector-odbc-noinstall-8.0.14-win32.7z (application/octet-stream, text), 2.74 MiB.

Hi Gary,

I attached a 7z archive with the test debug build of ODBC Driver 8.0 32-bit.
Unfortunately, the ZIP compression was not good enough to fit into the 3MB limit for attached files. You will need to use the free 7-zip program to unpack it.

You do not need to uninstall the current version of ODBC driver or replace it with the debug build. The best option is to add a debug driver separately.

Here are the steps of doing it:

1. Unpack the contents of the archive somewhere (I will use C:\TEMP)
2. Open a windows command line (press on the keyboard combinaion
Win+R and run

cmd.exe

3. In the command line run the following commands to enter the driver bin
directory:

C:
cd \TEMP\mysql-connector-odbc-noinstall-8.0.14-win32\bin

4. Now you need to register the driver by executing the following command line command (when you copy from the browser make sure it is one line, not wrapped as two or more):

myodbc-installer.exe -d -a -n "MyODBC 8.0 ANSI Debug" -t "DRIVER=C:\TEMP\mysql-connector-odbc-noinstall-8.0.14-win32\lib\myodbc8a.dll;SETUP=C:\TEMP\mysql-connector-odbc-noinstall-8.0.14-win32\lib\myodbc8S.dll"

The correct output will be as follows:
Success: Usage count is 1

5. Run a 32-bit version of ODBC Administrator
(better to do it using the full path because the default ODBC Admin is
64-bit). Press Win+R and execute

%windir%\SysWOW64\odbcad32.exe

6. You should be able to see the newly registered driver in the "Drivers" tab.

7. Select the "System" tab and click "Add..." button then select the driver
"MyODBC 8.0 ANSI Debug" and configure it in the usual way.

If you need to use the DSN-less connection the connection string DRIVER option
should be as this: DRIVER={MyODBC 8.0 ANSI Debug};

If the crash happens again the .pdb files in the lib directory contain the symbol information for the debugger, so it will point to the problem in the code in a more precise way.

I am looking forward to hearing from you.
Thanks for your help with this.

Hi Bogdan,

I tested with the debug DLL, it still crashes, but myodbc8a is unloaded by the time the crash happens so I can't see into your dll.

From my perspective there appear to be two separate problems. In the MySQL Connector/ODBC Data Source Configuration dialog I don't see a list of available databases to connect to, and after leaving the dialog my app crashes. In this scenario we aren't calling your DLL directly, but calling SQLManageDataSources. 

To determine if the unloading of myodbc8a is the cause of the crash I added these lines to our ODBC Client Driver:
    HMODULE hm = LoadLibraryExA ("F:\\cases\\14614892(DataLogger)\\mysql-connector-odbc-noinstall-8.0.14-win32\\lib\\myodbc8a.dll",
                                 NULL, NULL);
    if (hm > 0)
        {
        HMODULE* phModule (0);
        GetModuleHandleExA (GET_MODULE_HANDLE_EX_FLAG_PIN, "myodbc8a.dll", phModule);
        }
That keeps it loaded until the EXE ends, and no more crash.

From WinDBG, when it is crashing, with the debug DLL I see this:
ExceptionAddress: 08f94de0 (myodbc8a!my_thread_self)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000008
   Parameter[1]: 08f94de0
Attempt to execute non-executable address 08f94de0

and right before the unload message I see libeay32 (Open SLL) in the call stack. I don't have a PDB for that so don't know what's happening. 

Again I'll mention that our software has a 2 hour demo period available. The install is easy and I can give instructions to reproduce the issue or be available for a remote call to walk you through the crash. That way you could put a breakpoint in SQLGetTypeInfo.

Thanks for your efforts,
Gary

Please try version 8.0.20. Thanks.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".