MySQL Bugs: #21385: Handle leak MyODBC 3.51.12 in Windows Server 2003 SP1

Bug #21385	Handle leak MyODBC 3.51.12 in Windows Server 2003 SP1
Submitted:	1 Aug 2006 12:13	Modified:	8 Mar 2007 1:58
Reporter:	Leandro Becker	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Connector / ODBC	Severity:	S1 (Critical)
Version:	3.51.12	OS:	Windows (Windows Server 2003 SP1)
Assigned to:	Bogdan Degtyariov	CPU Architecture:	Any

Description:
We have a product that support various DBMS, including My SQL 5. We have a test server running Windows Server 2003 SP1, MySQL 5.0.15nt, MFC 7.1 e MyODBC 3.51.12. After a few days running, the handle count for our application program is about 2511 handles. Each time the application perform database accesses, this number increases.

We have discarded bugs in our application because with another DBMS (postgreSQL, MS Sql Server and Oracle) this behaviour do not happen. So I've used a demo version of "Memory Validator" and it showed handles allocated inside myodbc driver dlls that was never deallocated.

How to repeat:
To repeat this behaviour, if you wish, I can send a demo version of our application (language is PT-BR) to help reproduce the problem.

Hi Leandro and thanks for your problem report.
Could you please provide small C/C++ test case showing this behavior? The fact that other drivers do not leak handles does not mean your application works correctly.

Will be very hard to me create a sample source code due the fact that is a very large application, and to simulate all steps will be hard. I will try code some source, but can´t you run our app in a debug version of the driver? Or can you send me a version of the driver compiled with debug information ? Compile the source of this driver is a pain due the fact of using libraries (GUI) that is not available anymore to be downloaded. If you send me a debug version of mysql odbc driver and the .pdbs and the full path where the source was put to build the odbc driver maybe I can find by myself what is happening and report to you asap.

Thx

Leandro, it is impossible for me to create a test case based on info provided. I have uploaded myodbc3-51-12-debug.win32-2006-02-17.zip to ftp.mysql.com/pub/mysql/upload. Unzip and replace myodbc3.dll in winsys32 dir with this version. Backup original library first! Mind you, debug version is very slow in normal work due to large quantity of debug data it provides. You'll get myodbc.log file in the root of C drive and can analyze further then.

Support on using our products is available, for a reasonable fee, direct from our skilled support engineers at http://www.mysql.com/support/

Thank you for your interest in MySQL.
Regards

Leandro, sorry, it's /download/ not /upload/.

Hi Tonci
Sorry, but the folder ftp://ftp.mysql.com/pub/mysql/download is empty :-(
Maybe removed before I have a chance to get it ?

Sorry, I´ve put the file name together with download path and now I'm able to download it. Maybe it's hidden ...
I will work on it, I will need 1 or 2 weeks because I'm very busy right now.
When I found something I'll post the results.

Thank you in advance

Sorry again :-(

The Visual C++ 7.1 could not find the symbols for the dll. Can you send me the .pdb file ?

Thx

Leandro, I know, you will allways miss something. Try running your app and use provided dll as driver. You will get enough info. If you still want pdb, it's in ftp://ftp.mysql.com/pub/mysql/hidden/connectors/odbc/myodbc3-pdb.zip
Please, inform me of result.

Hi Tonci

I think I found the BUG. After build myodbc e mysql libs with debug info, I found using Memory Validator that the leaked handle is a semaphore and a critical section allocated at myodbc::dll.c::LibMain line 144 (my_thread_init). After think about the problem (the leakage take sometime to begin), I see the code LibMain:

int APIENTRY LibMain(HANDLE hInst,DWORD ul_reason_being_called,
		     LPVOID lpReserved)
{
  switch (ul_reason_being_called) {
  case DLL_PROCESS_ATTACH:  /* case of libentry call in win 3.x */
  if (!inited++)
  {
    s_hModule=hInst;
    myodbc_init();
  }
  break;
  case DLL_THREAD_ATTACH:
    threads++;
#ifdef THREAD
    my_thread_init();
#endif
    break;
  case DLL_PROCESS_DETACH:  /* case of wep call in win 3.x */
    if (!--inited)
      myodbc_end();
    break;
  case DLL_THREAD_DETACH:
#ifdef THREAD
    if ( threads )
    {
      my_thread_end();	  /* Last will be freed in my_end() */
      --threads;
    }
#else
    --threads;
#endif
    break;
  default:
    break;
  } /* switch */

  return TRUE;

  UNREFERENCED_PARAMETER(lpReserved);
} /* LibMain */

The problem is:

When many threads are created before load the myodbc driver, there is no call with DLL_THREAD_ATTACH (dll not loaded). When driver gets loaded inside some thread created later, DLL_THREAD_ATTACH gets called and threads counter is increased by one. But Windows call DLL_THREAD_DETACH even for threads that no have issued any DLL_THREAD_ATTACH. So when some of the threads that was running before driver gets loaded exits, the Windows call DLL_THREAD_DETACH but there is not TLS values for this thread and my_thread_end do nothing, but the threads counter is decresed. If reaches 0, when the the thread that really are using the driver terminates, Windows will call DLL_THREAD_DETACH but the threads counter is 0 and tls data are never released.

I think that a solution is make my_thread_end return FALSE if there is no tls index associated with this thread and if this happens, do not decrement threads counter. Maybe the same for increment thread counter. Or maybe remove the counter (I don't know the impact for this).

I don't know if I could explain the problem (sorry my english), I will happy to help and provide a test case. Steps for the test case:

1. Create a thread that perform Sleep(5000) and returns 0;
2. Sleep(1000);
2. Connect to database (myodbc3.dll) will be loaded;
3. Create another thread instance that perform Sleep(5000) and returns 0;
4. Perform a Sleep(20000);
5. Close database.

Thread 1 exits first and decrement threads counter. When thread 2 exits, the Tls values are not freed.

Best Regards

Leandro

Leandro, thanks for your input.

Leandro, I can't reproduce this behavior but I'm not sure I got everything right. Can you please make small test case exhibiting this behavior, in any variant of C, trying to follow your application logic or last gudelines you posted?

Test case to help reproduce the bug

Attachment: MySQL.zip (application/x-zip-compressed, text), 12.56 KiB.

I've done a small test case in VC++ 7.1.
I think that the source comments will help you.

Leandro thanks. Checking.

Leandro, delayed due to other tasks. Can you please check again with 3.51.14 and return with results?

No problem.

I'll install driver on our test server and monitor handle usage.
After testing, I'll post the result.

Thank you

Sorry, but I'm a little newby in MySQL community. Where can I get 3.51.14 driver?

Hi Leandro. 
Here it is:
ftp.mysql.com/pub/mysql/download/myodbc-3.51.14/mysql-connector-odbc-3.51.14-win32.msi

Thank you. 
I´ve installed it. 

But, if the source code that is in the ftp is the source for 3.51.14, probably the bug still there because the source code for LibMain is the same that I've checked in version 3.51.12 and had posted in this thread.

Regards

Leandro

I´ve tested the driver against my test case and the bug was not solved.
Do you wanna that I make a correction and submit to you aproval?

Leandro,

There's definitely interest from my side. You'd have to submit them under
our contributor license agreement, so I'd go read up on http://forge.mysql.com/wiki/MySQL_Contributor_License_Agreement to make sure
you're able/willing to do it (it's pretty stock, but it is the littlest bit of legal red tape we're able to get away with).

In forge I don't see odbc project. It is in there?

Leandro, create yourself an account at MySQL Contributor License Agreement page and just upload the file. Notify me here of the link. Or, you can post the patch here privately. I will make sure to push it for review.

I'm trying to upload, but I always get the error

".c" is not a recommended image file format.
from the upload system.

How upload a c source file?

Leandro, thanks for your efforts to help us. Can you submit patch in this report (private/public) or mail it directly to my e-mail address? We are looking forward to review it.

I've uploaded it. See comments in upload.

Leandro, ofcourse. I'll discuss patch with colleagues and get back to you.

Leandro, we're a bit tied-up right now. I'll inform you on progress.

Hi
How long you think that this correction can take?

Leandro, I think it's in review but will ask for actual status and get back to you.

Leandro, thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Explanation: Handle leak patch is available in revision 142. The patch is taken from the server code and is actually very short. You are noted in ChangeLog (rev. 143) as agreed.
Code:
    /* Main thread will free by my_end() */
    threads--;
    if (main_thread != GetCurrentThreadId())
      my_thread_end();
ChangeLog:
-- BUG#21385: fixed handle leak in LibMain (dll.c), used the same code as in the server part. Thanks to Mr. Leandro Becker for bringing this to our attention.

Ok.

I've made a little workaround in my product that consists in call
LoadLibrary(_T("myodbc3.dll"));
at beginning of execution. So dll is kept loaded all the time and the bug do not appear since all threads creation and termination will be caught by myodbc3.dll DLL_THREAD_ATTACH and DLL_THREAD_DETACH.

When odbc release arrives, I'll remove the workaround.

Thank you Tonci.

This is fixed in the repository and will be in the next release, as noted earlier.

I have faced to the same problem with handle leakage in MyODBC driver on Windows Server 2003 SP1. Are there any possibilities to download the patched binary from repository? Could I get the sources to compile them on my system? Does the patch affect to the ODBC connector only, or it affects to the server part too? When should I expect to download new release?

Thank you.

Regards,
Max Peters

Hello,

I was reading through this thread, and we have been seeing the same behavior, a semaphore handle leak in the MySQL ODBC driver, in our application as well.

Our application runs as a service and connects/disconnects from the MySQL database frequently as well.

Our application has the ability to talk to Microsoft SQL Server databases in addition to MySQL databases as well, and we are seeing this exact same problem when we configure our application to log data to both MSSQL and MySQL.

I can try to come up with a test application that can reproduce this bug, thus far I have only seen it in our app, even with the latest 3.51.16 build of the ODBC driver.

Thanks,
Ingmar.

Ingmar, thanks for info provided. Bogdan, Jim, can any of you recheck on:
  * Fixed handle leak in LibMain (Bug #21385, thanks to Leandro Becker) (3.51.14)?

Bug#27402 was marked as duplicate of this one even though it might be more related to Bug#14283: Handle leak when Sleep() under multithread, marked as !Bg
by Miguel.

Bug#15366 was marked as duplicate of this one.