Bug #37226 Explicit call of my_thread_init() on Windows for every new thread
Submitted: 5 Jun 2008 14:41 Modified: 1 Oct 2008 17:02
Reporter: Andrey Hristov Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: C API (client library) Severity:S1 (Critical)
Version:5.0.22+ OS:Microsoft Windows
Assigned to: Vladislav Vaintroub
Triage: D3 (Medium) / R4 (High) / E2 (Low)

[5 Jun 2008 14:41] Andrey Hristov
Description:
In dll.c, there is the following code:
  switch (ul_reason_being_called) {
  case DLL_PROCESS_ATTACH:      /* case of libentry call in win 3.x */
    if (!inited++)
    {
      s_hModule=hInst;
      libmysql_init();
      main_thread=GetCurrentThreadId();
    }
    break;
  case DLL_THREAD_ATTACH:
    threads++;
    my_thread_init();
    break;
  case DLL_PROCESS_DETACH:      /* case of wep call in win 3.x */
     if (!--inited)             /* Safety */
     {
       /* my_thread_init() */   /* This may give extra safety */
       my_end(0);
     }
    break;
  case DLL_THREAD_DETACH:
    /* Main thread will free by my_end() */
    threads--;
    if (main_thread != GetCurrentThreadId())
      my_thread_end();
    break;
  default:
    break;
  } /* switch */

For convinience of Windows applications that use libmysql, my_thread_init() is called implicitly. This is only on Windows. However, if libmysql is loaded with LoadLibrary() and unloaded with FreeLibrary() later and in the mean time few threads have been created but still working, FreeLibrary() will force a 5 Sec. delay in my_thr_init.c :
  pthread_mutex_lock(&THR_LOCK_threads);
  while (THR_thread_count > 0)
  {
    int error= pthread_cond_timedwait(&THR_COND_threads, &THR_LOCK_threads,
                                      &abstime);
    if (error == ETIMEDOUT || error == ETIME)

Because of the convenience, my_thread_init() was called in threads which never use libmysql.
In addition, the fine documentation of mysql_thread_init() reads:
"This function must be called early within each created thread to initialize thread-specific variables. However, you may not necessarily need to invoke it explicitly: mysql_thread_init() is automatically called by my_init(), which itself is automatically called by mysql_init(), mysql_library_init(), mysql_server_init(), and mysql_connect(). If you invoke any of those functions, mysql_thread_init() will be called for you."

One can say that this is not much of a problem. That the DLL rarely is loaded with LoadLibrary and the application should be smart enough to know what is going on. This could be the case but there is one exception - PHP. PHP, on Windows, uses dlls for its extensions. When a MySQL extension is loaded libmysql is also loaded and for every thread from this point my_thread_init() will be called. However, it could be the case that the web server or PHP itself (like in the case we found this bug) creates threads that outlive the MySQL extensions. The MySQL extension is unloaded, my_end(0) is called. libmysql counts the thread number, since about 5.0.26, a change by Monty on Nov 30rd 2006. Because the counter was incremented in the still living thread(s), and not decremented, my_end(0) will wait. Which means that PHP will wait. If you use even non-threaded server, like apache1 or a server in CGI mode, the script will need 5 seconds to finish.
This is a VERY serious problem. Because of it lot of Windows PHP developers downgrade their libmysql to what came with PHP 5.2.1 - libmysql 5.0.21, which is pretty, pretty old. Maybe there are other problems, because memory is not freed, but the process is ending anyway.

How to repeat:
See above

Suggested fix:
Find a solution, maybe with an ENV variable on Windows, to skip the call to my_thread_init() in dll.c . If generally changed it might break applications, but with an ENV variable the fix can cure other applications.
[19 Aug 2008 15:47] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51978

2666 Vladislav Vaintroub	2008-08-19
      his patch fixes
      Bug#37226 Explicit call of my_thread_init() on Windows for every new thread.
      Bug#33031 app linked to libmysql.lib crash if run as service in vista under 
      localsystem
      Bug#38522 5 seconds delay when closing application using embedded server
      
      There are some problems using DllMain hook functions on Windows that 
      automatically do global and per-thread initialization for a DLL in MySQL context
      
      1)per-thread initialization(DLL_THREAD_ATTACH)
      MySQL internally counts number of active threads that and causes a delay in in 
      my_end() if not all threads are exited. But,there are threads that can be 
      started either by Windows internally (often in TCP/IP scenarios) or by user 
      himself - those threads are not necessarily using libmysql.dll functionality, 
      but nonetheless the contribute to the count of open threads.
      
      2)process-initialization (DLL_PROCESS_ATTACH)
      my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in 
      Windows loader.
      
      
      Fix is to remove dll initialization code from libmysql.dll in general case. I
      still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value 
      will cause the old behavior (DLL init hooks will be called). This env.variable 
      exists only to prevent breakage of existing Windows-only applications that 
      don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is 
      discouraged and it will be removed in 6.0
[26 Aug 2008 2:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52514

2666 Vladislav Vaintroub	2008-08-26 [merge]
      This patch fixes
      Bug#37226 Explicit call of my_thread_init() on Windows for every new thread.
      Bug#33031 app linked to libmysql.lib crash if run as service in vista under 
      localsystem
      
            
      There are some problems using DllMain hook functions on Windows that 
      automatically do global and per-thread initialization for libmysqld.dll
            
      1)per-thread initialization(DLL_THREAD_ATTACH)
      MySQL internally counts number of active threads that and causes a delay in in 
      my_end() if not all threads are exited. But,there are threads that can be 
      started either by Windows internally (often in TCP/IP scenarios) or by user 
      himself - those threads are not necessarily using libmysql.dll functionality, 
      but nonetheless the contribute to the count of open threads.
            
      2)process-initialization (DLL_PROCESS_ATTACH)
      my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in 
      Windows loader.
            
      Fix is to remove dll initialization code from libmysql.dll in general case. I
      still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value 
      will cause the old behavior (DLL init hooks will be called). This env.variable 
      exists only to prevent breakage of existing Windows-only applications that 
      don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is 
      discouraged and it will be removed in 6.0
[26 Aug 2008 11:31] Vladislav Vaintroub
Reviewed by jimw on IRC.
[1 Sep 2008 16:23] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53011

2674 Vladislav Vaintroub	2008-09-01
      This patch fixes
      Bug#37226 Explicit call of my_thread_init() on Windows for every new thread.
      Bug#33031 app linked to libmysql.lib crash if run as service in vista under 
      localsystem
        
      
      There are some problems using DllMain hook functions on Windows that 
      automatically do global and per-thread initialization for libmysqld.dll
      
      1)per-thread initialization(DLL_THREAD_ATTACH)
      MySQL internally counts number of active threads that and causes a delay in in 
      my_end() if not all threads are exited. But,there are threads that can be 
      started either by Windows internally (often in TCP/IP scenarios) or by user 
      himself - those threads are not necessarily using libmysql.dll functionality, 
      but nonetheless the contribute to the count of open threads.
      
      2)process-initialization (DLL_PROCESS_ATTACH)
      my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in 
      Windows loader.
      
      Fix is to remove dll initialization code from libmysql.dll in general case. I
      still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value 
      will cause the old behavior (DLL init hooks will be called). This env.variable 
      exists only to prevent breakage of existing Windows-only applications that 
      don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is 
      discouraged and it will be removed in 6.0
[1 Sep 2008 18:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53018

2729 Vladislav Vaintroub	2008-09-01
      Bug#37226 Explicit call of my_thread_init() on Windows for every new thread.
      Bug#33031 app linked to libmysql.lib crash if run as service in vista under 
      localsystem
        
      
      There are some problems using DllMain hook functions on Windows that 
      automatically do global and per-thread initialization for libmysqld.dll
      
      1)per-thread initialization(DLL_THREAD_ATTACH)
      MySQL internally counts number of active threads that and causes a delay in in 
      my_end() if not all threads are exited. But,there are threads that can be 
      started either by Windows internally (often in TCP/IP scenarios) or by user 
      himself - those threads are not necessarily using libmysql.dll functionality, 
      but nonetheless the contribute to the count of open threads.
      
      2)process-initialization (DLL_PROCESS_ATTACH)
      my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in 
      Windows loader.
      
      Fix is to remove dll initialization code from libmysql.dll in general case. I
      still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value 
      will cause the old behavior (DLL init hooks will be called). This env.variable 
      exists only to prevent breakage of existing Windows-only applications that 
      don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is 
      discouraged and it will be removed in 6.0
[1 Sep 2008 18:20] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53019

2729 Vladislav Vaintroub	2008-09-01
      Bug#37226 Explicit call of my_thread_init() on Windows for every new thread.
      Bug#33031 app linked to libmysql.lib crash if run as service in vista under 
      localsystem
        
      
      There are some problems using DllMain hook functions on Windows that 
      automatically do global and per-thread initialization for libmysqld.dll
      
      1)per-thread initialization(DLL_THREAD_ATTACH)
      MySQL internally counts number of active threads that and causes a delay in in 
      my_end() if not all threads are exited. But,there are threads that can be 
      started either by Windows internally (often in TCP/IP scenarios) or by user 
      himself - those threads are not necessarily using libmysql.dll functionality, 
      but nonetheless the contribute to the count of open threads.
      
      2)process-initialization (DLL_PROCESS_ATTACH)
      my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in 
      Windows loader.
      
      Fix is to remove dll initialization code from libmysql.dll in general case. I
      still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value 
      will cause the old behavior (DLL init hooks will be called). This env.variable 
      exists only to prevent breakage of existing Windows-only applications that 
      don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is 
      discouraged and it will be removed in 6.0
[1 Sep 2008 21:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53026

2675 Vladislav Vaintroub	2008-09-01
      Bug#37226 Explicit call of my_thread_init() on Windows for every new thread.
      Bug#33031 app linked to libmysql.lib crash if run as service in vista under 
      localsystem
        
      
      There are some problems using DllMain hook functions on Windows that 
      automatically do global and per-thread initialization for libmysqld.dll
      
      1)per-thread initialization(DLL_THREAD_ATTACH)
      MySQL internally counts number of active threads that and causes a delay in in 
      my_end() if not all threads are exited. But,there are threads that can be 
      started either by Windows internally (often in TCP/IP scenarios) or by user 
      himself - those threads are not necessarily using libmysql.dll functionality, 
      but nonetheless the contribute to the count of open threads.
      
      2)process-initialization (DLL_PROCESS_ATTACH)
      my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in 
      Windows loader.
      
      Fix is to remove dll initialization code from libmysql.dll in general case. I
      still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value 
      will cause the old behavior (DLL init hooks will be called). This env.variable 
      exists only to prevent breakage of existing Windows-only applications that 
      don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is 
      discouraged and it will be removed in 6.0
[1 Sep 2008 22:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53031

2675 Vladislav Vaintroub	2008-09-01
      Bug#37226 Explicit call of my_thread_init() on Windows for every new thread.
      Bug#33031 app linked to libmysql.lib crash if run as service in vista under 
      localsystem
        
      
      There are some problems using DllMain hook functions on Windows that 
      automatically do global and per-thread initialization for libmysqld.dll
      
      1)per-thread initialization(DLL_THREAD_ATTACH)
      MySQL internally counts number of active threads that and causes a delay in in 
      my_end() if not all threads are exited. But,there are threads that can be 
      started either by Windows internally (often in TCP/IP scenarios) or by user 
      himself - those threads are not necessarily using libmysql.dll functionality, 
      but nonetheless the contribute to the count of open threads.
      
      2)process-initialization (DLL_PROCESS_ATTACH)
      my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in 
      Windows loader.
      
      Fix is to remove dll initialization code from libmysql.dll in general case. I
      still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value 
      will cause the old behavior (DLL init hooks will be called). This env.variable 
      exists only to prevent breakage of existing Windows-only applications that 
      don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is 
      discouraged and it will be removed in 6.0
[13 Sep 2008 20:48] Bugs System
Pushed into 6.0.7-alpha  (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:hakan@mysql.com-20080725175322-8wgujj5xuzrjz3ke) (pib:3)
[15 Sep 2008 8:11] Bugs System
Pushed into 5.0.70  (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (pib:3)
[15 Sep 2008 21:15] Roman Zolotarev
I am meeting the same problem in my server with linux fedora 8 and Lotus Domino.

OS: Linux Fedora 8
Lotus Domino server 7.0.3

I am sorry, but I am not programmer, just administrator. I try to describe problem.

Server - Linux Fedora 8
mysql server - 5.0.67
compiled from source:
./configure --prefix=/usr --with-extra-charsets=all --enable-thread-safe-client --with-pthread --with-unix-socket-path=/var/lib/mysql/mysql.sock --datadir=/var/lib/mysql

ODBC connector:
mysql-connector-odbc-5.1.5r1144 also compiled from source.

I have error messages in domino log when I get access to mysql through LSX connector (ODBC mode):
Error in my_thread_global_end(): 9 threads didn't exit
Error in my_thread_global_end(): 9 threads didn't exit

When error appear, server get delay in answer, but all requests to mysql, after delay, works ok.

I changed peace of code in file dll.c:

void myodbc_end()
{
  if (!--myodbc_inited)
  {
    my_free(decimal_point,MYF(0));
    my_free(default_locale,MYF(0));
    my_free(thousands_sep,MYF(0));

    /* my_thread_end_wait_time was added in 5.1.14 and 5.0.32 */
#if !defined(NONTHREADSAFE) && \ 
    (MYSQL_VERSION_ID >= 50114 || \ 
     (MYSQL_VERSION_ID >= 50032 && MYSQL_VERSION_ID < 50100)) 
    /* 
       This eliminates the delay when my_end() is called and other threads 
       have been initialized but not ended. 
    */
    my_thread_end_wait_time= 0;
#endif 

!!!!    my_thread_end_wait_time= 0;   !!! I insert this line.

#ifdef MY_DONT_FREE_DBUG 
    /* 
       Function my_end() was changed to deallocate DBUG memory by default, 
       a flag MY_DONT_FREE_DBUG was added to disable this new behaviour 
    */
    my_end(MY_DONT_FREE_DBUG);
#else 
    my_end(0);
#endif 
  }
}

After recompile and start, delay is disappear, and all works fine, but messages in still appear in log. I don't know, is it possible to use it in production mode servers.
Is it possible memory leak or another troubles ?
[15 Sep 2008 21:28] Vladislav Vaintroub
Roman, the symptoms look similar and the cause is much probably the same. But the problem/fix here was for libmysql.dll and not for ODBC connector. Connectors have their own rules and live in a different source code repository (frankly speaking I do not even know where it is). Would it be possible for you to create a new bug for "connectors/odbc" , describe what you see and add reference to Bug #37226 to your bug report.

Many thanks in advance.
[15 Sep 2008 21:29] Vladislav Vaintroub
Roman, the symptoms look similar and the cause is much probably the same. But the problem/fix here was for libmysql.dll and not for ODBC connector. Connectors have their own rules and live in a different source code repository Frankly speaking I do not even know where it is. Would it be possible for you to create a new bug for connectors/odbc , describe what you see and add reference to Bug #37226 to your bug report.

Many thanks in advance.
[16 Sep 2008 7:04] Tonci Grgin
Roman, I have seen this occasionally and if I remember correctly, there is a C-client API bug posted on this by Georg Richter. Please search BugsDB before opening new report.
[29 Sep 2008 16:52] Paul Dubois
This fix is actually not yet in 6.0.x, apparently.
[29 Sep 2008 17:05] Paul Dubois
Noted in 5.0.70 changelog.

There were some problems using DllMain() hook functions on Windows
that automatically do global and per-thread initialization for
libmysqld.dll:

* Per-thread initialization: MySQL internally counts the number of
active threads, which causes a delay in my_end() if not all threads
have exited. But there are threads that can be started either by
Windows internally (often in TCP/IP scenarios) or by users. Those
threads do not necessarily use libmysql.dll functionality but still
contribute to the open-thread count. (One symptom is a five-second
delay in times for PHP scripts to finish.)
        
* Process-initialization: my_init() calls WSAStartup that itself loads
DLLs and can lead to a deadlock in the Windows loader.
      
To correct these problems, DLL initialization code now is not invoked
from libmysql.dll by default. To obtain the previous behavior (DLL
initialization code will be called), set the LIBMYSQL_DLLINIT
environment variable to any value. This variable exists only to
prevent breakage of existing Windows-only applications that do not
call mysql_thread_init() and work okay today. Use of LIBMYSQL_DLLINIT
is discouraged and it will be removed in MySQL 6.0.

Setting report to NDI pending push into 5.1.x/6.0.x.
[29 Sep 2008 17:21] Paul Dubois
Added to 6.0.7 changelog.

Leaving report in NDI pending push into 5.1.x.
[29 Sep 2008 17:48] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/54682

2842 Vladislav Vaintroub	2008-09-29
      Bug#37226 -  Explicit call of my_thread_init() on Windows for every 
      new thread.
      Bug#33031 - app linked to libmysql.lib crash if run as service in 
      vista under localsystem.
      
      This patch completely removes DllMain() from the libmysql.dll in 6.0
      eliminating explicit my_thread_init()/WSAStartup() during dll load
      or thread attach. 
      
      This patch is slightly different from what is done in 5.0/5.1.
      The difference is that there is no way that to reactivate DllMain() 
      code (in 5.x it is possible  with LIBMYSQL_DLLINIT environment variable)
[29 Sep 2008 17:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/54683

2842 Vladislav Vaintroub	2008-09-29
      Bug#37226 -  Explicit call of my_thread_init() on Windows for every 
      new thread.
      Bug#33031 - app linked to libmysql.lib crash if run as service in 
      vista under localsystem.
      
      This patch completely removes DllMain() from the libmysql.dll in 6.0
      eliminating explicit my_thread_init()/WSAStartup() during dll load
      or thread attach. 
      
      This patch is slightly different from what is done in 5.0/5.1.
      The difference is that there is no way that to reactivate DllMain() 
      code (in 5.x it is possible  with LIBMYSQL_DLLINIT environment variable)
[1 Oct 2008 16:03] Bugs System
Pushed into 5.1.28  (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:vvaintroub@mysql.com-20080901215154-1reds0fnzqdvaqsm) (pib:4)
[1 Oct 2008 17:02] Paul Dubois
Noted in 5.1.28 changelog.
[28 Oct 2008 21:05] Bugs System
Pushed into 5.1.29-ndb-6.2.17  (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:tomas.ulin@sun.com-20081028140209-u4emkk1xphi5tkfb) (pib:5)
[28 Oct 2008 22:24] Bugs System
Pushed into 5.1.29-ndb-6.3.19  (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:tomas.ulin@sun.com-20081028194045-0353yg8cvd2c7dd1) (pib:5)
[1 Nov 2008 9:50] Bugs System
Pushed into 5.1.29-ndb-6.4.0  (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:jonas@mysql.com-20081101082305-qx5a1bj0z7i8ueys) (pib:5)