| Bug #37226 | Explicit call of my_thread_init() on Windows for every new thread | ||
|---|---|---|---|
| Submitted: | 5 Jun 2008 14:41 | Modified: | 1 Oct 2008 17:02 |
| Reporter: | Andrey Hristov | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: C API (client library) | Severity: | S1 (Critical) |
| Version: | 5.0.22+ | OS: | Windows |
| Assigned to: | Vladislav Vaintroub | CPU Architecture: | Any |
[19 Aug 2008 15:47]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/51978 2666 Vladislav Vaintroub 2008-08-19 his patch fixes Bug#37226 Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 app linked to libmysql.lib crash if run as service in vista under localsystem Bug#38522 5 seconds delay when closing application using embedded server There are some problems using DllMain hook functions on Windows that automatically do global and per-thread initialization for a DLL in MySQL context 1)per-thread initialization(DLL_THREAD_ATTACH) MySQL internally counts number of active threads that and causes a delay in in my_end() if not all threads are exited. But,there are threads that can be started either by Windows internally (often in TCP/IP scenarios) or by user himself - those threads are not necessarily using libmysql.dll functionality, but nonetheless the contribute to the count of open threads. 2)process-initialization (DLL_PROCESS_ATTACH) my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in Windows loader. Fix is to remove dll initialization code from libmysql.dll in general case. I still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value will cause the old behavior (DLL init hooks will be called). This env.variable exists only to prevent breakage of existing Windows-only applications that don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is discouraged and it will be removed in 6.0
[26 Aug 2008 2:05]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/52514 2666 Vladislav Vaintroub 2008-08-26 [merge] This patch fixes Bug#37226 Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 app linked to libmysql.lib crash if run as service in vista under localsystem There are some problems using DllMain hook functions on Windows that automatically do global and per-thread initialization for libmysqld.dll 1)per-thread initialization(DLL_THREAD_ATTACH) MySQL internally counts number of active threads that and causes a delay in in my_end() if not all threads are exited. But,there are threads that can be started either by Windows internally (often in TCP/IP scenarios) or by user himself - those threads are not necessarily using libmysql.dll functionality, but nonetheless the contribute to the count of open threads. 2)process-initialization (DLL_PROCESS_ATTACH) my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in Windows loader. Fix is to remove dll initialization code from libmysql.dll in general case. I still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value will cause the old behavior (DLL init hooks will be called). This env.variable exists only to prevent breakage of existing Windows-only applications that don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is discouraged and it will be removed in 6.0
[26 Aug 2008 11:31]
Vladislav Vaintroub
Reviewed by jimw on IRC.
[1 Sep 2008 16:23]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/53011 2674 Vladislav Vaintroub 2008-09-01 This patch fixes Bug#37226 Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 app linked to libmysql.lib crash if run as service in vista under localsystem There are some problems using DllMain hook functions on Windows that automatically do global and per-thread initialization for libmysqld.dll 1)per-thread initialization(DLL_THREAD_ATTACH) MySQL internally counts number of active threads that and causes a delay in in my_end() if not all threads are exited. But,there are threads that can be started either by Windows internally (often in TCP/IP scenarios) or by user himself - those threads are not necessarily using libmysql.dll functionality, but nonetheless the contribute to the count of open threads. 2)process-initialization (DLL_PROCESS_ATTACH) my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in Windows loader. Fix is to remove dll initialization code from libmysql.dll in general case. I still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value will cause the old behavior (DLL init hooks will be called). This env.variable exists only to prevent breakage of existing Windows-only applications that don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is discouraged and it will be removed in 6.0
[1 Sep 2008 18:17]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/53018 2729 Vladislav Vaintroub 2008-09-01 Bug#37226 Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 app linked to libmysql.lib crash if run as service in vista under localsystem There are some problems using DllMain hook functions on Windows that automatically do global and per-thread initialization for libmysqld.dll 1)per-thread initialization(DLL_THREAD_ATTACH) MySQL internally counts number of active threads that and causes a delay in in my_end() if not all threads are exited. But,there are threads that can be started either by Windows internally (often in TCP/IP scenarios) or by user himself - those threads are not necessarily using libmysql.dll functionality, but nonetheless the contribute to the count of open threads. 2)process-initialization (DLL_PROCESS_ATTACH) my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in Windows loader. Fix is to remove dll initialization code from libmysql.dll in general case. I still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value will cause the old behavior (DLL init hooks will be called). This env.variable exists only to prevent breakage of existing Windows-only applications that don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is discouraged and it will be removed in 6.0
[1 Sep 2008 18:20]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/53019 2729 Vladislav Vaintroub 2008-09-01 Bug#37226 Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 app linked to libmysql.lib crash if run as service in vista under localsystem There are some problems using DllMain hook functions on Windows that automatically do global and per-thread initialization for libmysqld.dll 1)per-thread initialization(DLL_THREAD_ATTACH) MySQL internally counts number of active threads that and causes a delay in in my_end() if not all threads are exited. But,there are threads that can be started either by Windows internally (often in TCP/IP scenarios) or by user himself - those threads are not necessarily using libmysql.dll functionality, but nonetheless the contribute to the count of open threads. 2)process-initialization (DLL_PROCESS_ATTACH) my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in Windows loader. Fix is to remove dll initialization code from libmysql.dll in general case. I still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value will cause the old behavior (DLL init hooks will be called). This env.variable exists only to prevent breakage of existing Windows-only applications that don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is discouraged and it will be removed in 6.0
[1 Sep 2008 21:49]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/53026 2675 Vladislav Vaintroub 2008-09-01 Bug#37226 Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 app linked to libmysql.lib crash if run as service in vista under localsystem There are some problems using DllMain hook functions on Windows that automatically do global and per-thread initialization for libmysqld.dll 1)per-thread initialization(DLL_THREAD_ATTACH) MySQL internally counts number of active threads that and causes a delay in in my_end() if not all threads are exited. But,there are threads that can be started either by Windows internally (often in TCP/IP scenarios) or by user himself - those threads are not necessarily using libmysql.dll functionality, but nonetheless the contribute to the count of open threads. 2)process-initialization (DLL_PROCESS_ATTACH) my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in Windows loader. Fix is to remove dll initialization code from libmysql.dll in general case. I still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value will cause the old behavior (DLL init hooks will be called). This env.variable exists only to prevent breakage of existing Windows-only applications that don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is discouraged and it will be removed in 6.0
[1 Sep 2008 22:45]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/53031 2675 Vladislav Vaintroub 2008-09-01 Bug#37226 Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 app linked to libmysql.lib crash if run as service in vista under localsystem There are some problems using DllMain hook functions on Windows that automatically do global and per-thread initialization for libmysqld.dll 1)per-thread initialization(DLL_THREAD_ATTACH) MySQL internally counts number of active threads that and causes a delay in in my_end() if not all threads are exited. But,there are threads that can be started either by Windows internally (often in TCP/IP scenarios) or by user himself - those threads are not necessarily using libmysql.dll functionality, but nonetheless the contribute to the count of open threads. 2)process-initialization (DLL_PROCESS_ATTACH) my_init() calls WSAStartup that itself loads DLLs and can lead to a deadlock in Windows loader. Fix is to remove dll initialization code from libmysql.dll in general case. I still leave an environment variable LIBMYSQL_DLLINIT, which if set to any value will cause the old behavior (DLL init hooks will be called). This env.variable exists only to prevent breakage of existing Windows-only applications that don't do mysql_thread_init() and work ok today. Use of LIBMYSQL_DLLINIT is discouraged and it will be removed in 6.0
[13 Sep 2008 20:48]
Bugs System
Pushed into 6.0.7-alpha (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:hakan@mysql.com-20080725175322-8wgujj5xuzrjz3ke) (pib:3)
[15 Sep 2008 8:11]
Bugs System
Pushed into 5.0.70 (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (pib:3)
[15 Sep 2008 21:15]
Roman Zolotarev
I am meeting the same problem in my server with linux fedora 8 and Lotus Domino.
OS: Linux Fedora 8
Lotus Domino server 7.0.3
I am sorry, but I am not programmer, just administrator. I try to describe problem.
Server - Linux Fedora 8
mysql server - 5.0.67
compiled from source:
./configure --prefix=/usr --with-extra-charsets=all --enable-thread-safe-client --with-pthread --with-unix-socket-path=/var/lib/mysql/mysql.sock --datadir=/var/lib/mysql
ODBC connector:
mysql-connector-odbc-5.1.5r1144 also compiled from source.
I have error messages in domino log when I get access to mysql through LSX connector (ODBC mode):
Error in my_thread_global_end(): 9 threads didn't exit
Error in my_thread_global_end(): 9 threads didn't exit
When error appear, server get delay in answer, but all requests to mysql, after delay, works ok.
I changed peace of code in file dll.c:
void myodbc_end()
{
if (!--myodbc_inited)
{
my_free(decimal_point,MYF(0));
my_free(default_locale,MYF(0));
my_free(thousands_sep,MYF(0));
/* my_thread_end_wait_time was added in 5.1.14 and 5.0.32 */
#if !defined(NONTHREADSAFE) && \
(MYSQL_VERSION_ID >= 50114 || \
(MYSQL_VERSION_ID >= 50032 && MYSQL_VERSION_ID < 50100))
/*
This eliminates the delay when my_end() is called and other threads
have been initialized but not ended.
*/
my_thread_end_wait_time= 0;
#endif
!!!! my_thread_end_wait_time= 0; !!! I insert this line.
#ifdef MY_DONT_FREE_DBUG
/*
Function my_end() was changed to deallocate DBUG memory by default,
a flag MY_DONT_FREE_DBUG was added to disable this new behaviour
*/
my_end(MY_DONT_FREE_DBUG);
#else
my_end(0);
#endif
}
}
After recompile and start, delay is disappear, and all works fine, but messages in still appear in log. I don't know, is it possible to use it in production mode servers.
Is it possible memory leak or another troubles ?
[15 Sep 2008 21:28]
Vladislav Vaintroub
Roman, the symptoms look similar and the cause is much probably the same. But the problem/fix here was for libmysql.dll and not for ODBC connector. Connectors have their own rules and live in a different source code repository (frankly speaking I do not even know where it is). Would it be possible for you to create a new bug for "connectors/odbc" , describe what you see and add reference to Bug #37226 to your bug report. Many thanks in advance.
[15 Sep 2008 21:29]
Vladislav Vaintroub
Roman, the symptoms look similar and the cause is much probably the same. But the problem/fix here was for libmysql.dll and not for ODBC connector. Connectors have their own rules and live in a different source code repository Frankly speaking I do not even know where it is. Would it be possible for you to create a new bug for connectors/odbc , describe what you see and add reference to Bug #37226 to your bug report. Many thanks in advance.
[16 Sep 2008 7:04]
Tonci Grgin
Roman, I have seen this occasionally and if I remember correctly, there is a C-client API bug posted on this by Georg Richter. Please search BugsDB before opening new report.
[29 Sep 2008 16:52]
Paul DuBois
This fix is actually not yet in 6.0.x, apparently.
[29 Sep 2008 17:05]
Paul DuBois
Noted in 5.0.70 changelog.
There were some problems using DllMain() hook functions on Windows
that automatically do global and per-thread initialization for
libmysqld.dll:
* Per-thread initialization: MySQL internally counts the number of
active threads, which causes a delay in my_end() if not all threads
have exited. But there are threads that can be started either by
Windows internally (often in TCP/IP scenarios) or by users. Those
threads do not necessarily use libmysql.dll functionality but still
contribute to the open-thread count. (One symptom is a five-second
delay in times for PHP scripts to finish.)
* Process-initialization: my_init() calls WSAStartup that itself loads
DLLs and can lead to a deadlock in the Windows loader.
To correct these problems, DLL initialization code now is not invoked
from libmysql.dll by default. To obtain the previous behavior (DLL
initialization code will be called), set the LIBMYSQL_DLLINIT
environment variable to any value. This variable exists only to
prevent breakage of existing Windows-only applications that do not
call mysql_thread_init() and work okay today. Use of LIBMYSQL_DLLINIT
is discouraged and it will be removed in MySQL 6.0.
Setting report to NDI pending push into 5.1.x/6.0.x.
[29 Sep 2008 17:21]
Paul DuBois
Added to 6.0.7 changelog. Leaving report in NDI pending push into 5.1.x.
[29 Sep 2008 17:48]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/54682 2842 Vladislav Vaintroub 2008-09-29 Bug#37226 - Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 - app linked to libmysql.lib crash if run as service in vista under localsystem. This patch completely removes DllMain() from the libmysql.dll in 6.0 eliminating explicit my_thread_init()/WSAStartup() during dll load or thread attach. This patch is slightly different from what is done in 5.0/5.1. The difference is that there is no way that to reactivate DllMain() code (in 5.x it is possible with LIBMYSQL_DLLINIT environment variable)
[29 Sep 2008 17:49]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/54683 2842 Vladislav Vaintroub 2008-09-29 Bug#37226 - Explicit call of my_thread_init() on Windows for every new thread. Bug#33031 - app linked to libmysql.lib crash if run as service in vista under localsystem. This patch completely removes DllMain() from the libmysql.dll in 6.0 eliminating explicit my_thread_init()/WSAStartup() during dll load or thread attach. This patch is slightly different from what is done in 5.0/5.1. The difference is that there is no way that to reactivate DllMain() code (in 5.x it is possible with LIBMYSQL_DLLINIT environment variable)
[1 Oct 2008 16:03]
Bugs System
Pushed into 5.1.28 (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:vvaintroub@mysql.com-20080901215154-1reds0fnzqdvaqsm) (pib:4)
[1 Oct 2008 17:02]
Paul DuBois
Noted in 5.1.28 changelog.
[28 Oct 2008 21:05]
Bugs System
Pushed into 5.1.29-ndb-6.2.17 (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:tomas.ulin@sun.com-20081028140209-u4emkk1xphi5tkfb) (pib:5)
[28 Oct 2008 22:24]
Bugs System
Pushed into 5.1.29-ndb-6.3.19 (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:tomas.ulin@sun.com-20081028194045-0353yg8cvd2c7dd1) (pib:5)
[1 Nov 2008 9:50]
Bugs System
Pushed into 5.1.29-ndb-6.4.0 (revid:vvaintroub@mysql.com-20080901214637-vk62y7c3aqpfdnsf) (version source revid:jonas@mysql.com-20081101082305-qx5a1bj0z7i8ueys) (pib:5)
Description: In dll.c, there is the following code: switch (ul_reason_being_called) { case DLL_PROCESS_ATTACH: /* case of libentry call in win 3.x */ if (!inited++) { s_hModule=hInst; libmysql_init(); main_thread=GetCurrentThreadId(); } break; case DLL_THREAD_ATTACH: threads++; my_thread_init(); break; case DLL_PROCESS_DETACH: /* case of wep call in win 3.x */ if (!--inited) /* Safety */ { /* my_thread_init() */ /* This may give extra safety */ my_end(0); } break; case DLL_THREAD_DETACH: /* Main thread will free by my_end() */ threads--; if (main_thread != GetCurrentThreadId()) my_thread_end(); break; default: break; } /* switch */ For convinience of Windows applications that use libmysql, my_thread_init() is called implicitly. This is only on Windows. However, if libmysql is loaded with LoadLibrary() and unloaded with FreeLibrary() later and in the mean time few threads have been created but still working, FreeLibrary() will force a 5 Sec. delay in my_thr_init.c : pthread_mutex_lock(&THR_LOCK_threads); while (THR_thread_count > 0) { int error= pthread_cond_timedwait(&THR_COND_threads, &THR_LOCK_threads, &abstime); if (error == ETIMEDOUT || error == ETIME) Because of the convenience, my_thread_init() was called in threads which never use libmysql. In addition, the fine documentation of mysql_thread_init() reads: "This function must be called early within each created thread to initialize thread-specific variables. However, you may not necessarily need to invoke it explicitly: mysql_thread_init() is automatically called by my_init(), which itself is automatically called by mysql_init(), mysql_library_init(), mysql_server_init(), and mysql_connect(). If you invoke any of those functions, mysql_thread_init() will be called for you." One can say that this is not much of a problem. That the DLL rarely is loaded with LoadLibrary and the application should be smart enough to know what is going on. This could be the case but there is one exception - PHP. PHP, on Windows, uses dlls for its extensions. When a MySQL extension is loaded libmysql is also loaded and for every thread from this point my_thread_init() will be called. However, it could be the case that the web server or PHP itself (like in the case we found this bug) creates threads that outlive the MySQL extensions. The MySQL extension is unloaded, my_end(0) is called. libmysql counts the thread number, since about 5.0.26, a change by Monty on Nov 30rd 2006. Because the counter was incremented in the still living thread(s), and not decremented, my_end(0) will wait. Which means that PHP will wait. If you use even non-threaded server, like apache1 or a server in CGI mode, the script will need 5 seconds to finish. This is a VERY serious problem. Because of it lot of Windows PHP developers downgrade their libmysql to what came with PHP 5.2.1 - libmysql 5.0.21, which is pretty, pretty old. Maybe there are other problems, because memory is not freed, but the process is ending anyway. How to repeat: See above Suggested fix: Find a solution, maybe with an ENV variable on Windows, to skip the call to my_thread_init() in dll.c . If generally changed it might break applications, but with an ENV variable the fix can cure other applications.