MySQL Bugs: #20430: mysqld.exe windows service stuck in "SERVICE_STOP

Bug #20430	mysqld.exe windows service stuck in "SERVICE_STOP_PENDING" status
Submitted:	13 Jun 2006 15:44	Modified:	8 Dec 2008 15:49
Reporter:	Sergio Nalin	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: General	Severity:	S2 (Serious)
Version:	4.1.20 and 5.0.22	OS:	Windows (Windows)
Assigned to:	Vladislav Vaintroub	CPU Architecture:	Any

Description:
Seems like that there is a synchronization issue with the Windows service wrapper in mysqld.exe.

If you stop the service while it's starting the wrapper locks in a SERVICE_STOP_PENDING status and you have to kill the process.

How to repeat:
Open two console windows,
from the first one launch:

net start MySQL

while the service is starting, from the other launch:

net stop MySQL

There is a very good change that the MySQL Windows service locks into a "stop pending" status.

It seems you can still access and use the DB, but there is no way to stop the service other then manually killing the mysqld.exe process from task manager.

Thank you for taking the time to report a problem.  Unfortunately
you are not using a current version of the product your reported a
problem with -- the problem might already be fixed. Please download
a new version from http://www.mysql.com/downloads/

If you are able to reproduce the bug with one of the latest versions,
please change the version on this bug report to the version you
tested and change the status back to "Open".  Again, thank you for
your continued support of MySQL.

Additional info:

I can not repeat it.

Replicated with versions 4.1.20 and 5.0.22.
I have been able to reproduce the issue on the following platforms:

Windows XP SP2 (up to date with all MS patches)
Windows 2003 Server (up to date with all MS patches)

It seems that it happens less often with 5.0.22, but the problem is still there.

To reproduce it you have to concurrently start/stop MySQL windows service.
I did that "by hand" just running "net start" and "net stop" commands, but you can reproduce it with more luck by concurrently running the following CMD scripts:

::Start script
:: --------------
:loop
net start MySQL
goto :loop
:: --------------

::Stop script
:: --------------
:loop
net stop MySQL
goto :loop
:: --------------

Launch both scripts, then open the service console and monitor the status of the MySQL service.
It stucks in the "stopping service" status almost immediately.

Verified just as described (using you scripts) with 5.0.22 on XP SP2. To repeat:

1. Run "start" script from one cmd.exe window
2. Immediately run "stop" script from cmd.exe window.
3. Wait for some time. "stop" script will hand, and even after stopping "start" script service is hanging in Stopping state forever.

Can be related to bug #17893.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58660

2712 Vladislav Vaintroub	2008-11-13
      Bug #20430 mysqld.exe windows service stuck in "SERVICE_STOP_PENDING" status
      
      The problem apppear to be a race condition, when service is stopped right
      after startup. We set the service status to SERVICE_RUNNING way too early
      it cannot yet handle stop requests -  initialization has not finished and 
      hEventShutdown  that signals server to stop is not yet created. If somebody 
      issues "net stop MySQL" at this time, MySQL is not informed about the stop
      and continue to run as usual, while NTService::ServiceMain() stucks forever 
      waiting for mysql's "main" thread to finish.
      
      Solution is to remain in SERVICE_START_PENDING status until after server 
      initialization  is fully complete and only then change the status to 
      SERVICE_RUNNING. In  SERVICE_START_PENDING we do not accept service control 
      requests, i.e it is not possible to stop service in that time.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58678

2712 Vladislav Vaintroub	2008-11-13
      Bug #20430 mysqld.exe windows service stuck in "SERVICE_STOP_PENDING" 
      status
            
      The problem appears to be a race condition, when service is being 
      stopped right after startup. We set the service status to SERVICE_RUNNING 
      way too early it cannot yet handle stop requests -  initialization has 
      not finished and  hEventShutdown  that signals server to stop is not yet
      created. If somebody issues "net stop MySQL" at this time, MySQL is not 
      informed about the stop and continues to run as usual, while 
      NTService::ServiceMain() stucks forever waiting for mysql's "main" thread 
      to finish.
            
      Solution is to remain in SERVICE_START_PENDING status until after server 
      initialization  is fully complete and only then change the status to 
      SERVICE_RUNNING. In  SERVICE_START_PENDING we do not accept service control 
      requests, i.e it is not possible to stop service in that time.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58714

2712 Vladislav Vaintroub	2008-11-14
      Bug #20430 mysqld.exe windows service stuck in "SERVICE_STOP_PENDING"
      status
      
      The problem appears to be a race condition, when service is being
      stopped right after startup. We set the service status to SERVICE_RUNNING
      way too early it cannot yet handle stop requests -  initialization has
      not finished and  hEventShutdown  that signals server to stop is not yet
      created. If somebody issues "net stop MySQL" at this time, MySQL is not
      informed about the stop and continues to run as usual, while
      NTService::ServiceMain() stucks forever waiting for mysql's "main" thread
      to finish.
      
      Solution is to remain in SERVICE_START_PENDING status until after server
      initialization  is fully complete and only then change the status to
      SERVICE_RUNNING. In  SERVICE_START_PENDING we do not accept service control
      requests, i.e it is not possible to stop service in that time.

OK, with #ifdef EMBEDDED_LIBRARY modification.

Pushed into 5.0.74  (revid:vvaintroub@mysql.com-20081114010141-5whvsiaz2lg34n0p) (version source revid:matthias.leich@sun.com-20081128184708-uyjg2dfrj2va5o3d) (pib:5)

Noted in 5.0.74 changelog.

A race condition between the mysqld.exe server and the Windows
service manager could lead to inability to stop the server from the
service manager.

Resetting report to NDI pending push into 5.1.x, 6.0.x.

Pushed into 5.1.31  (revid:vvaintroub@mysql.com-20081114010141-5whvsiaz2lg34n0p) (version source revid:patrick.crews@sun.com-20081126180318-v685u61mpgoc176x) (pib:5)

Pushed into 6.0.9-alpha  (revid:vvaintroub@mysql.com-20081114010141-5whvsiaz2lg34n0p) (version source revid:ingo.struewing@sun.com-20081122154620-4gmeq3eizn5kuhew) (pib:5)

Noted in 5.1.31, 6.0.9 changelogs.

Pushed into 5.1.31-ndb-6.2.17 (revid:tomas.ulin@sun.com-20090119095303-uwwvxiibtr38djii) (version source revid:tomas.ulin@sun.com-20090108105244-8opp3i85jw0uj5ib) (merge vers: 5.1.31-ndb-6.2.17) (pib:6)

Pushed into 5.1.31-ndb-6.3.21 (revid:tomas.ulin@sun.com-20090119104956-guxz190n2kh31fxl) (version source revid:tomas.ulin@sun.com-20090119104956-guxz190n2kh31fxl) (merge vers: 5.1.31-ndb-6.3.21) (pib:6)

Pushed into 5.1.31-ndb-6.4.1 (revid:tomas.ulin@sun.com-20090119144033-4aylstx5czzz88i5) (version source revid:tomas.ulin@sun.com-20090119144033-4aylstx5czzz88i5) (merge vers: 5.1.31-ndb-6.4.1) (pib:6)