Bug #22379 im_daemon_life_cycle.test fails on merge of 5.1 -> 5.1-engines
Submitted: 15 Sep 2006 7:44 Modified: 4 Oct 2006 19:22
Reporter: Ingo Strüwing Email Updates:
Status: Closed Impact on me:
None 
Category:Instance Manager Severity:S1 (Critical)
Version:5.0 + 5.1 OS:Linux (Linux)
Assigned to: Magnus Blåudd CPU Architecture:Any

[15 Sep 2006 7:44] Ingo Strüwing
Description:
im_cmd_line                    [ pass ]           2112
mysql-test-run: WARNING: check_expected_crash_and_restart couldn't find an entry for pid: 21818
im_daemon_life_cycle           [ pass ]           3038
Warning: [Thu Sep 14 23:12:57 2006] [im_daemon_life_cycle] Instance Manager did not shutdown gracefully.
Warning: [Thu Sep 14 23:12:57 2006] Mysqld is accepting connections on port 13048, but there is no process information.
mysql-test-run: *** ERROR: Can not kill Instance Manager or its children.
make: *** [test-ns] Error 1

Checked it twice to exclude a temporary flaw.

It might be a test script problem instead of an instance manager problem. I did not find a running instance manager process after the test. So the check by the script might be incorrect. If so, please change category.

How to repeat:
Log in on 'production'.
Stay at your home directory to use the local disk.
Do not work on a nfs disk.
bk clone bk-internal:/home/bk/mysql-5.1-engines mysql-5.1-engines
cd mysql-5.1-engines
Pull bk-internal:/home/bk/mysql-5.1
BUILD/compile-pentium-debug-max
make test
[19 Sep 2006 15:29] Magnus Blåudd
Can be seen in most clones.
The significant error message is "mysql-test-run: *** ERROR: Can not kill Instance Manager or its children."

This is cause by the mysqld started by im is still alive but there is no pid file for it.
[21 Sep 2006 15:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12354

ChangeSet@1.2348, 2006-09-21 17:56:36+02:00, msvensson@shellback.(none) +3 -0
  Bug #22379 im_daemon_life_cycle.test fails on merge of 5.1 -> 5.1-engines
   - Remove race situations that occur when removing pidfiles. 
     Primarily each process should remove it's own pidfile, secondly
     it should be removed by the process that created it and _only_ if it's
     certain the process is dead. Third, mysql-test-run.pl will remove
     the pidfile when process has been killed.
[21 Sep 2006 17:20] Magnus Blåudd
A check that fscanf returns 1 is also needed to avoid race, see below patch:

[msvensson@shellback instance-manager]$ bk diffs -u instance_options.cc
===== instance_options.cc 1.36 vs edited =====
--- 1.36/server-tools/instance-manager/instance_options.cc      2006-09-21 19:19:52 +02:00
+++ edited/instance_options.cc  2006-09-21 18:53:28 +02:00
@@ -406,7 +406,8 @@
   {
     pid_t pid;

-    fscanf(pid_file_stream, "%i", &pid);
+    if (fscanf(pid_file_stream, "%i", &pid) != 1)
+      pid= 0;
     my_fclose(pid_file_stream, MYF(0));
     return pid;
   }
[21 Sep 2006 18:23] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12360

ChangeSet@1.2281, 2006-09-21 20:23:12+02:00, msvensson@shellback.(none) +4 -0
  Bug #22379 im_daemon_life_cycle.test fails on merge of 5.1 -> 5.1-engines
  - Remove race situations that occur when removing pidfiles. 
    Primarily each process should remove its own pidfile, secondly
    it should be removed by the process that created it and _only_ if it's
    certain the process is dead. Third, mysql-test-run.pl will remove
    the pidfile when process has been killed.
[22 Sep 2006 13:54] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12405

ChangeSet@1.2355, 2006-09-22 15:54:09+02:00, msvensson@neptunus.(none) +1 -0
  Bug#22379 im_daemon_life_cycle.test fails on merge of 5.1 -> 5.1-engines
[22 Sep 2006 14:14] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12407

ChangeSet@1.2286, 2006-09-22 16:14:23+02:00, msvensson@neptunus.(none) +1 -0
  Bug#22379  im_daemon_life_cycle.test fails on merge of 5.1 -> 5.1-engines
   - Set state to STARTING before calling instance->start()
   - Check that pidfile has been created before changing STARTING => STARTED
[25 Sep 2006 7:29] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12457

ChangeSet@1.2245, 2006-09-25 09:28:52+02:00, msvensson@neptunus.(none) +3 -0
  Bug #22379 im_daemon_life_cycle.test fails on merge of 5.1 -> 5.1-engines
  Remove race situations that occur when removing pidfiles. Primarily each process should remove its own
  pidfile, secondly it should be removed by the process that created it and _only_ if it's
  certain the process is dead. Third, mysql-test-run.pl will remove the pidfile when process has been killed.
  - Set state of an instance to STARTING _before_ calling instance->start()
  - Check that pidfile of instance has been created before changing STARTING => STARTED
  - Only remove the pidfile if IM kills an instance with SIGKILL, otherwise the instance will remove it itself
[28 Sep 2006 9:38] Petr Chardin
pushed to 5.1.12
[3 Oct 2006 19:47] Dmitry Lenev
Pushed to 5.0.27
[4 Oct 2006 19:22] Paul Dubois
Noted in 5.0.27, 5.1.12 changelogs.

Instance Manager had a race condition involving mysqld PID file
removal.
[24 Oct 2006 14:24] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14267

ChangeSet@1.2325, 2006-10-24 18:25:24+04:00, anozdrin@alik. +1 -0
  Enable im_daemon_life_cycle.imtest, since BUG#22379 is closed.
[25 Oct 2006 16:45] Paul Dubois
The 5.0.x fix is in 5.0.30.