Bug #38133 Myisamlog test fails on Windows
Submitted: 15 Jul 2008 13:02 Modified: 26 Mar 2009 23:38
Reporter: Chuck Bell Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Backup Severity:S3 (Non-critical)
Version:6.0.6 OS:Windows
Assigned to: Ingo Strüwing CPU Architecture:Any

[15 Jul 2008 13:02] Chuck Bell
Description:
The myisamlog test is failing on Windows machines in the mysql-6.0-backup trees. The error is:

Warning: error 13, expected 0 on command extra at 328
Got error 13, expected 0 on command re-open at 1218
mysqltest: At line 36: command "$MYISAMLOG -F $MYSQLTEST_VARDIR/master-data -u $MYSQLTEST_VARDIR/master-data/myisam.log " failed

How to repeat:
Run tests in backup tree.
[16 Jul 2008 8:56] Bugs System
Pushed into 6.0.6-alpha
[16 Jul 2008 21:03] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/49892

2667 Ingo Struewing	2008-07-16
      Bug#38133 - Myisamlog test fails on Windows
      
      Updating of a table from the myisam.log file was not possible in most cases.
      - The initialization of myisamlog set the number of usable file descriptors
        to one (1!) on Windows.
      - Every log command to a different file required a close of the previous file.
        After a 'close' command the closing of the previous file failed due to a
        lack of closable files and myisamlog stopped.
      - File locking on Windows failed for the sequence "exclusive lock, shared lock,
        unlock, exclusive lock". It blocked on the last try to acquire an exclusive lock.
      
      Fixed by
      - requesting a reasonable number of file descriptors,
      - before re-open, close a file only if all file descriptors are in use,
      - do not re-open a file, if the command is 'close',
      - unlock a file before every lock.
[23 Jul 2008 9:58] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50283

2672 Ingo Struewing	2008-07-23
      Bug#38133 - Myisamlog test fails on Windows
      
      Updating of a table from the myisam.log file was not possible in most cases.
      - The initialization of myisamlog set the number of usable file descriptors
      to one (1!) on Windows.
      - Every log command to a different file required a close of the previous file.
      After a 'close' command the closing of the previous file failed due to a
      lack of closable files and myisamlog stopped.
      - File locking on Windows failed for the sequence "exclusive lock, shared lock,
      unlock, exclusive lock". It blocked on the last try to acquire an exclusive lock.
      
      Fixed by
      - requesting a reasonable number of file descriptors,
      - before re-open, close a file only if all file descriptors are in use,
      - do not re-open a file, if the command is 'close',
      - unlock a file before every lock.
[28 Jul 2008 22:20] Chuck Bell
Ok to push.
[14 Sep 2008 3:06] Bugs System
Pushed into 6.0.7-alpha  (revid:cbell@mysql.com-20080715200229-64xwsuz5norps6ju) (version source revid:sven@mysql.com-20080818175803-c1nutd5773r6b4gd) (pib:3)
[25 Sep 2008 20:44] Guilhem Bichot
started review
[13 Nov 2008 8:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58606

2730 Ingo Struewing	2008-11-13
      Bug#38133 - Myisamlog test fails on Windows
      
      Updating of a table from the myisam.log file was not possible in most cases.
      - The initialization of myisamlog set the number of usable file descriptors
        to one (1!) on Windows.
      - Every log command to a different file required a close of the previous file.
        After a 'close' command the closing of the previous file failed because
        all files were closed already and myisamlog stopped.
      - File locking on Windows failed for the sequence "exclusive lock, shared lock,
        unlock, exclusive lock". It blocked on the last try to acquire an exclusive lock.
      
      Fixed by
      - requesting a reasonable number of file descriptors,
      - before re-open, close a file only if all file descriptors are in use,
      - do not re-open a file, if the command is 'close',
      - unlock a file before every lock.
[21 Nov 2008 16:23] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/59568

2740 Ingo Struewing	2008-11-21
      Bug#38133 - Myisamlog test fails on Windows
      
      Updating of a table from the myisam.log file was not possible in most cases
      on Windows.
      - The initialization of myisamlog set the number of usable file descriptors
      to one (1!) on Windows.
      - Every log command to a different file required a close of the previous file.
      After a 'close' command the closing of the previous file failed because
      all files were closed already and myisamlog stopped.
      - File locking on Windows failed for the sequence "exclusive lock, shared lock,
      unlock, exclusive lock". It blocked on the last try to acquire an exclusive lock.
      
      Fixed by
      - requesting a reasonable number of file descriptors,
      - before re-open, close a file only if all file descriptors are in use,
      - do not re-open a file, if the command is 'close',
      - unlock a file before changing the lock type.
[10 Dec 2008 18:43] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/61276

2740 Ingo Struewing	2008-12-10
      Bug#38133 - Myisamlog test fails on Windows
      
      Updating of a table from the myisam.log file was not possible in most cases
      on Windows.
      - The initialization of myisamlog set the number of usable file descriptors
      to one (1!) on Windows.
      - Every log command to a different file required a close of the previous file.
      After a 'close' command the closing of the previous file failed because
      all files were closed already and myisamlog stopped.
      - File locking on Windows failed for the sequence "exclusive lock, shared lock,
      unlock, exclusive lock". It blocked on the last try to acquire an exclusive lock.
      
      Fixed by
      - requesting a reasonable number of file descriptors,
      - before re-open, close a file only if all file descriptors are in use,
      - do not re-open a file, if the command is 'close',
      - unlock a file before changing the lock type.
[15 Dec 2008 13:23] Guilhem Bichot
Approved with minor comments
[22 Feb 2009 10:17] Ingo Strüwing
Bug#43008 (main.mysqlbinlog fails on Windows in PB2) may be a duplicate of this one.
[22 Feb 2009 18:02] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/67115

2773 Ingo Struewing	2009-02-22
      Bug#38133 - Myisamlog test fails on Windows
      
      Updating of a table from the myisam.log file was not possible in most cases
      on Windows.
      - The initialization of myisamlog set the number of usable file descriptors
      to one (1!) on Windows.
      - Every log command to a different file required a close of the previous file.
      After a 'close' command the closing of the previous file failed because
      all files were closed already and myisamlog stopped.
      - File locking on Windows failed for the sequence "exclusive lock, shared lock,
      unlock, exclusive lock". It blocked on the last try to acquire an exclusive lock.
      
      Fixed by
      - requesting a reasonable number of file descriptors,
      - before re-open, close a file only if all file descriptors are in use,
      - do not re-open a file, if the command is 'close',
      - unlock a file before changing the lock type.
     @ .bzrignore
        Bug#38133 - Myisamlog test fails on Windows
        Added symbolic link libmysqld/bml.cc.
     @ mysql-test/r/myisamlog.result
        Bug#38133 - Myisamlog test fails on Windows
        fixed test result.
     @ mysql-test/r/myisamlog_coverage.result
        Bug#38133 - Myisamlog test fails on Windows
        New test result.
     @ mysql-test/t/myisamlog.test
        Bug#38133 - Myisamlog test fails on Windows
        Re-enabled the test case for Windows.
        Changed line endings to UNIX format.
        Added server restart to be independent from earlier execution.
     @ mysql-test/t/myisamlog_coverage-master.opt
        Bug#38133 - Myisamlog test fails on Windows
        New options file for new test case.
     @ mysql-test/t/myisamlog_coverage.test
        Bug#38133 - Myisamlog test fails on Windows
        New test case.
     @ mysys/my_lock.c
        Bug#38133 - Myisamlog test fails on Windows
        Added unlocking before locking to prevent multiple locks by the same process.
     @ sql/handler.cc
        Bug#38133 - Myisamlog test fails on Windows
        Changed abort() to DBUG_ABORT() to get rid of pop-up windows.
     @ storage/myisam/mi_close.c
        Bug#38133 - Myisamlog test fails on Windows
        Moved free of share to end of mi_close().
     @ storage/myisam/mi_examine_log.c
        Bug#38133 - Myisamlog test fails on Windows
        Added member lock_type to file_info.
        Added ERROR_INJECT for coverage testing, available in debug build only.
        Changed tracking of allocated memory in 'buff' and 'file_info' so that
        it can be freed in case of errors.
        Added a check for valid data in the log file.
        Disabled re-opening for a close command.
        Reduced closing of files to cases where file descriptors are used up.
        Added error handling to the call to tree_insert().
        Pulled close_some_files() out of reopen_closed_file() to have access
        to max_files.
        Added code to re-lock a file after re-open.
     @ storage/myisam/mi_locking.c
        Bug#38133 - Myisamlog test fails on Windows
        Set my_errno on error in mi_lock_database(), like in other branches.
     @ storage/myisam/mi_log.c
        Bug#38133 - Myisamlog test fails on Windows
        Disable logging for temporary tables.
        Replaced several info->s by share.
     @ storage/myisam/myisamlog.c
        Bug#38133 - Myisamlog test fails on Windows
        Allowed for more of one (1!) file descriptor on platforms that
        do not raise the maximum above the request.
        Added --debug option to support debugging with new mysql-test-run.
        Added fflush(stdout) to have repeatable order of output lines
        to support testing.
[22 Feb 2009 19:05] Ingo Strüwing
Queued to mysql-6.0-backup.
[26 Mar 2009 12:34] Bugs System
Pushed into 6.0.11-alpha (revid:alik@sun.com-20090326121822-pt84kzxxayzho4mn) (version source revid:rafal.somla@sun.com-20090302164601-znhm4tadplfi2iqu) (merge vers: 6.0.11-alpha) (pib:6)
[26 Mar 2009 23:38] Paul DuBois
Noted in 6.0.11 changelog.

Locking of myisam.log did not work correctly on Windows.
[31 Dec 2009 16:10] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/95838

3018 Ingo Struewing	2009-12-31
      WL#5101 - MySQL Backup back port - MS09
      Merged revid:ingo.struewing@sun.com-20090222180216-riba1s5xckocagnp
        Bug#38133 - Myisamlog test fails on Windows
        
        Updating of a table from the myisam.log file was not possible in most cases
        on Windows.
        - The initialization of myisamlog set the number of usable file descriptors
        to one (1!) on Windows.
        - Every log command to a different file required a close of the previous file.
        After a 'close' command the closing of the previous file failed because
        all files were closed already and myisamlog stopped.
        - File locking on Windows failed for the sequence "exclusive lock, shared lock,
        unlock, exclusive lock". It blocked on the last try to acquire an exclusive lock.
        
        Fixed by
        - requesting a reasonable number of file descriptors,
        - before re-open, close a file only if all file descriptors are in use,
        - do not re-open a file, if the command is 'close',
        - unlock a file before changing the lock type.
      
       @ .bzrignore
          Bug#38133 - Myisamlog test fails on Windows
          Added symbolic link libmysqld/bml.cc.
      
       @ mysql-test/r/myisamlog.result
          Bug#38133 - Myisamlog test fails on Windows
          fixed test result.
      
       @ mysql-test/r/myisamlog_coverage.result
          Bug#38133 - Myisamlog test fails on Windows
          New test result.
      
       @ mysql-test/t/myisamlog.test
          Bug#38133 - Myisamlog test fails on Windows
          Re-enabled the test case for Windows.
          Changed line endings to UNIX format.
          Added server restart to be independent from earlier execution.
      
       @ mysql-test/t/myisamlog_coverage-master.opt
          Bug#38133 - Myisamlog test fails on Windows
          New options file for new test case.
      
       @ mysql-test/t/myisamlog_coverage.test
          Bug#38133 - Myisamlog test fails on Windows
          New test case.
      
       @ mysys/my_lock.c
          Bug#38133 - Myisamlog test fails on Windows
          Added unlocking before locking to prevent multiple locks by the same process.
      
       @ sql/handler.cc
          Bug#38133 - Myisamlog test fails on Windows
          Changed abort() to DBUG_ABORT() to get rid of pop-up windows.
      
       @ storage/myisam/mi_close.c
          Bug#38133 - Myisamlog test fails on Windows
          Moved free of share to end of mi_close().
      
       @ storage/myisam/mi_examine_log.c
          Bug#38133 - Myisamlog test fails on Windows
          Added member lock_type to file_info.
          Added ERROR_INJECT for coverage testing, available in debug build only.
          Changed tracking of allocated memory in 'buff' and 'file_info' so that
          it can be freed in case of errors.
          Added a check for valid data in the log file.
          Disabled re-opening for a close command.
          Reduced closing of files to cases where file descriptors are used up.
          Added error handling to the call to tree_insert().
          Pulled close_some_files() out of reopen_closed_file() to have access
          to max_files.
          Added code to re-lock a file after re-open.
      
       @ storage/myisam/mi_locking.c
          Bug#38133 - Myisamlog test fails on Windows
          Set my_errno on error in mi_lock_database(), like in other branches.
      
       @ storage/myisam/mi_log.c
          Bug#38133 - Myisamlog test fails on Windows
          Disable logging for temporary tables.
          Replaced several info->s by share.
      
       @ storage/myisam/myisamlog.c
          Bug#38133 - Myisamlog test fails on Windows
          Allowed for more of one (1!) file descriptor on platforms that
          do not raise the maximum above the request.
          Added --debug option to support debugging with new mysql-test-run.
          Added fflush(stdout) to have repeatable order of output lines
          to support testing.
      
      Note: main.myisamlog-coverage.test fails. This is to be fixed by later merges.
          
      original changeset: 2599.122.1
     @ mysql-test/extra/rpl_tests/rpl_row_sp003.test
        WL#5101 - MySQL Backup back port - MS09
        Sabilized test case.
     @ mysql-test/r/myisamlog.result
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            fixed test result.
     @ mysql-test/r/myisamlog_coverage.result
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            New test result.
     @ mysql-test/suite/rpl/r/rpl_row_sp003.result
        WL#5101 - MySQL Backup back port - MS09
        Updated test result.
     @ mysql-test/t/myisamlog.test
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            Re-enabled the test case for Windows.
            Changed line endings to UNIX format.
            Added server restart to be independent from earlier execution.
     @ mysql-test/t/myisamlog_coverage-master.opt
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            New options file for new test case.
     @ mysql-test/t/myisamlog_coverage.test
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            New test case.
     @ sql/handler.cc
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            Changed abort() to DBUG_ABORT() to get rid of pop-up windows.
     @ sql/mysqld.cc
        WL#5101 - MySQL Backup back port - MS09
        Free backupdir variable value to avoid memory leak.
     @ storage/myisam/mi_close.c
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            Moved free of share to end of mi_close().
     @ storage/myisam/mi_examine_log.c
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            Added member lock_type to file_info.
            Added ERROR_INJECT for coverage testing, available in debug build only.
            Changed tracking of allocated memory in 'buff' and 'file_info' so that
            it can be freed in case of errors.
            Added a check for valid data in the log file.
            Disabled re-opening for a close command.
            Reduced closing of files to cases where file descriptors are used up.
            Added error handling to the call to tree_insert().
            Pulled close_some_files() out of reopen_closed_file() to have access
            to max_files.
            Added code to re-lock a file after re-open.
     @ storage/myisam/mi_locking.c
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            Set my_errno on error in mi_lock_database(), like in other branches.
     @ storage/myisam/mi_log.c
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            Disable logging for temporary tables.
            Replaced several info->s by share.
     @ storage/myisam/mi_open.c
        WL#5101 - MySQL Backup back port - MS09
        Fixed DBUG_PRINT to avoid compiler warnings.
     @ storage/myisam/myisamlog.c
        WL#5101 - MySQL Backup back port - MS09
            Bug#38133 - Myisamlog test fails on Windows
            Allowed for more of one (1!) file descriptor on platforms that
            do not raise the maximum above the request.
            Added --debug option to support debugging with new mysql-test-run.
            Added fflush(stdout) to have repeatable order of output lines
            to support testing.