Bug #44068 RESTORE can disable the MyISAM Key Cache
Submitted: 3 Apr 2009 8:26 Modified: 25 Apr 2009 23:54
Reporter: Ingo Strüwing Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Backup Severity:S2 (Serious)
Version:6.0 OS:Any
Assigned to: Ingo Strüwing CPU Architecture:Any

[3 Apr 2009 8:26] Ingo Strüwing
Description:
When restoring a MyISAM table with the native MyISAM restore driver, the MyISAM Key Cache can be turned off.

How to repeat:
Run the test backup_myisam_sync before the test myisam_keycache_coverage. The latter will fail with "". The reason is that the former implicitly disables the key cache, so that the error injection in the latter test does not take effect.

Suggested fix:
--- storage/myisam/mi_examine_log.c     2009-02-22 18:02:16 +0000
+++ storage/myisam/mi_examine_log.c     2009-04-02 19:13:31 +0000
@@ -175,6 +175,7 @@ int mi_examine_log(MI_EXAMINE_LOG_PARAM 
     { { 11, 14 }, { 15, 22 }, { 15, 22 }, { 11, 14 }, { 11, 14 }, { 11, 14 },
       { 11, 14 }, { 11, 14 }, {  9, 16 }, {  9, 16 }, {  7, 12 }  };
   uint has_pid_and_result[]= {1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0};
+  int key_cache_blocks;
   DBUG_ENTER("mi_examine_log");
   DBUG_PRINT("myisamlog", ("max_files: %u  update: %u",
                            mi_exl->max_files, mi_exl->update));
@@ -208,8 +209,13 @@ int mi_examine_log(MI_EXAMINE_LOG_PARAM 
   bzero(mi_exl->com_count,sizeof(mi_exl->com_count));
   init_tree(&tree,0,0,sizeof(file_info),(qsort_cmp2) file_info_compare,1,
            (tree_element_free) file_info_free, NULL);
-  (void) init_key_cache(dflt_key_cache,KEY_CACHE_BLOCK_SIZE,KEY_CACHE_SIZE,
-                        0, 0);
+  /*
+    init_key_cache() returns the number of cache blocks allocated, if
+    *this* call allocated them. If the function fails, or if the cache
+    was already initialized before, it returns zero.
+  */
+  key_cache_blocks= init_key_cache(dflt_key_cache, KEY_CACHE_BLOCK_SIZE,
+                                   KEY_CACHE_SIZE, 0, 0);
 
   /*
     Initialize members of file_info that are used for pointing to
@@ -778,7 +784,9 @@ int mi_examine_log(MI_EXAMINE_LOG_PARAM 
   }
   DBUG_PRINT("myisamlog", ("end loop access_time: %lu  cmd_cnt: %lu",
                            access_time, mi_exl->number_of_commands));
-  end_key_cache(dflt_key_cache,1);
+  /* End the key cache only if it was initialized by this function. */
+  if (key_cache_blocks)
+    end_key_cache(dflt_key_cache,1);
   delete_tree(&tree);
   (void) end_io_cache(&cache);
   (void) my_close(log_file,MYF(0));
@@ -803,7 +811,9 @@ int mi_examine_log(MI_EXAMINE_LOG_PARAM 
   fflush(stderr);
  end:
   DBUG_PRINT("myisamlog", ("end label"));
-  end_key_cache(dflt_key_cache, 1);
+  /* End the key cache only if it was initialized by this function. */
+  if (key_cache_blocks)
+    end_key_cache(dflt_key_cache, 1);
   delete_tree(&tree);
   (void) end_io_cache(&cache);
   (void) my_close(log_file,MYF(0));
[3 Apr 2009 8:31] Ingo Strüwing
Suggested triage values:
Defect: serious. Disabling the MyISAM key cache is a serious problem.
Workaround: unacceptable. One could reactivate the cache after restore by setting a new cache size.
Impact: widespread. MyISAM is in wide use. Restoring of MyISAM tables is expected to be a frequent operation.
[3 Apr 2009 9:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/71299

2810 Ingo Struewing	2009-04-03
      Bug#44068 - RESTORE can disable the MyISAM Key Cache
      
      The test case myisam_keycache_coverage used to fail if it ran
      after backup_myisam_sync.
      
      The reason was that the latter test case disabled the key cache
      during RESTORE. The error injection in myisam_keycache_coverage
      became void as the key cache wasn't used any more. Reads of the
      index file bypassed the cache, and the statements succeeded.
      
      Fixed by avoiding de-initialization of the key cache if it was
      not initialized by the MyISAM log reading function mi_examine_log().
      The function will initialize and de-initialize the key cache if it
      is used by the myisamlog utility. It won't touch the key cache any
      more, if it is called from inside the server.
      
      Additional change: To make the state of the key cache visible,
      I added cleanup for the status variables blocks_used and
      blocks_unused. This improves the testability.
     @ mysql-test/r/myisam_keycache_coverage.result
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Updated test result.
     @ mysql-test/suite/backup/r/backup_myisam_sync.result
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Updated test result.
     @ mysql-test/suite/backup/t/backup_myisam_sync.test
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added checks for the state of the key cache.
     @ mysql-test/t/myisam_keycache_coverage.test
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added debug variable initialization to be safe.
        Added FORCE INDEX to the reference statement to make it
        equal to the tested statement.
        Removed old attempts to stabilize the test case.
     @ mysys/mf_keycache.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added cleanup for blocks_used and blocks_unused.
        Added DBUG.
     @ storage/myisam/mi_examine_log.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added local variable key_cache_blocks to be used to call
        end_key_cache() only if init_key_cache() did in fact
        initialize the cache.
[7 Apr 2009 19:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/71579

2810 Ingo Struewing	2009-04-07
      Bug#44068 - RESTORE can disable the MyISAM Key Cache
      
      The test case myisam_keycache_coverage used to fail if it ran
      after backup_myisam_sync.
      
      The reason was that the latter test case disabled the key cache
      during RESTORE. The error injection in myisam_keycache_coverage
      became void as the key cache wasn't used any more. Reads of the
      index file bypassed the cache, and the statements succeeded.
      
      Fixed by moving initialization and de-initialization of the key
      cache from mi_examine_log() to myisamlog. In the server, the
      initialization is already done.
      
      Additional change: To make the state of the key cache visible,
      I added cleanup for the status variables blocks_used and
      blocks_unused. This improves the testability.
     @ mysql-test/r/myisam_keycache_coverage.result
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Updated test result.
     @ mysql-test/suite/backup/r/backup_myisam_sync.result
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Updated test result.
     @ mysql-test/suite/backup/t/backup_myisam_sync.test
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added checks for the state of the key cache.
     @ mysql-test/t/myisam_keycache_coverage.test
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added FORCE INDEX to the reference statement to make it
        equal to the tested statement.
        Removed old attempts to stabilize the test case.
        Added proper set and unset of the debug variable
        to disturb debugging as little as possible.
     @ mysys/mf_keycache.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added cleanup for blocks_used and blocks_unused.
        Added DBUG.
     @ storage/myisam/mi_examine_log.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Moved calls to init_key_cache() and end_key_cache()
        from here to myisamlog.c
     @ storage/myisam/myisamlog.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Moved calls to init_key_cache() and end_key_cache()
        from mi_examine_log.c to here.
[14 Apr 2009 10:36] Rafal Somla
Good to push.
[14 Apr 2009 13:40] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72007

2810 Ingo Struewing	2009-04-14
      Bug#44068 - RESTORE can disable the MyISAM Key Cache
      
      The test case myisam_keycache_coverage used to fail if it ran
      after backup_myisam_sync.
      
      The reason was that the latter test case disabled the key cache
      during RESTORE. The error injection in myisam_keycache_coverage
      became void as the key cache wasn't used any more. Reads of the
      index file bypassed the cache, and the statements succeeded.
      
      Fixed by moving initialization and de-initialization of the key
      cache from mi_examine_log() to myisamlog. In the server, the
      initialization is already done.
      
      Additional change: To make the state of the key cache visible,
      I added cleanup for the status variables blocks_used and
      blocks_unused. This improves the testability.
     @ mysql-test/r/myisam_keycache_coverage.result
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Updated test result.
     @ mysql-test/suite/backup/r/backup_myisam_sync.result
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Updated test result.
     @ mysql-test/suite/backup/t/backup_myisam_sync.test
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added checks for the state of the key cache.
     @ mysql-test/t/myisam_keycache_coverage.test
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added FORCE INDEX to the tested statement.
        Removed old attempts to stabilize the test case.
        Added proper resetting of the debug variable
        to disturb debugging as little as possible.
        Removed clearing of the debug variable.
     @ mysys/mf_keycache.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Added cleanup for blocks_used and blocks_unused.
        Added DBUG.
     @ storage/myisam/mi_examine_log.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Moved calls to init_key_cache() and end_key_cache()
        from here to myisamlog.c
     @ storage/myisam/myisamlog.c
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        Moved calls to init_key_cache() and end_key_cache()
        from mi_examine_log.c to here.
[14 Apr 2009 15:07] Ingo Strüwing
Queued to 6.0-backup.
[23 Apr 2009 7:19] Bugs System
Pushed into 6.0.11-alpha (revid:alik@sun.com-20090423070920-e5lq3vrrqi016z2c) (version source revid:alik@sun.com-20090423070920-e5lq3vrrqi016z2c) (merge vers: 6.0.11-alpha) (pib:6)
[25 Apr 2009 23:54] Paul Dubois
Noted in 6.0.11 changelog.

A RESTORE operation that restored a MyISAM table using the native
MyISAM restore driver could cause the MyISAM key cache to be 
disabled.
[31 Dec 2009 19:09] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/95842

3020 Ingo Struewing	2009-12-31
      WL#5101 - MySQL Backup back port - MS09
      Merged revid:ingo.struewing@sun.com-20090414134006-e8sgm8i1ncf3hzkp
        Bug#44068 - RESTORE can disable the MyISAM Key Cache
        
        The test case myisam_keycache_coverage used to fail if it ran
        after backup_myisam_sync.
        
        The reason was that the latter test case disabled the key cache
        during RESTORE. The error injection in myisam_keycache_coverage
        became void as the key cache wasn't used any more. Reads of the
        index file bypassed the cache, and the statements succeeded.
        
        Fixed by moving initialization and de-initialization of the key
        cache from mi_examine_log() to myisamlog. In the server, the
        initialization is already done.
        
        Additional change: To make the state of the key cache visible,
        I added cleanup for the status variables blocks_used and
        blocks_unused. This improves the testability.
      
       @ mysql-test/r/myisam_keycache_coverage.result
          Bug#44068 - RESTORE can disable the MyISAM Key Cache
          Updated test result.
      
       @ mysql-test/suite/backup/r/backup_myisam_sync.result
          Bug#44068 - RESTORE can disable the MyISAM Key Cache
          Updated test result.
      
       @ mysql-test/suite/backup/t/backup_myisam_sync.test
          Bug#44068 - RESTORE can disable the MyISAM Key Cache
          Added checks for the state of the key cache.
      
       @ mysql-test/t/myisam_keycache_coverage.test
          Bug#44068 - RESTORE can disable the MyISAM Key Cache
          Added FORCE INDEX to the tested statement.
          Removed old attempts to stabilize the test case.
          Added proper resetting of the debug variable
          to disturb debugging as little as possible.
          Removed clearing of the debug variable.
      
       @ mysys/mf_keycache.c
          Bug#44068 - RESTORE can disable the MyISAM Key Cache
          Added cleanup for blocks_used and blocks_unused.
          Added DBUG.
      
       @ storage/myisam/myisamlog.c
          Bug#44068 - RESTORE can disable the MyISAM Key Cache
          Moved calls to init_key_cache() and end_key_cache()
          from mi_examine_log.c to here.
      
      original changeset: 2599.130.19
     @ mysql-test/r/backup_myisam2.result
        WL#5101 - MySQL Backup back port - MS09
            Bug#44068 - RESTORE can disable the MyISAM Key Cache
            Updated test result.
     @ mysql-test/r/myisam_keycache_coverage.result
        WL#5101 - MySQL Backup back port - MS09
            Bug#44068 - RESTORE can disable the MyISAM Key Cache
            Updated test result.
     @ mysql-test/t/backup_myisam2.test
        WL#5101 - MySQL Backup back port - MS09
            Bug#44068 - RESTORE can disable the MyISAM Key Cache
            Added checks for the state of the key cache.
     @ mysql-test/t/myisam_keycache_coverage.test
        WL#5101 - MySQL Backup back port - MS09
            Bug#44068 - RESTORE can disable the MyISAM Key Cache
            Added FORCE INDEX to the tested statement.
            Removed old attempts to stabilize the test case.
            Added proper resetting of the debug variable
            to disturb debugging as little as possible.
            Removed clearing of the debug variable.
     @ storage/myisam/mi_examine_log.c
        WL#5101 - MySQL Backup back port - MS09
            Bug#44068 - RESTORE can disable the MyISAM Key Cache
            Moved calls to init_key_cache() and end_key_cache()
            from here to myisamlog.c
     @ storage/myisam/myisamlog.c
        WL#5101 - MySQL Backup back port - MS09
            Bug#44068 - RESTORE can disable the MyISAM Key Cache
            Moved calls to init_key_cache() and end_key_cache()
            from mi_examine_log.c to here.