Bug #45377 ARCHIVE tables aren't discoverable after OPTIMIZE
Submitted: 8 Jun 2009 16:08 Modified: 16 Aug 2010 11:42
Reporter: Stewart Smith Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Archive storage engine Severity:S3 (Non-critical)
Version:5.1, 6.0 bzr OS:Any
Assigned to: Davi Arnaut CPU Architecture:Any
Triage: Triaged: D3 (Medium)

[8 Jun 2009 16:08] Stewart Smith
Description:
OPTIMIZE TABLE on an ARCHIVE table rewrites the ARZ file.

It does not copy the FRM stored in the ARZ across. So an ARCHIVE table that has been OPTIMIZEd cannot be discovered.

How to repeat:
CREATE TABLE t1 (a int) ENGINE=ARCHIVE;
OPTIMIZE TABLE t1;
rm t1.frm
select * from t1;

You can also verify with archive_reader

Suggested fix:
Something like this (as fixed in Drizzle):

@@ -1011,6 +1193,15 @@ int ha_archive::optimize(Session *, HA_C
     share->archive_write_open= false;
   }
 
+  char* proto_string;
+  proto_string= (char*)malloc(sizeof(char) * archive.frm_length);
+  if(!proto_string)
+  {
+    return ENOMEM;
+  }
+
+  azread_frm(&archive, proto_string);
+
   /* Lets create a file to contain the new data */
   fn_format(writer_filename, share->table_name.c_str(), "", ARN,
             MY_REPLACE_EXT | MY_UNPACK_FILENAME);
@@ -1018,6 +1209,8 @@ int ha_archive::optimize(Session *, HA_C
   if (!(azopen(&writer, writer_filename, O_CREAT|O_RDWR, AZ_METHOD_BLOCK)))
     return(HA_ERR_CRASHED_ON_USAGE);
 
+  azwrite_frm(&writer, proto_string, archive.frm_length);
+
   /*
     An extended rebuild is a lot more effort. We open up each row and re-record it.
     Any dead rows are removed (aka rows that may have been partially recorded).
[8 Jun 2009 20:51] Sveta Smirnova
Thank you for the report.

Verified as described.
[16 Jun 2009 0:15] Stewart Smith
How does breaking a feature if somebody has ever run an OPTIMIZE on the table qualify at all as a Feature Request?
[17 Jun 2009 1:33] Stewart Smith
This should be re-triaged as I strongly disagree with "feature request". A feature stops working (table discovery) iff at any point somebody ran OPTIMIZE TABLE on the table. A reason that this could be very rare/not been seen in wild is that people don't tend to OPTIMIZE ARCHIVE tables too often/at all as it's only needed if you did a *lot* of reads while writing it. So it's quite possible this hasn't been seen in the wild, but the potential error modes from it are rather nasty.
[20 Jul 2009 14:13] Stewart Smith
patch to fix bug 45377

Attachment: archive_discover_after_optimize.patch (text/x-patch), 1.60 KiB.

[11 May 2010 8:34] Stewart Smith
any progress on this?

I wrote the patch ages ago (back when I worked for Sun).
[23 Jul 2010 12:04] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/114217

3133 Davi Arnaut	2010-07-23
      Bug#45377: ARCHIVE tables aren't discoverable after OPTIMIZE
      
      The problem was that the optimize method of the ARCHIVE storage
      engine was not preserving the FRM embedded in the ARZ file when
      rewriting the ARZ file for optimization. The ARCHIVE engine stores
      the FRM in the ARZ file so it can be transferred from machine to
      machine without also copying the FRM -- the engine restores the
      embedded FRM during discovery.
      
      The solution is to copy over the FRM when rewriting the ARZ file.
      In addition, some initial error checking is performed to ensure
      garbage is not copied over.
     @ mysql-test/t/archive.test
        Add test case for Bug#45377.
     @ storage/archive/azio.c
        Add error checking to ensure that the I/O operations are
        successful.
     @ storage/archive/ha_archive.cc
        Copy over the embedded FRM.
[26 Jul 2010 15:54] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/114362

3152 Davi Arnaut	2010-07-26
      Bug#45377: ARCHIVE tables aren't discoverable after OPTIMIZE
      
      The problem was that the optimize method of the ARCHIVE storage
      engine was not preserving the FRM embedded in the ARZ file when
      rewriting the ARZ file for optimization. The ARCHIVE engine stores
      the FRM in the ARZ file so it can be transferred from machine to
      machine without also copying the FRM -- the engine restores the
      embedded FRM during discovery.
      
      The solution is to copy over the FRM when rewriting the ARZ file.
      In addition, some initial error checking is performed to ensure
      garbage is not copied over.
     @ mysql-test/t/archive.test
        Add test case for Bug#45377.
     @ storage/archive/azio.c
        Add error checking to ensure that the I/O operations are
        successful.
     @ storage/archive/ha_archive.cc
        Copy over the embedded FRM.
[29 Jul 2010 2:15] Davi Arnaut
Queued to mysql-trunk-bugfixing and up.
[4 Aug 2010 7:51] Bugs System
Pushed into mysql-trunk 5.5.6-m3 (revid:alik@sun.com-20100731131027-1n61gseejyxsqk5d) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 8:09] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804080001-bny5271e65xo34ig) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 8:25] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804081533-c1d3rbipo9e8rt1s) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 9:04] Bugs System
Pushed into mysql-next-mr (revid:alik@ibmvm-20100804081630-ntapn8bf9pko9vj3) (version source revid:marko.makela@oracle.com-20100621094008-o9fa153s3f09merw) (pib:20)
[16 Aug 2010 11:42] Tony Bedford
An entry has been added to the 5.5.6 changelog:

        The optimize method of the ARCHIVE storage engine did not
        preserve the FRM embedded in the ARZ file when rewriting the ARZ
        file for optimization. This meant an ARCHIVE table that had been
        optimized could not be discovered.

        The ARCHIVE engine stores the FRM in the ARZ file so it can be
        transferred from machine to machine without also needing to copy
        the FRM file. The engine subsequently restores the embedded FRM
        during discovery.