Bug #56134 mysql_admin_table should check the return value of open_and_lock_tables
Submitted: 20 Aug 2010 6:16 Modified: 11 May 2011 8:41
Reporter: Yuan WANG Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: DDL Severity:S3 (Non-critical)
Version:5.1.49 OS:Any
Assigned to: Jon Olav Hauglid CPU Architecture:Any

[20 Aug 2010 6:16] Yuan WANG
Description:
We found a problem of function mysql_admin_table() when developing a customized storage engine. mysql_admin_table() calls open_and_lock_tables(), as shown in the following codes.

      if (view_operator_func == NULL)
        table->required_type=FRMTYPE_TABLE;

      open_and_lock_tables(thd, table);
      thd->no_warnings_for_error= 0;

Our storage engine may fail in locking tables due to lock wait timeout. In this case, we return error in external_lock() and we thought then MySQL will abort the operation. However, we found that even when our storage engine return error in external_lock(), MySQL will continue. We read the codes and we think the problem is in the codes above in function mysql_admin_table(), where open_and_lock_tables() is called to lock the table but the return value of it is not checked. We think the return value of open_and_lock_tables() should be checked and if it failed MySQL should abort the operation.

How to repeat:
We currently are not able to construct a test case with other storage engines.

Suggested fix:
Check the return value of open_and_lock_tables() and abort the operation if it fails.
[21 Aug 2010 11:48] Sveta Smirnova
Thank you for the report.

Verified as described.

To repeat modify sources as follow:

$bzr diff
=== modified file 'storage/myisam/ha_myisam.cc'
--- storage/myisam/ha_myisam.cc 2010-03-02 09:45:50 +0000
+++ storage/myisam/ha_myisam.cc 2010-08-21 10:47:21 +0000
@@ -1879,6 +1879,7 @@
 
 int ha_myisam::external_lock(THD *thd, int lock_type)
 {
+  DBUG_EXECUTE_IF("get_error_in_mysql_admin_table", {return HA_ADMIN_INTERNAL_ERROR;});
   return mi_lock_database(file, !table->s->tmp_table ?
                          lock_type : ((lock_type == F_UNLCK) ?
                                       F_UNLCK : F_EXTRA_LCK));

Then create test for MTR and run it:

create table t1(f1 int);
insert into t1 values(1),(2),(3);
SET SESSION debug="+d,get_error_in_mysql_admin_table";
optimize table t1;

You get:

=====mysql-5.1=====
=====bug56134=====
create table t1(f1 int);
insert into t1 values(1),(2),(3);
SET SESSION debug="+d,get_error_in_mysql_admin_table";
optimize table t1;
Table   Op      Msg_type        Msg_text
test.t1 optimize        Error   Can't lock file (errno: -4)
test.t1 optimize        status  OK

Why status is OK?
[11 May 2011 8:41] Jon Olav Hauglid
In 5.5+ we check the result of open_and_lock_tables() in mysql_admin_table().
Closing as "Can't reproduce" since the bug was triaged to be fixed in 5.6+ only.