Bug #49146 RESTORE:Server crashes when attempt to restore PBXT table while pbxt is disabled
Submitted: 26 Nov 2009 18:49 Modified: 7 Mar 2010 19:05
Reporter: Alexey Stroganov Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Backup Severity:S1 (Critical)
Version:6.0.14-backup(2814) OS:Any
Assigned to: Chuck Bell CPU Architecture:Any

[26 Nov 2009 18:49] Alexey Stroganov
Description:
Attempt to restore PBXT table while PBXT engine was disabled causes server crash with following back trace:

(gdb) bt
#0  0x00002b5835befea3 in pthread_kill () from /lib64/libpthread.so.0
#1  0x000000000061193e in handle_segfault (sig=11) at mysqld.cc:2765
#2  <signal handler called>
#3  bcat_reset (catalogue=<value optimized out>) at ../../sql/backup/backup_aux.h:45
#4  0x0000000000a002cc in bstream_rd_catalogue (s=0x1487678, cat=0x14270a0) at stream_v1.c:812
#5  0x00000000009f6ed4 in Backup_restore_ctx::prepare_for_restore (this=0x4b094850, backupdir=<value optimized out>,
    orig_loc={str = 0x147b348 "/tmp/pbxt.bk", length = 12}, query=<value optimized out>, skip_gap_event=false)
    at stream.h:251
#6  0x00000000009f8109 in execute_backup_command (thd=0x1467f90, lex=0x14697c8, backupdir=0x4b095df0, overwrite=true,
    skip_gap_event=false) at kernel.cc:284
#7  0x00000000006232f7 in mysql_execute_command (thd=0x1467f90) at sql_parse.cc:2480
#8  0x0000000000627065 in mysql_parse (thd=0x1467f90, inBuf=0x147b278 "restore from '/tmp/pbxt.bk' overwrite", length=37,
    found_semicolon=0x4b097120) at sql_parse.cc:5983
#9  0x0000000000627f64 in dispatch_command (command=COM_QUERY, thd=0x1467f90,
    packet=0x1473241 "restore from '/tmp/pbxt.bk' overwrite", packet_length=<value optimized out>) at sql_parse.cc:1078
#10 0x000000000061b555 in handle_one_connection (arg=<value optimized out>) at sql_connect.cc:1164
#11 0x00002b5835beb143 in start_thread () from /lib64/libpthread.so.0
#12 0x00002b583646374d in clone () from /lib64/libc.so.6
#13 0x0000000000000000 in ?? ()

How to repeat:
1. Compile and start server with PBXT engine

2. Create PBXT table and BACKUP database
   use test; 
   create t_pbxt(a int) engine=pbxt;
   backup database test to '/tmp/pbxt.bk'

3. Restart server with --skip-pbxt

4. connect to server and execute:
   RESTORE from '/tmp/pbxt.bk' overwrite;

5. Server crashes
[9 Dec 2009 17:11] Chuck Bell
ANALYSIS
--------
The problem is in the backup code:

    case BI_NATIVE:
    {
      backup::LEX_STRING name_lex(snap->engine.name.begin, 
                                  snap->engine.name.end);
-1->  storage_engine_ref se= get_se_by_name(name_lex);
      handlerton *hton= se_hton(se);

      /* Error condition insertion for ER_BACKUP_CANT_FIND_SE. */
      DBUG_EXECUTE_IF("ER_BACKUP_CANT_FIND_SE", se= 0;);

-2->  if (!se || !hton)
      {
        log.report_error(ER_BACKUP_CANT_FIND_SE, name_lex.str);
        return BSTREAM_ERROR;
      }

The highlighted line (-1->) returns an invalid result when the storage engine is not found. The gate (-2->) protects against a NULL result but too late. The invalid pointer is dereferenced in the constructor which causes the system crash.

SOLUTIONS
---------
Possibilities:
* Fix get_se_by_name() to return NULL and/or the constructor to return NULL.
* Move the gate up and detect error earlier.
[9 Dec 2009 17:15] Chuck Bell
Here's the culprit:

inline
handlerton* se_hton(storage_engine_ref se)
{ return (handlerton*)(se->data); }

This code does not check for NULL and attempts to read the memory location. .oO
[9 Dec 2009 17:19] Chuck Bell
Fixing this logic error solves the problem. The code will correctly identify the missing storage engine as shown below.

mysql> restore from 'p1.bak' overwrite;
ERROR 1695 (HY000): Backup image contains data created by a native driver of PBXT storage engine but this engine can not be found on this server
[9 Dec 2009 17:41] Chuck Bell
Fix:

--- sql/backup/backup_aux.h	2009-12-02 08:01:43 +0000
+++ sql/backup/backup_aux.h	2009-12-09 17:38:22 +0000
@@ -42,7 +42,12 @@ uint se_version(storage_engine_ref se)
 
 inline
 handlerton* se_hton(storage_engine_ref se)
-{ return (handlerton*)(se->data); }
+{ 
+  if (se)
+    return (handlerton*)(se->data);
+  else 
+    return NULL;
+}
[9 Dec 2009 18:39] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/93382

2904 Chuck Bell	2009-12-09
      BUG#49146 : RESTORE:Server crashes when attempt to restore PBXT
                  table while pbxt is disabled
      
      The problem occurs when a storage engine that has a native 
      backup driver is no longer on the system (e.g. the plugin was 
      unloaded or the storage engine does not exis). In this case,
      when a backup image is restored that contains tables from the
      missing storage engine with the native driver, the system will 
      crash because of a logic error in how the storage
      engine reference is found. If a storage engine search returned
      NULL, the code was not detecting this and instead trying to 
      reference an attribute without checking the pointer for NULL.
      
      This patch corrects the logic error by detecting if the storage
      engine search returned NULL. A test case is added to ensure
      this problem is detected. A new backup image file that contains 
      one PBXT table is also included as input to the test case.
     @ mysql-test/suite/backup/r/restore_missing_native_driver.result
        New result file.
     @ mysql-test/suite/backup/std_data/backup_pbxt.bak
        New backup image containing one table from a PBXT native backup.
     @ mysql-test/suite/backup/t/restore_missing_native_driver.test
        New test to test missing native driver in backup image.
     @ sql/backup/backup_aux.h
        Changed code to check for NULL se attribute.
[10 Dec 2009 10:24] Rafal Somla
Approved.
[10 Dec 2009 14:18] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/93518

2905 Chuck Bell	2009-12-10
      BUG#49146 : RESTORE:Server crashes when attempt to restore PBXT
                  table while pbxt is disabled
      
      The problem occurs when a storage engine that has a native 
      backup driver is no longer on the system (e.g. the plugin was 
      unloaded or the storage engine does not exis). In this case,
      when a backup image is restored that contains tables from the
      missing storage engine with the native driver, the system will 
      crash because of a logic error in how the storage
      engine reference is found. If a storage engine search returned
      NULL, the code was not detecting this and instead trying to 
      reference an attribute without checking the pointer for NULL.
      
      This patch corrects the logic error by detecting if the storage
      engine search returned NULL. A test case is added to ensure
      this problem is detected. A new backup image file that contains 
      one PBXT table is also included as input to the test case.
     @ mysql-test/suite/backup/r/restore_missing_native_driver.result
        New result file.
     @ mysql-test/suite/backup/std_data/backup_pbxt.bak
        New backup image containing one table from a PBXT native backup.
     @ mysql-test/suite/backup/t/restore_missing_native_driver.test
        New test to test missing native driver in backup image.
     @ sql/backup/backup_aux.h
        Changed code to check for NULL se attribute.
[10 Dec 2009 14:19] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/93519
[10 Dec 2009 14:54] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/93534

2907 Chuck Bell	2009-12-10
      BUG#49146 : RESTORE:Server crashes when attempt to restore PBXT
                  table while pbxt is disabled
      
      Removed platform dependent comment found during merge.
     @ mysql-test/suite/backup/r/restore_missing_native_driver.result
        Fixed result file.
     @ mysql-test/suite/backup/t/restore_missing_native_driver.test
        Removed platform dependent comment.
[14 Dec 2009 14:22] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/93919

2907 Chuck Bell	2009-12-14
      BUG#49146 : RESTORE:Server crashes when attempt to restore PBXT
                  table while pbxt is disabled
      
      The problem occurs when a storage engine that has a native 
      backup driver is no longer on the system (e.g. the plugin was 
      unloaded or the storage engine does not exis). In this case,
      when a backup image is restored that contains tables from the
      missing storage engine with the native driver, the system will 
      crash because of a logic error in how the storage
      engine reference is found. If a storage engine search returned
      NULL, the code was not detecting this and instead trying to 
      reference an attribute without checking the pointer for NULL.
      
      This patch corrects the logic error by detecting if the storage
      engine search returned NULL. A test case is added to ensure
      this problem is detected. A new backup image file that contains 
      one PBXT table is also included as input to the test case.
     @ mysql-test/suite/backup/r/restore_missing_native_driver.result
        New result file.
     @ mysql-test/suite/backup/std_data/backup_pbxt.bak
        New backup image containing one table from a PBXT native backup.
     @ mysql-test/suite/backup/t/restore_missing_native_driver.test
        New test to test missing native driver in backup image.
     @ sql/backup/backup_aux.h
        Changed code to check for NULL se attribute.
[15 Dec 2009 20:25] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/94271

2911 Chuck Bell	2009-12-15
      BUG#49146 : RESTORE:Server crashes when attempt to restore PBXT
                  table while pbxt is disabled
      
      The problem occurs when a storage engine that has a native 
      backup driver is no longer on the system (e.g. the plugin was 
      unloaded or the storage engine does not exis). In this case,
      when a backup image is restored that contains tables from the
      missing storage engine with the native driver, the system will 
      crash because of a logic error in how the storage
      engine reference is found. If a storage engine search returned
      NULL, the code was not detecting this and instead trying to 
      reference an attribute without checking the pointer for NULL.
      
      This patch corrects the logic error by detecting if the storage
      engine search returned NULL. A test case is added to ensure
      this problem is detected. A new backup image file that contains 
      one PBXT table is also included as input to the test case.
     @ mysql-test/suite/backup/r/restore_missing_native_driver.result
        New result file.
     @ mysql-test/suite/backup/std_data/backup_pbxt.bak
        New backup image containing one table from a PBXT native backup.
     @ mysql-test/suite/backup/t/restore_missing_native_driver.test
        New test to test missing native driver in backup image.
     @ sql/backup/backup_aux.h
        Changed code to check for NULL se attribute.
[20 Feb 2010 9:19] Bugs System
Pushed into 6.0.14-alpha (revid:ingo.struewing@sun.com-20100218152520-s4v1ld76bif06eqn) (version source revid:ingo.struewing@sun.com-20100119103538-wtp5alpz4p2jayl5) (merge vers: 6.0.14-alpha) (pib:16)
[7 Mar 2010 19:05] Paul DuBois
Noted in 6.0.14 changelog.

RESTORE crashed if a native backup driver required a storage engine
that was not loaded.