Bug #34902 Backup: crash if view depends on dropped object
Submitted: 27 Feb 2008 19:39 Modified: 8 Sep 2008 18:05
Reporter: Peter Gulutzan Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Backup Severity:S3 (Non-critical)
Version:6.0.5-alpha-debug OS:Linux (SUSE 10 | 32-bit)
Assigned to: Jørgen Løland CPU Architecture:Any

[27 Feb 2008 19:39] Peter Gulutzan
Description:
I'm using mysql-6.0-backup.

I create a base table.
I create a view of the base table.
I drop the base table.
I say BACKUP DATABASE.
Crash.

Or:
I create a function.
I create a view that uses the function.
I drop the database containing the function.
I say BACKUP DATABASE.
Crash.

How to repeat:
create database d2;
use d2
create table v (s1 int);
create view v2 as select * from v;
drop table v;
backup database d2 to '3';

or

create database d7;
create function d7.f () returns int return 5;
create database d8;
create view d8.v as select d7.f();
drop database d7;
backup database d8 to '10';
[27 Feb 2008 21:16] MySQL Verification Team
Thank you for the bug report. Verified as described.

080227 18:13:07 [Note] Backup: Backup completed
mysqld: sql_class.cc:425: void Diagnostics_area::set_eof_status(THD*): Assertion `! is_set()' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 1158191440 (LWP 25558)]
0x0000003bfca30ec5 in raise () from /lib64/libc.so.6
(gdb)
[21 May 2008 17:52] Rafal Somla
Here is a call stack which I get for the first test:

#3  <signal handler called>
#4  0x08c24cb0 in ?? ()
#5  0x0879a85e in ~Obj (this=0x8c23320) at image_info.h:362
#6  0x0879aa4b in ~Dbobj (this=0x8c23320) at image_info.h:451
#7  0x087ac650 in ~Image_info (this=0x8c41ff0) at image_info.cc:81
#8  0x087ae63a in ~Backup_info (this=0x8c41ff0) at backup_info.cc:360
#9  0x0879814b in ~Backup_restore_ctx (this=0x47c284c4) at kernel.cc:342
#10 0x08799214 in execute_backup_command (thd=0x8bca708, lex=0x8bcb7ac) at kernel.cc:215
#11 0x082afec4 in mysql_execute_command (thd=0x8bca708) at sql_parse.cc:2153
#12 0x082b88a8 in mysql_parse (thd=0x8bca708, inBuf=0x8c2da68 "backup database d2 to '3'", length=25, 
    found_semicolon=0x47c29310) at sql_parse.cc:5747
#13 0x082b9300 in dispatch_command (command=COM_QUERY, thd=0x8bca708, 
    packet=0x8c1ee39 "backup database d2 to '3'", packet_length=25) at sql_parse.cc:1045
#14 0x082ba560 in do_command (thd=0x8bca708) at sql_parse.cc:722
#15 0x082a7b9b in handle_one_connection (arg=0x8bca708) at sql_connect.cc:1134
#16 0x400270bd in start_thread () from /lib/tls/libpthread.so.0
#17 0x402779ee in clone () from /lib/tls/libc.so.6

The line image_info.h:362 is

359: Image_info::Obj::~Obj()
360: {
361:  // Delete corresponding server object if present.
362:   delete m_obj_ptr;
363: }

(gdb) p m_obj_ptr
$1 = (class obs::Obj *) 0x8c24cb8
(gdb) p *m_obj_ptr
warning: can't find linker symbol for virtual table for `obs::Obj' value
$2 = {_vptr.Obj = 0x402db5b0}

Server crashes since it tries to execute invalid instruction and apparently m_obj_ptr is a dangling pointer.

So far I have no idea why this happens...
[5 Jun 2008 14:28] Jørgen Løland
Testing with this script:

CREATE DATABASE d2;
USE d2;
CREATE TABLE v (s1 int);
CREATE TABLE x (s2 int);
CREATE VIEW v2 AS SELECT * FROM v JOIN x on v.s1=x.s2;
DROP TABLE v;
BACKUP DATABASE d2 TO '3';

The crash happens because we try to open a table that does not exist:

#0  open_table (thd=0x8cd8560, table_list=0x8d160a8, mem_root=0xb0391de4, 
    refresh=0xb0391e37, flags=0) at sql_base.cc:2497
#1  0x0833616a in open_tables (thd=0x8cd8560, start=0xb0391ea0, 
    counter=0xb0391ea8, flags=0) at sql_base.cc:4518
#2  0x084e3b83 in obs::ViewBaseObjectsIterator::create (thd=0x8c4be50, 
    db_name=0x8ce24bc, view_name=0x8ce24d0, 
    iterator_type=obs::ViewBaseObjectsIterator::GET_BASE_VIEWS)
    at si_objects.cc:1353
#3  0x084e3df1 in obs::get_view_base_views (thd=0x8c4be50, db_name=0x8ce24bc, 
    view_name=0x8ce24d0) at si_objects.cc:1558
#4  0x0882411d in Backup_info::add_view_deps (this=0x8cd03f0, obj=@0x8ce24b8)
    at backup_info.cc:889
#5  0x088245de in Backup_info::add_db_object (this=0x8cd03f0, db=@0x8cbddd8, 
    type=BSTREAM_IT_VIEW, obj=0x8ce24b8) at backup_info.cc:1044
#6  0x088246f2 in Backup_info::add_objects (this=0x8cd03f0, db=@0x8cbddd8, 
    type=BSTREAM_IT_VIEW, it=@0x8cbf880) at backup_info.cc:639
#7  0x08825f68 in Backup_info::add_db_items (this=0x8cd03f0, db=@0x8cbddd8)
    at backup_info.cc:738
#8  0x0882646b in Backup_info::add_dbs (this=0x8cd03f0, dbs=@0x8c4d51c)
    at backup_info.cc:530
#9  0x0880c82f in execute_backup_command (thd=0x8c4be50, lex=0x8c4cf1c)
    at kernel.cc:163

(#0) open_table returns NULL (i.e, "open failed") when processing table
d2.v. When open_tables (#1) gets a NULL return, it stops opening
tables and returns an error code (-1):

4518            tables->table= open_table(thd, tables, &new_frm_mem, &refresh, flags);
4525        if (!tables->table)
4527          free_root(&new_frm_mem, MYF(MY_KEEP_PREALLOC));
4529          if (tables->view)
4560          if (tables->parent_l)
4568          if (refresh)                              // Refresh in progress
4590          if (safe_to_ignore_table)
4597          result= -1;                               // Fatal error
(...)
4713      DBUG_RETURN(result);

Possible paths to a fix:

 - After building the TABLE_LIST tl in
   ViewBaseObjectsIterator::create, non-existing tables can be
   removed before calling sql_base.cc#open_tables. This can, e.g,
   be done by using sql_base.cc#check_if_table_exists. Note that
   in this case, the transaction needs to own LOCK_open mutex and
   must have checked that there are no name-locks on the table.
 - Modify sql_base.cc#open_tables to ignore non-existing tables
   if a flag has been set.
 - Find a way to not call open_tables at all - not sure yet why
   we need to open the tables when all we want is the view
   definition?
[11 Jun 2008 11:51] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/47726

2630 Jorgen Loland	2008-06-11
      Bug#34902: "Backup: crash if view depends on dropped object"
      
      Pre-fix behavior: When adding a view to the backup image fails, 
      the object representing the view is deleted but not removed 
      from the database catalog. This view object is later tried deleted
      as part of catalog delete.
      
      Fix: If adding an object to the catalog fails, the object is 
      removed from the catalog before deleting the object.
[11 Jun 2008 11:56] Jørgen Løland
If I run the above script through the client, I get: 

mysql> CREATE DATABASE d2;
Query OK, 1 row affected (0.00 sec)

mysql> USE d2;
Database changed
mysql> CREATE TABLE v (s1 int);
Query OK, 0 rows affected (0.03 sec)

mysql> CREATE TABLE x (s2 int);
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE VIEW v2 AS SELECT * FROM v;
Query OK, 0 rows affected (0.01 sec)

mysql> DROP TABLE v;
Query OK, 0 rows affected (0.00 sec)

mysql> BACKUP DATABASE d2 TO '3';
ERROR 1683 (HY000): Failed to add view `d2`.`v2` to the catalog
mysql> 

Thus, the originally reported problem is fixed by the patch.

However, with this script, mtr reports:

Warning: Not freed memory segments: 1
Warning: Memory that was not free'ed (256 bytes):
	   256 bytes at 0x00813fdc0, allocated at line   39 in 'string.c'

When I add this ...

-------- bzr diff --------
@@ -6386,14 +6387,19 @@ void run_query(struct st_connection *cn,
 
   init_dynamic_string(&ds_warnings, NULL, 0, 256);
 
+  DBUG_PRINT("mydbg", ("mysqltest.c - ds_warning: %p", ds_warnings));
+
   /* Scan for warning before sending to server */
   scan_command_for_warnings(command);
-------- bzr diff --------

to mysqltest.c, the trace file reports this:

(...)
>run_query
| >init_dynamic_string
| | >_mymalloc
| | | enter: Size: 256
| | | exit: ptr: 0x813fdc0
| | <_mymalloc
| <init_dynamic_string
| mydbg: mysqltest.c - ds_warning: 0x813fdc0
| >scan_command_for_warnings
| | enter: query: BACKUP DATABASE d2 TO '3'
| <scan_command_for_warnings
| >run_query_normal
(...)
User time 0.00, System time 0.01
Maximum resident set size 0, Integral resident set size 0
Non-physical pagefaults 1305, Physical pagefaults 0, Swaps 0
Blocks in 0 out 112, Messages in 0 out 0, Signals 0
Voluntary context switches 19, Involuntary context switches 20
| | | | | >TERMINATE
| | | | | | safe: sf_malloc_count: 1
| | | | | | safe: Memory that was not free'ed (256 bytes):
| | | | | | safe:    256 bytes at 0x00813fdc0, allocated at line   39 in 'string.c'
| | | | | | safe: Maximum memory usage: 134010 bytes (131k)
| | | | | <TERMINATE

Hence, ds_warnings (mysqltest.c#run_query) owns the non-freed memory (note the memory addresses), and that it would have been freed if the thread reached the end of the run_query method. So far, I have no
idea why run_query_normal does not return.

Conclusion:
The attached patch fixes the originally reported problem
There is another bug somewhere in mysqltest.c
[11 Jun 2008 12:34] Rafal Somla
I think that in the case when we backup a view whose base table doesn't exist we should not interrupt BACKUP commad with an error but store views definition regardless and continue.

This would be more consistent with the case when we backup a view whose base table is in a separate database not included in the backup image. In that case BACKUP doesn't singal error and completes its work. It is leaved to the user to ensure that all dependencies are satisfied if he needs it. But it is also possible to restore a view with broken dependencies (i.e. one of the base tables doesn't exist because it was in a database which was not backed up).
[11 Jun 2008 17:26] Chuck Bell
I see the merit in attempting to include the view in the backup. But this presents two problems that must be overcome:

1) How do we suppress the errors that occur during open tables calls?
   Possible answer: reset the diagnostic area after processing metadata
2) How do we allow the CREATE to succeed on restore? Currently, the code will thrown an error and stop. This will need to be changed else backing up a view whose dependencies are missing would be useless (can't restore it).

If 2) cannot be made to work then we cannot permit backup of a view whose dependencies are missing and therefore the original patch for this bug report must remain: throw an error on backup and stop.
[30 Jul 2008 7:43] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50692

2673 Jorgen Loland	2008-07-30
      Bug#34902: "Backup: crash if view depends on dropped object"
            
      Pre-fix behavior: When adding a view to the backup image fails, 
      the object representing the view is deleted but not removed 
      from the database catalog. This view object is later tried deleted
      as part of catalog delete.
            
      Fix: Add the object and all its dependencies to dependency list
      before adding the object to the catalog. If any dependant objects
      fail to be added to dep list, the object is not added to the
      catalog.
[31 Jul 2008 11:11] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50775

2673 Jorgen Loland	2008-07-31
      Bug#34902: "Backup: crash if view depends on dropped object"
      
      Pre-fix behavior: When adding a view to the backup image fails, 
      the object representing the view is deleted but not removed 
      from the database catalog. This view object is later tried deleted
      as part of catalog delete.
                  
      Fix: Add the object and all its dependencies to dependency list
      before adding the object to the catalog. If any dependant objects
      fail to be added to dep list, the object is not added to the
      catalog.
[31 Jul 2008 11:39] Rafal Somla
Good to push.
[4 Aug 2008 20:07] Chuck Bell
Patch approved.
[5 Aug 2008 8:04] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50907

2674 Jorgen Loland	2008-08-05
      Bug#34902: "Backup: crash if view depends on dropped object"
            
      Pre-fix behavior: When adding a view to the backup image fails, 
      the object representing the view is deleted but not removed 
      from the database catalog. This view object is later tried deleted
      as part of catalog delete.
                        
      Fix: Add the object and all its dependencies to dependency list
      before adding the object to the catalog. If any dependant objects
      fail to be added to dep list, the object is not added to the
      catalog.
[5 Aug 2008 8:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50909

2674 Jorgen Loland	2008-08-05
      Bug#34902: "Backup: crash if view depends on dropped object"
            
      Pre-fix behavior: When adding a view to the backup image fails, 
      the object representing the view is deleted but not removed 
      from the database catalog. This view object is later tried deleted
      as part of catalog delete.
                        
      Fix: Add the object and all its dependencies to dependency list
      before adding the object to the catalog. If any dependant objects
      fail to be added to dep list, the object is not added to the
      catalog.
[8 Sep 2008 18:05] Paul DuBois
Noted in 6.0.7 changelog.

If a view depended on a base table that had been dropped, BACKUP
DATABASE caused a server crash.
[14 Sep 2008 2:48] Bugs System
Pushed into 6.0.7-alpha  (revid:jorgen.loland@sun.com-20080805080430-bk4d4eytvutz9izr) (version source revid:sven@mysql.com-20080818175803-c1nutd5773r6b4gd) (pib:3)