Bug #59048 truncate table or create index could leave index->page to be FIL_NULL
Submitted: 20 Dec 2010 6:10 Modified: 7 Dec 2011 16:04
Reporter: Jimmy Yang Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S2 (Serious)
Version:5.5 OS:Any
Assigned to: Jimmy Yang CPU Architecture:Any

[20 Dec 2010 6:10] Jimmy Yang
Description:
There are cases that we could leave an index's metadata (rootpage info) in a corrupted situation with truncate table and/or create index if we are running out of space. index->page could left as FIL_NULL, and InnoDB refuse to load any table/index with index->page as FIL_NULL.

There are two cases to reproduce the problem, one is the truncate table, and the other is the create index case Sunny had described. In the truncate table case, it could trigger a crash, and then the index will have incorrect pageno, and this could affect Primary index. In the Sunny's case, it could easily corrupt the secondary index.

The first scenario:

In the truncate table case,  we can trigger the error in dict_truncate_index_tree() if btr_create() calls return FIL_NULL (no space). And the subsequent statistics update call with dict_stats_update() could crash the server:

#0  0x00fc5422 in __kernel_vsyscall ()
#1  0x009034d1 in *__GI_raise (sig=6)
   at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#2  0x00906932 in *__GI_abort () at abort.c:92
#3  0x0868625f in fil_io (type=10, sync=1, space_id=0, zip_size=0,
   block_offset=4294966143, byte_offset=0, len=16384, buf=0xb494c000,
   message=0xb46943b4)
   at /home/jy/work/nextmr/mr/storage/innobase/fil/fil0fil.c:4407
#4  0x0865005d in buf_read_page_low (err=0xb1cab034, sync=1, mode=132,
   space=0, zip_size=0, unzip=0, tablespace_version=1, offset=4294967295)
   at /home/jy/work/nextmr/mr/storage/innobase/buf/buf0rea.c:148
#5  0x0865012b in buf_read_page (space=0, zip_size=0, offset=4294967295)
   at /home/jy/work/nextmr/mr/storage/innobase/buf/buf0rea.c:188
#6  0x0863a1a3 in buf_page_get_gen (space=0, zip_size=0, offset=4294967295,
   rw_latch=2, guess=0x0, mode=10,
   file=0x88a3a9c "/home/jy/work/nextmr/mr/storage/innobase/btr/btr0btr.c",
   line=140, mtr=0xb1cab1b0)
   at /home/jy/work/nextmr/mr/storage/innobase/buf/buf0buf.c:2974
#7  0x0860c207 in btr_block_get_func (space=0, zip_size=0, page_no=4294967295,
   mode=2,
   file=0x88a3a9c "/home/jy/work/nextmr/mr/storage/innobase/btr/btr0btr.c",
   line=140, mtr=0xb1cab1b0)
   at /home/jy/work/nextmr/mr/storage/innobase/include/btr0btr.ic:55
---Type <return> to continue, or q <return> to quit---
#8  0x0860cd60 in btr_root_block_get (index=0x902d258, mtr=0xb1cab1b0)
   at /home/jy/work/nextmr/mr/storage/innobase/btr/btr0btr.c:140
#9  0x0860ce7f in btr_root_get (index=0x902d258, mtr=0xb1cab1b0)
   at /home/jy/work/nextmr/mr/storage/innobase/btr/btr0btr.c:167
#10 0x0860d646 in btr_get_size (index=0x902d258, flag=2)
   at /home/jy/work/nextmr/mr/storage/innobase/btr/btr0btr.c:424
#11 0x08676023 in dict_stats_update_transient (table=0x90327e8)
   at /home/jy/work/nextmr/mr/storage/innobase/dict/dict0stats.c:171
#12 0x08679698 in dict_stats_update (table=0x90327e8,
   stats_upd_option=DICT_STATS_RECALC_PERSISTENT_SILENT,

When you reboot, the index would have its page number set to FIL_NULL, and you will not be able to load the index and table:

InnoDB: Error: trying to load index GEN_CLUST_INDEX for table test/test1
InnoDB: but the index tree has been freed!
101219 20:21:36 [ERROR] Cannot find or open table test/test1 from
the internal data dictionary of InnoDB though the .frm file for the
table exists. Maybe you have deleted and recreated InnoDB data
files but have forgotten to delete the corresponding .frm files

The second scenario is a problem with error handling on the create index. Seems this is for secondary index, I need to check if this can happen on primary index (we simply deallocate the temp table in primary index case).

The calling stack in question is ha_innobase::add_index -> row_merge_create_index->row_merge_create_index_graph->...->dict_create_index_step->dict_create_index_tree_step

In dict_create_index_step(), the index is created for SYS_INDEXES, and then it enters into dict_create_index_tree_step() to create index root page. If it fails, it will discard the index metadata (dict_index_t) in memory, however, the SYS_INDEX row is still there. ha_innobase::add_index in fact doing the error processing by  calling row_merge_drop_indexes:

                       if (error != DB_SUCCESS) {
                               row_merge_drop_indexes(trx, indexed_table,
                                                      index, num_created);
                       }

Unfortunately, since dict_create_index_step() calls dict_index_remove_from_cache() to remove the in memory index metadata, it has nothing to remove. Thus left the SYS_INDEXES row on disk. So there is a crash recovery, we will see it restore the FIL_NULL rootpage for the index and then we will see:

InnoDB: Error: trying to load index idx for table test/ut
InnoDB: but the index tree has been freed!
101219 19:45:01 [ERROR] Cannot find or open table test/ut from
the internal data dictionary of InnoDB though the .frm file for the
...

How to repeat:
See above, use a debugger to force btr_create return FIL_NULL

Suggested fix:
Correct error handling
[9 Feb 2011 7:43] Jimmy Yang
Fix checked in mysql-trunk-innodb
[9 Feb 2011 12:07] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/130857

3489 Jimmy Yang	2011-02-08
      Fix Bug #59048 truncate table or create index could leave index->page
      to be FIL_NULL
      
      rb://545, approved by Sunny Bains
[5 Sep 2012 7:54] MySQL Verification Team
This fix is in 5.5.11 release