| Bug #39375 | Online backup crashes with simple combination of tables/triggers and SP | ||
|---|---|---|---|
| Submitted: | 10 Sep 2008 17:52 | Modified: | 11 Feb 2009 3:26 |
| Reporter: | Omer Barnir (OCA) | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Backup | Severity: | S3 (Non-critical) |
| Version: | 6.0.5 / 6.0.8 | OS: | Any |
| Assigned to: | Rafal Somla | CPU Architecture: | Any |
[10 Sep 2008 17:52]
Omer Barnir
[10 Sep 2008 17:56]
Omer Barnir
Dump of the systest1 database crashing online backup
Attachment: systest1.dump.tar.gz (application/x-gzip, text), 166.30 KiB.
[10 Sep 2008 17:57]
Omer Barnir
test file used to generate the schema used in creating systest1
Attachment: init.test (application/octet-stream, text), 15.67 KiB.
[26 Sep 2008 14:33]
Omer Barnir
The above init.test file references an include file setting some default values. Replace the include call line with the following: --disable_query_log eval SET @NUM_VAL = IF( '$NUM_VAL' = '', 15, '$NUM_VAL'); eval SET @LOAD_LINES = IF( '$LOAD_LINES' = '', 500, '$LOAD_LINES'); eval SET @LOG_UPPER = IF( '$LOG_UPPER' = '', 5000, '$LOG_UPPER'); eval SET @LOG_LOWER = IF( '$LOG_LOWER' = '', 4000, '$LOG_LOWER'); eval SET @ENG1 = IF( '$ENG1' = '', 'myisam', '$ENG1'); eval SET @ENG1_ALT = IF( '$ENG1_ALT' = '', 'innodb', '$ENG1_ALT'); eval SET @ENG2 = IF( '$ENG2' = '', 'innodb', '$ENG2'); eval SET @ENG_LOG = IF( '$ENG_LOG' = '', 'innodb', '$ENG_LOG'); eval SET @CLIENT_HOST = IF( '$CLIENT_HOST' = '', 'localhost', '$CLIENT_HOST'); eval SET @IUDS_UNIT = IF( '$IUDS_UNIT' = '', 10, '$IUDS_UNIT'); eval SET @TB1_ENG1_UPPER = IF( '$TB1_ENG1_UPPER' = '', @LOAD_LINES, '$TB1_ENG1_UPPER'); eval SET @WITH_TRIGGERS = IF( '$WITH_TRIGGERS' = '', 0, '$WITH_TRIGGERS'); eval SET @CYCLE = IF( '@CYCLE' = '', 60, '$CYCLE'); eval SET @ddl_sleep = IF( '$ddl_sleep' = '', 60, '$ddl_sleep'); --enable_query_log Sorry of missing it with the original upload
[26 Sep 2008 16:57]
Rafal Somla
Here is a gdb backtrace from the crash I managed to get:
#6 0x401e305f in __assert_fail () from /lib/tls/libc.so.6
#7 0x0826d767 in Item_func_in::val_int (this=0x9724eb0) at item_cmpfunc.cc:3820
#8 0x0846ce0f in get_all_tables (thd=0x9650e08, tables=0x4a4150b4, cond=0x9724eb0) at sql_show.cc:3327
#9 0x084e429b in (anonymous namespace)::open_schema_table (thd=0x9650e08, st=0x8ba1b08, db_list=0x4a415358) at si_objects.cc:323
#10 0x084e43b7 in obs::InformationSchemaIterator::prepare_is_table (thd=0x9650e08, is_table=0x4a415354, ha=0x4a415350,
orig_columns=0x4a41534c, is_table_idx=SCH_TABLES, db_list=@0x4a415358) at si_objects.cc:1061
#11 0x084e7441 in obs::create_is_iterator<obs::DbViewsIterator> (thd=0x9650e08, is_table_idx=SCH_TABLES, db_name=0x96d3110)
at si_objects.cc:1471
#12 0x084e4526 in obs::get_db_views (thd=0x9650e08, db_name=0x96d3110) at si_objects.cc:1512
#13 0x089bc546 in Backup_info::add_db_items (this=0x96e6710, db=@0x96d3108) at backup_info.cc:754
#14 0x089bcaaa in Backup_info::add_dbs (this=0x96e6710, dbs=@0x965263c) at backup_info.cc:554
#15 0x089a5efa in execute_backup_command (thd=0x9650e08, lex=0x9652038, backupdir=0x4a415c9c) at kernel.cc:186
Looks like the COND object passed from open_schema_table() to get_all_tables() is not completely valid. This COND is created by obs::create_db_select_condition() at #9.
[22 Dec 2008 13:19]
Rafal Somla
I'm not able to reproduce crash with the current tree. Please consider changing priority of this bug accordingly.
[22 Dec 2008 13:21]
Rafal Somla
Smallest test case with which I could reproduce problem.
Attachment: bug.test (application/octet-stream, text), 5.18 KiB.
[22 Dec 2008 13:23]
Rafal Somla
With the attached test case I do not observe crashes any more. However, the produced backup image is corrupted (inside meta-data storing section). RESTORE detects and reports error. Need to investigate cause of the corruption and remove it.
[23 Dec 2008 20:11]
Omer Barnir
triage: value updated based on description of not crashing anymore
[5 Jan 2009 13:15]
Rafal Somla
DETAILED PROBLEM DESCRIPTION
BACKUP command produces corrupted backup file in certain cases. This is due to a bug in the low level backup stream writing code (bstream library). The function bstream_write_part() in sql/backup/stream_v1_transport.c contains this code:
/*
To avoid copying bytes to the internal output buffer we try to cut a prefix
of the data to be written which forms a valid fragment and write this
fragment to output stream.
Note: after call to biggest_fragment_prefix() blob fragment contains the
bytes which didn't fit into the prefix.
*/
*(s->buf.header)= biggest_fragment_prefix(&fragment);
/*
We use this method only if it will actually write enough of the bytes
to be written - if it is only few bytes we save them into the output
buffer anyway.
*/
if (fragment.end > (s->buf.pos + MIN_WRITE_SIZE))
{
...
The bug is in the condition of the if statement. We want to check if the prefix determined by biggest_fragment_prefix() is bigger than MIN_WRITE_SIZE. However, at this moment blob fragment does not describe the prefix but the remainder of the data, as in this picture:
the prefix remainder
|------------|===========================|
^ ^ ^
buf.pos fragment.begin fragment.end
Thus the correct condition should be:
if (fragment.begin > (s->buf.pos + MIN_WRITE_SIZE))
{
...
This can be fixed trivially, however more work is needed to create good test for this problem.
[7 Jan 2009 14:44]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/62610 2744 Rafal Somla 2009-01-07 Bug #39375 - Online backup crashes with simple combination of tables/triggers and SP This patch fixes the internal buffering logic inside bstream_write_part() function (stream_v1_transport.c). Error in the logic could lead to corrupted backup image being written by BACKUP command. Attached test case tries to create such situation.
[8 Jan 2009 11:36]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/62675 2744 Rafal Somla 2009-01-08 Bug #39375 - Online backup crashes with simple combination of tables/triggers and SP This patch fixes the internal buffering logic inside bstream_write_part() function (stream_v1_transport.c). Error in the logic could lead to corrupted backup image being written by BACKUP command. Attached test case tries to create such situation.
[20 Jan 2009 11:37]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/63603 2752 Rafal Somla 2009-01-20 Bug #39375 - Online backup crashes with simple combination of tables/triggers and SP This patch fixes the internal buffering logic inside bstream_write_part() function (stream_v1_transport.c). Error in the logic could lead to corrupted backup image being written by BACKUP command. Attached test case tries to create such situation.
[20 Jan 2009 11:41]
Rafal Somla
Pushed into mysql-6.0-backup tree.
[2 Feb 2009 16:07]
Bugs System
Pushed into 6.0.10-alpha (revid:sergefp@mysql.com-20090202090240-dlkxhmc1asrar5rl) (version source revid:sergefp@mysql.com-20090129100938-qvke7a9krg24l8pl) (merge vers: 6.0.10-alpha) (pib:6)
[11 Feb 2009 3:26]
Paul DuBois
Noted in 6.0.10 changelog. The internal buffering logic for BACKUP DATABASE had a problem that could lead to corrupt backup images.
