| Bug #47073 | valgrind errs, corruption,failed repair of partition,low myisam_sort_buffer_size | ||
|---|---|---|---|
| Submitted: | 2 Sep 2009 19:14 | Modified: | 18 Dec 2009 13:23 |
| Reporter: | Shane Bester (Platinum Quality Contributor) | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: MyISAM storage engine | Severity: | S3 (Non-critical) |
| Version: | 5.0.90, 5.1.37, 5.1.39 | OS: | Any |
| Assigned to: | Sergey Vojtovich | CPU Architecture: | Any |
| Tags: | corruption, myisam_sort_buffer_size, repair table, valgrind | ||
[3 Sep 2009 3:32]
Valeriy Kravchuk
Thank you for the bug report. Verified just as described with recent 5.1.39 from bzr on Mac OS X.
[8 Sep 2009 7:38]
Sergey Vojtovich
We have two different problems here: - REPAIR doesn't fall back to key cache, leaving table in an inconsistent state - valgrind errors, which are coming from bulk inserts I expect easy fixes for both problems, but it may take some time to find real cause. Both problems should be pure MyISAM, at least I was able to repeat REPAIR issue without partitioning.
[6 Oct 2009 17:35]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/85949 3146 Sergey Vojtovich 2009-10-06 BUG#47073 - valgrind errs, corruption,failed repair of partition, low myisam_sort_buffer_size Repair by sort (default) or parallel repair of a MyISAM table (doesn't matter partitioned or not) as well as bulk inserts and enable indexes some times didn't failover to repair with key cache. The problem was that after unsuccessful attempt, data file was closed. Whereas repair with key cache requires open data file. Fixed by reopening data file. Also fixed a valgrind warning, which may appear during repair by sort or parallel repair with certain myisam_sort_buffer_size number of rows and length of an index entry (very dependent). @ mysql-test/r/myisam.result A test case for BUG#47073. @ mysql-test/t/myisam.test A test case for BUG#47073. @ storage/myisam/mi_check.c Reopen data file, when repair by sort or parallel repair fails. When repair by sort is requested to rebuild data file, data file gets rebuilt while fixing first index. When rebuild is completed, info->dfile is pointing to temporary data file, original data file is closed. It may happen that repair has successfully fixed first index and rebuilt data file, but failed to fix second index. E.g. myisam_sort_buffer_size was big enough to fix first shorter index, but not enough to fix subsequent longer index. In this case we end up with info->dfile pointing to temporary file, which is removed and info->dfile is set to -1. Though repair by sort failed, the upper layer may still want to try repair with key cache. But it needs info->dfile pointing to valid data file. @ storage/myisam/sort.c When performing a copy of IO_CACHE structure, current_pos and current_end must be updated separatly to point to memory we're copying to (not to memory we're copying from). As t_file2 is always WRITE cache, proper members are write_pos and write_end accordingly.
[9 Oct 2009 16:16]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/86429 3146 Sergey Vojtovich 2009-10-09 BUG#47073 - valgrind errs, corruption,failed repair of partition, low myisam_sort_buffer_size Repair by sort (default) or parallel repair of a MyISAM table (doesn't matter partitioned or not) as well as bulk inserts and enable indexes some times didn't failover to repair with key cache. The problem was that after unsuccessful attempt, data file was closed. Whereas repair with key cache requires open data file. Fixed by reopening data file. Also fixed a valgrind warning, which may appear during repair by sort or parallel repair with certain myisam_sort_buffer_size number of rows and length of an index entry (very dependent). @ mysql-test/r/myisam.result A test case for BUG#47073. @ mysql-test/t/myisam.test A test case for BUG#47073. @ storage/myisam/ha_myisam.cc Reverted fix for BUG25289. Not needed anymore. @ storage/myisam/mi_check.c Reopen data file, when repair by sort or parallel repair fails. When repair by sort is requested to rebuild data file, data file gets rebuilt while fixing first index. When rebuild is completed, info->dfile is pointing to temporary data file, original data file is closed. It may happen that repair has successfully fixed first index and rebuilt data file, but failed to fix second index. E.g. myisam_sort_buffer_size was big enough to fix first shorter index, but not enough to fix subsequent longer index. In this case we end up with info->dfile pointing to temporary file, which is removed and info->dfile is set to -1. Though repair by sort failed, the upper layer may still want to try repair with key cache. But it needs info->dfile pointing to valid data file. @ storage/myisam/sort.c When performing a copy of IO_CACHE structure, current_pos and current_end must be updated separatly to point to memory we're copying to (not to memory we're copying from). As t_file2 is always WRITE cache, proper members are write_pos and write_end accordingly.
[14 Oct 2009 14:39]
Bugs System
Pushed into 5.1.41 (revid:joro@sun.com-20091014143611-cphb0enjlx6lpat1) (version source revid:satya.bn@sun.com-20091013071829-zc4c3go44j6re592) (merge vers: 5.1.40) (pib:13)
[15 Oct 2009 7:31]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/86898 3170 Sergey Vojtovich 2009-10-15 Disabled part of test for BUG#47073 until additional fix is pushed. @ mysql-test/r/myisam.result Disabled part of test for BUG#47073 until additional fix is pushed. @ mysql-test/t/myisam.test Disabled part of test for BUG#47073 until additional fix is pushed.
[15 Oct 2009 11:49]
Bugs System
Pushed into 5.1.41 (revid:joro@sun.com-20091015114812-i3gm8km6gfruny5x) (version source revid:svoj@sun.com-20091015073111-t5qm270p7noyrx53) (merge vers: 5.1.41) (pib:13)
[22 Oct 2009 6:36]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091022063126-l0qzirh9xyhp0bpc) (version source revid:alik@sun.com-20091019135554-s1pvptt6i750lfhv) (merge vers: 6.0.14-alpha) (pib:13)
[22 Oct 2009 7:08]
Bugs System
Pushed into 5.5.0-beta (revid:alik@sun.com-20091022060553-znkmxm0g0gm6ckvw) (version source revid:alik@sun.com-20091019132831-5j6xs19ld16m7r8j) (merge vers: 5.5.0-beta) (pib:13)
[28 Oct 2009 22:07]
Paul DuBois
Noted in 5.1.41, 5.5.0, 6.0.14 changelogs. Repair by sort or parallel repair of MyISAM tables could fail to fail over to repair with key cache.
[4 Nov 2009 9:26]
Bugs System
Pushed into 5.1.41 (revid:joro@sun.com-20091104092152-qz96bzlf2o1japwc) (version source revid:kristofer.pettersson@sun.com-20091103162305-08l4gkeuif2ozsoj) (merge vers: 5.1.41) (pib:13)
[11 Nov 2009 6:53]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091110093407-rw5g8dys2baqkt67) (version source revid:alik@sun.com-20091109080109-7dxapd5y5pxlu08w) (merge vers: 6.0.14-alpha) (pib:13)
[11 Nov 2009 7:01]
Bugs System
Pushed into 5.5.0-beta (revid:alik@sun.com-20091109115615-nuohp02h8mdrz8m2) (version source revid:alik@sun.com-20091105092355-jzukkw6wnd7hmgrj) (merge vers: 5.5.0-beta) (pib:13)
[18 Dec 2009 10:36]
Bugs System
Pushed into 5.1.41-ndb-7.1.0 (revid:jonas@mysql.com-20091218102229-64tk47xonu3dv6r6) (version source revid:jonas@mysql.com-20091218095730-26gwjidfsdw45dto) (merge vers: 5.1.41-ndb-7.1.0) (pib:15)
[18 Dec 2009 10:52]
Bugs System
Pushed into 5.1.41-ndb-6.2.19 (revid:jonas@mysql.com-20091218100224-vtzr0fahhsuhjsmt) (version source revid:jonas@mysql.com-20091217101452-qwzyaig50w74xmye) (merge vers: 5.1.41-ndb-6.2.19) (pib:15)
[18 Dec 2009 11:07]
Bugs System
Pushed into 5.1.41-ndb-6.3.31 (revid:jonas@mysql.com-20091218100616-75d9tek96o6ob6k0) (version source revid:jonas@mysql.com-20091217154335-290no45qdins5bwo) (merge vers: 5.1.41-ndb-6.3.31) (pib:15)
[18 Dec 2009 11:21]
Bugs System
Pushed into 5.1.41-ndb-7.0.11 (revid:jonas@mysql.com-20091218101303-ga32mrnr15jsa606) (version source revid:jonas@mysql.com-20091218064304-ezreonykd9f4kelk) (merge vers: 5.1.41-ndb-7.0.11) (pib:15)
[18 Dec 2009 13:23]
MC Brown
Already noted in earlier changelogs.
[24 Feb 2010 16:32]
MySQL Verification Team
this bug actually affects 5.0.90 still...
Description: when myisam_sort_buffer_size is too low, partitioned tables get corrupted during a repair table. non-partitioned tables didn't. There's invalid memory reads happening too: Output of testcase (cut to fit without wrapping!): mysql> check table `t1`; +---------+-------+----------+----------+ | Table | Op | Msg_type | Msg_text | +---------+-------+----------+----------+ | test.t1 | check | status | OK | +---------+-------+----------+----------+ 1 row in set (7.78 sec) mysql> repair table `t1`; +---------+--------+----------+----------------------------------------+ | Table | Op | Msg_type | Msg_text | +---------+--------+----------+----------------------------------------+ | test.t1 | repair | error | myisam_sort_buffer_size is too small | | test.t1 | repair | warning | Number of rows changed from 0 to 14244 | | test.t1 | repair | error | myisam_sort_buffer_size is too small | | test.t1 | repair | error | Partition p5 returned error | | test.t1 | repair | status | Operation failed | +---------+--------+----------+----------------------------------------+ 5 rows in set (2 min 1.88 sec) mysql> check table `t1`; +---------+-------+----------+------------------------------------------ | Table | Op | Msg_type | Msg_text +---------+-------+----------+------------------------------------------ | test.t1 | check | warning | Table is marked as crashed and last repai | test.t1 | check | warning | Size of indexfile is: 387072 Should | test.t1 | check | error | Found 8485 keys of 0 | test.t1 | check | error | Partition p5 returned error | test.t1 | check | error | Corrupt +---------+-------+----------+------------------------------------------ 5 rows in set (4.30 sec) 5.1.37 valgrind output: Invalid read of size 4 at: reinit_io_cache (mf_iocache.c:388) by: _create_index_by_sort (sort.c:209) by: mi_repair_by_sort (mi_check.c:2430) by: ha_myisam::repair (ha_myisam.cc:1156) by: ha_myisam::enable_indexes(unsigned) (ha_myisam.cc:1455) by: ha_myisam::end_bulk_insert() (ha_myisam.cc:1584) by: ha_partition::end_bulk_insert() (handler.h:1230) by: select_insert::send_eof() (handler.h:1230) by: do_select (sql_select.cc:10920) by: JOIN::exec() (sql_select.cc:2199) by: mysql_select (sql_select.cc:2388) by: handle_select (sql_select.cc:268) [Note] Retrying repair of: './test/t1' with keycache [Note] Retrying repair of: './test/t1' with keycache [Note] Retrying repair of: './test/t1' failed. Please try REPAIR EXTENDED or myisamchk How to repeat: run mysqld under valgrind to catch the valgrind errors. --- set session myisam_sort_buffer_size=4; drop table if exists `t1`; create table `t1`( `a` char(1) character set ucs2 collate ucs2_general_ci not null, `ordc` int,unique key `a` (`a`), key `a_2` (`a`,`ordc`), key `ordc` (`ordc`),key `a_3` (`a`)) engine=myisam partition by key (`a`) partitions 8; set @`a`:=6; insert into `t1` values (char(0),0),(char(1),1),(char(2),2), (char(3),3),(char(4),4),(char(5),5); insert ignore into `t1` select char(@`a`:=@`a`+1),@`a` from `t1`,`t1` `a`,`t1` `b`,`t1` `c`,`t1` `d`,`t1` `e`; check table `t1`; repair table `t1`; check table `t1`; ---- Suggested fix: o) fix that invalid memory read o) try not to make entire table crashed. behave like unpartitioned repair does. o) try improve the error message verbosity in the mysql error log.