| Bug #54486 | assert in my_seek, concurrent DROP/CREATE SCHEMA, CREATE TABLE, REPAIR | ||
|---|---|---|---|
| Submitted: | 14 Jun 2010 14:15 | Modified: | 6 Jan 2011 15:23 |
| Reporter: | Matthias Leich | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server | Severity: | S3 (Non-critical) |
| Version: | 5.1.49 | OS: | Any |
| Assigned to: | Dmitry Shulga | CPU Architecture: | Any |
| Tags: | assert | ||
[14 Jun 2010 14:17]
Matthias Leich
Protocol of the RQG run
Attachment: prt1 (application/octet-stream, text), 108.47 KiB.
[17 Jun 2010 11:39]
Matthias Leich
Result on 5.1.49-debug-log
mysql-5.1 revno: 3410 2010-06-03
--------------------------------
Thread 1 (process 30415):
#0 0x00007f7923825ce6 in pthread_kill () from /lib64/libpthread.so.0
#1 0x0000000000b21a1f in my_write_core (sig=6) at stacktrace.c:329
#2 0x00000000006bd0ed in handle_segfault (sig=6) at mysqld.cc:2571
#3 <signal handler called>
#4 0x00007f7922a2e5c5 in raise () from /lib64/libc.so.6
#5 0x00007f7922a2fbb3 in abort () from /lib64/libc.so.6
#6 0x00007f7922a271e9 in __assert_fail () from /lib64/libc.so.6
#7 0x0000000000afd4a1 in my_seek (fd=-1, pos=0, whence=2, MyFlags=0) at my_seek.c:58
#8 0x0000000000afea58 in init_io_cache (info=0x44a104b0, file=-1, cachesize=262108, type=READ_CACHE, seek_offset=0, use_async_io=1 '\001', cache_myflags=16) at mf_iocache.c:209
#9 0x0000000000a3eca7 in mi_repair (param=0x44a101d0, info=0x197c118, name=0x44a0fec0 "./testdb/t1", rep_quick=0) at mi_check.c:1557
#10 0x0000000000a24721 in ha_myisam::repair (this=0x1cdd308, thd=0x1c345d8, param=@0x44a101d0, do_optimize=false) at ha_myisam.cc:1156
#11 0x0000000000a24f34 in ha_myisam::repair (this=0x1cdd308, thd=0x1c345d8, check_opt=0x1c36b48) at ha_myisam.cc:1031
#12 0x000000000080ad7f in handler::ha_repair (this=0x1cdd308, thd=0x1c345d8, check_opt=0x1c36b48) at handler.cc:3158
#13 0x000000000083a267 in mysql_admin_table (thd=0x1c345d8, tables=0x1c165f0, check_opt=0x1c36b48, operator_name=0xca8365 "repair", lock_type=TL_WRITE, open_for_modify=false, no_warnings_for_error=false, extra_open_options=32,
prepare_func=0x829761 <prepare_for_repair>, operator_func=0x80ad44 <handler::ha_repair(THD*, st_ha_check_opt*)>, view_operator_func=0) at sql_table.cc:4810
#14 0x000000000083b684 in mysql_repair_table (thd=0x1c345d8, tables=0x1c165f0, check_opt=0x1c36b48) at sql_table.cc:5071
#15 0x00000000006d15c7 in mysql_execute_command (thd=0x1c345d8) at sql_parse.cc:3006
#16 0x00000000006d81ac in mysql_parse (thd=0x1c345d8, inBuf=0x1c16508 "REPAIR TABLE testdb . t1", length=24, found_semicolon=0x44a1aee0) at sql_parse.cc:5994
#17 0x00000000006d9034 in dispatch_command (command=COM_QUERY, thd=0x1c345d8, packet=0x1c36fa9 "", packet_length=24) at sql_parse.cc:1241
#18 0x00000000006da502 in do_command (thd=0x1c345d8) at sql_parse.cc:882
#19 0x00000000006c688c in handle_one_connection (arg=0x1c345d8) at sql_connect.cc:1134
#20 0x00007f7923821040 in start_thread () from /lib64/libpthread.so.0
#21 0x00007f7922acf08d in clone () from /lib64/libc.so.6
#22 0x0000000000000000 in ?? ()
[1 Sep 2010 9:27]
Jon Olav Hauglid
Not able to repeat this bug in 5.5 (5.5-runtime tree). Still able to repeat it in 5.1.
[1 Nov 2010 14:54]
Jon Olav Hauglid
Editing the version tag since this bug is not repeatable in 5.5. Still repeatable in 5.1 however.
[19 Nov 2010 5:45]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/124353 3511 Dmitry Shulga 2010-11-19 Fixed bug#54486 - assert in my_seek, concurrent DROP/CREATE SCHEMA, CREATE TABLE, REPAIR. The cause of assert was concurrent execution of DROP DATABASE and REPAIR TABLE where first statement deleted table's file .TMD at the same time when REPAIR TABLE statement tried replace old file by temporary file that has just been removed. Additionally was fixed trouble when DROP TABLE try delete all files belong to table being dropped at the same time when REPAIR TABLE statement has just deleted .TMD file. @ sql/sql_db.cc mysql_rm_known_files() modified: ignore possible errors when trying delete all table's files. Such aggressigve algorithm permits skip already deleted (in another thread) files. @ storage/myisam/mi_check.c mi_repair() was modified: set param->retry_repair= 0 in order to don't call following failover procedure in ha_myisam::repair() in case of error in access to .TMD file.
[29 Nov 2010 10:08]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/125310 3511 Dmitry Shulga 2010-11-29 Fixed bug#54486 - assert in my_seek, concurrent DROP/CREATE SCHEMA, CREATE TABLE, REPAIR. The cause of assert was concurrent execution of DROP DATABASE and REPAIR TABLE where first statement deleted table's file .TMD at the same time when REPAIR TABLE statement tried replace old file by temporary file that has just been removed. Additionally was fixed trouble when DROP TABLE try delete all files belong to table being dropped at the same time when REPAIR TABLE statement has just deleted .TMD file. @ sql/sql_db.cc mysql_rm_known_files() modified: ignore possible errors when trying delete all table's files. Such aggressigve algorithm permits skip already deleted (in another thread) files. @ storage/myisam/mi_check.c mi_repair() was modified: set param->retry_repair= 0 in order to don't call following failover procedure in ha_myisam::repair(); set an error in diagnostic area in case of error in access to .TMD file.
[3 Dec 2010 14:10]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/125939 3511 Dmitry Shulga 2010-12-03 Fixed bug#54486 - assert in my_seek, concurrent DROP/CREATE SCHEMA, CREATE TABLE, REPAIR. The cause of assert was concurrent execution of DROP DATABASE and REPAIR TABLE where first statement deleted table's file .TMD at the same time when REPAIR TABLE statement tried replace old file by temporary file that has just been removed. Additionally was fixed trouble when DROP TABLE try delete all files belong to table being dropped at the same time when REPAIR TABLE statement has just deleted .TMD file. Regression test isn't added because for such one need insert synchonization point (DEBUG_SYNC) to function my_redel() defined in mysys/my_redel.c. DEBUG_SYNC is defined in sql/debug_sync.h. So, in order to add this one we need add additional dependency, i.e. mysys/my_redel.c will depend on sql.debug_sync.h, that is wrong. @ sql/sql_db.cc mysql_rm_known_files() modified: ignore possible errors when trying delete all table's files. Such aggressigve algorithm permits skip already deleted (in another thread) files. @ storage/myisam/mi_check.c mi_repair() was modified: set param->retry_repair= 0 in order to don't call following failover procedure in ha_myisam::repair(); set an error in diagnostic area in case of error in access to .TMD file.
[9 Dec 2010 15:49]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/126441 3514 Dmitry Shulga 2010-12-09 Fixed bug#54486 - assert in my_seek, concurrent DROP/CREATE SCHEMA, CREATE TABLE, REPAIR. The cause of assert was concurrent execution of DROP DATABASE and REPAIR TABLE where first statement deleted table's file .TMD at the same time as REPAIR TABLE tried to read file details from the old file that was just removed. Additionally was fixed trouble when DROP TABLE try delete all files belong to table being dropped at the same time when REPAIR TABLE statement has just deleted .TMD file. No regression test added because this would require adding a sync point to mysys/my_redel.c. Since this bug is not present in 5.5+, adding test coverage was considered unnecessary. The patch has been verified using RQG testing. @ sql/sql_db.cc mysql_rm_known_files() modified: ignore possible errors when trying delete all table's files. Such aggressive algorithm permits skip already deleted (in another thread) files. @ storage/myisam/mi_check.c mi_repair() was modified: set param->retry_repair= 0 in order to don't call following failover procedure in ha_myisam::repair().
[10 Dec 2010 7:50]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/126502 3515 Dmitry Shulga 2010-12-10 Fixed bug#54486 - assert in my_seek, concurrent DROP/CREATE SCHEMA, CREATE TABLE, REPAIR. The cause of assert was concurrent execution of DROP DATABASE and REPAIR TABLE where first statement deleted table's file .TMD at the same time as REPAIR TABLE tried to read file details from the old file that was just removed. Additionally was fixed trouble when DROP TABLE try delete all files belong to table being dropped at the same time when REPAIR TABLE statement has just deleted .TMD file. No regression test added because this would require adding a sync point to mysys/my_redel.c. Since this bug is not present in 5.5+, adding test coverage was considered unnecessary. The patch has been verified using RQG testing. @ sql/sql_db.cc mysql_rm_known_files() modified: ignore possible ENOENT error when trying delete all table's files. Such aggressive algorithm permits skip already deleted (in another thread) files. Installation of Drop_table_error_handler as internal error handler moved from mysql_rm_db() to mysql_rm_knowns_files() near to place where source of possible errors (call to mysql_rm_table_part2) located. @ storage/myisam/mi_check.c mi_repair() was modified: set param->retry_repair= 0 in order to don't call following failover procedure in ha_myisam::repair().
[10 Dec 2010 11:35]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/126517 3183 Dmitry Shulga 2010-12-10 [merge] Manual merge from mysql-5.1-bugteam for bug#54486.
[10 Dec 2010 12:57]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/126524 3419 Dmitry Shulga 2010-12-10 [merge] Merge from mysql-5.5-bugteam for bug#54486.
[10 Dec 2010 12:59]
Dmitry Shulga
Patch was pushed to mysql-5.1-bugteam, mysql-5.5-bugteam, mysql-trunk-bugfixing.
[17 Dec 2010 12:48]
Bugs System
Pushed into mysql-5.1 5.1.55 (revid:georgi.kodinov@oracle.com-20101217124435-9imm43geck5u55qw) (version source revid:dmitry.shulga@oracle.com-20101210074850-0cttphw0aga0j8km) (merge vers: 5.1.55) (pib:24)
[17 Dec 2010 12:52]
Bugs System
Pushed into mysql-5.5 5.5.9 (revid:georgi.kodinov@oracle.com-20101217124733-p1ivu6higouawv8l) (version source revid:dmitry.shulga@oracle.com-20101210113258-5kuldljyahvu390t) (merge vers: 5.5.8) (pib:24)
[17 Dec 2010 12:56]
Bugs System
Pushed into mysql-trunk 5.6.1 (revid:georgi.kodinov@oracle.com-20101217125013-y8pb3az32rtbplc9) (version source revid:dmitry.shulga@oracle.com-20101210125255-6pb95vnvpowxphti) (merge vers: 5.6.1) (pib:24)
[6 Jan 2011 15:23]
Paul DuBois
Noted in 5.1.55, 5.5.9 changelogs. An assertion could be raised during concurrent execution of DROP DATABASE and REPAIR TABLE if the drop deleted a table's .TMD file at the same time the repair tried to read details from the old file that was just removed. A problem could also occur when DROP TABLE tried to remove all files belonging to a table at the same time REPAIR TABLE had just deleted the table's .TMD file.
Description: The assert happens in mysys/my_ssek.c line 57: /* Make sure we are using a valid file descriptor! */ DBUG_ASSERT(fd != -1); <------------------- #if defined (_WIN32) newpos= my_win_lseek(fd, pos, whence); #else newpos= lseek(fd, pos, whence); #endif Result on mysql-trunk-runtime revno: 3055 2010-06-12 (mysql-trunk revno: 3026 2010-05-24 shows the same effect) ---------------------------------------------------------- .... Program terminated with signal 6, Aborted. ... #0 0x00007ffdefd5cce6 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007ffdefd5cce6 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000a13395 in my_write_core (sig=6) at .../mysys/stacktrace.c:326 #2 0x0000000000545d16 in handle_segfault (sig=6) at .../sql/mysqld.cc:2791 #3 <signal handler called> #4 0x00007ffdeed725c5 in raise () from /lib64/libc.so.6 #5 0x00007ffdeed73bb3 in abort () from /lib64/libc.so.6 #6 0x00007ffdeed6b1e9 in __assert_fail () from /lib64/libc.so.6 #7 0x0000000000a0f957 in my_seek (fd=-1, pos=0, whence=2, MyFlags=0) at .../mysys/my_seek.c:57 #8 0x00000000009f0ecb in init_io_cache (info=0x40f822a0, file=-1, cachesize=262108, type=READ_CACHE, seek_offset=0, use_async_io=1 '\001', cache_myflags=16) at .../mysys/mf_iocache.c:209 #9 0x0000000000983c10 in mi_repair (param=0x40f81fc0, info=0x7ffde006da48, name=0x40f81cb0 "./testdb/t1", rep_quick=0) at .../storage/myisam/mi_check.c:1557 #10 0x000000000097d3d9 in ha_myisam::repair (this=0x7ffde00704a8, thd=0x1e1ac68, param=@0x40f81fc0, do_optimize=false) at .../storage/myisam/ha_myisam.cc:1057 #11 0x000000000097dbeb in ha_myisam::repair (this=0x7ffde00704a8, thd=0x1e1ac68, check_opt=0x1e1d250) at .../storage/myisam/ha_myisam.cc:932 #12 0x0000000000738029 in handler::ha_repair (this=0x7ffde00704a8, thd=0x1e1ac68, check_opt=0x1e1d250) at .../sql/handler.cc:3187 #13 0x0000000000657f48 in mysql_admin_table (thd=0x1e1ac68, tables=0x1e24ac0, check_opt=0x1e1d250, operator_name=0xad9515 "repair", lock_type=TL_WRITE, open_for_modify=false, no_warnings_for_error=false, extra_open_options=32, prepare_func=0x64a97c <prepare_for_repair>, operator_func=0x737fee <handler::ha_repair(THD*, st_ha_check_opt*)>, view_operator_func=0) at .../sql/sql_table.cc:4937 #14 0x0000000000659305 in mysql_repair_table (thd=0x1e1ac68, tables=0x1e24ac0, check_opt=0x1e1d250) at .../sql/sql_table.cc:5212 #15 0x00000000005d6ba5 in mysql_execute_command (thd=0x1e1ac68) at .../sql/sql_parse.cc:3029 #16 0x00000000005dc5e2 in mysql_parse (thd=0x1e1ac68, inBuf=0x1e249d8 "REPAIR TABLE testdb . t1", length=24, parser_state=0x40f8c9b0) at .../sql/sql_parse.cc:5889 #17 0x00000000005dd1de in dispatch_command (command=COM_QUERY, thd=0x1e1ac68, packet=0x1e1d999 "", packet_length=24) at .../sql/sql_parse.cc:1114 #18 0x00000000005de701 in do_command (thd=0x1e1ac68) at .../sql/sql_parse.cc:800 #19 0x00000000006a9116 in do_handle_one_connection (thd_arg=0x1e1ac68) at .../sql/sql_connect.cc:1194 #20 0x00000000006a91db in handle_one_connection (arg=0x1e1ac68) at .../sql/sql_connect.cc:1133 #21 0x00007ffdefd58040 in start_thread () from /lib64/libpthread.so.0 #22 0x00007ffdeee1308d in clone () from /lib64/libc.so.6 #23 0x0000000000000000 in ?? () ... .../sql/mysqld(my_print_stacktrace+0x32)[0xa13318] .../sql/mysqld(handle_segfault+0x2a6)[0x545b29] /lib64/libpthread.so.0[0x7ffdefd5fb30] /lib64/libc.so.6(gsignal+0x35)[0x7ffdeed725c5] /lib64/libc.so.6(abort+0x183)[0x7ffdeed73bb3] /lib64/libc.so.6(__assert_fail+0xe9)[0x7ffdeed6b1e9] .../sql/mysqld(my_seek+0xb7)[0xa0f957] .../sql/mysqld(init_io_cache+0x24c)[0x9f0ecb] .../sql/mysqld(mi_repair+0x257)[0x983c10] .../sql/mysqld(_ZN9ha_myisam6repairEP3THDR17st_mi_check_paramb+0x573)[0x97d3d9] .../sql/mysqld(_ZN9ha_myisam6repairEP3THDP15st_ha_check_opt+0x1c9)[0x97dbeb] .../sql/mysqld(_ZN7handler9ha_repairEP3THDP15st_ha_check_opt+0x3b)[0x738029] .../sql/mysqld[0x657f48] .../sql/mysqld(_Z18mysql_repair_tableP3THDP10TABLE_LISTP15st_ha_check_opt+0xb1)[0x659305] .../sql/mysqld(_Z21mysql_execute_commandP3THD+0x25e0)[0x5d6ba5] .../sql/mysqld(_Z11mysql_parseP3THDPKcjP12Parser_state+0x1c7)[0x5dc5e2] .../sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xaa3)[0x5dd1de] .../sql/mysqld(_Z10do_commandP3THD+0x22b)[0x5de701] .../sql/mysqld(_Z24do_handle_one_connectionP3THD+0x137)[0x6a9116] .../sql/mysqld(handle_one_connection+0x2d)[0x6a91db] /lib64/libpthread.so.0[0x7ffdefd58040] /lib64/libc.so.6(clone+0x6d)[0x7ffdeee1308d] How to repeat: 1.yy: ----- query: CREATE DATABASE testdb | DROP DATABASE testdb | CREATE TABLE testdb . t1 ENGINE = MyISAM AS SELECT 13 | REPAIR TABLE testdb . t1 ; 1.zz (content is not important) ------------------------------- $tables = { rows => [0, 1, 10 ], pk => [ 'int' , 'int auto_increment' ] }; $fields = { types => [ 'int' ], indexes => [ 'key', undef ] }; $data = { numbers => [ 'digit', 'null', undef ], strings => [ 'letter', 'english' ], blobs => [ 'data' ], temporals => ['date', 'year', 'null', undef ] } My command line: ---------------- nice -20 perl runall.pl --duration=1200 --queries=10000 \ --reporter=Deadlock,Backtrace,ErrorLog --threads=12 \ --basedir=/work2/repo/mysql-trunk-runtime \ --gendata=1.zz --mysqld=--lock-wait-timeout=1 \ --mysqld=--innodb-lock-wait-timeout=1 \ --mysqld=--log-output=table \ --mysqld=--loose-innodb-lock-wait-timeout=1 \ --grammar=1.yy --vardir=/dev/shm/var_1276507699 \ --mask-level=0 --mask=0 --seed=1