| Bug #38996 | Race condition in ANALYZE TABLE | ||
|---|---|---|---|
| Submitted: | 25 Aug 2008 6:39 | Modified: | 18 Jun 2010 12:59 |
| Reporter: | Vasil Dimov | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: InnoDB storage engine | Severity: | S3 (Non-critical) |
| Version: | 5.1 | OS: | Any |
| Assigned to: | Satya B | CPU Architecture: | Any |
| Tags: | analyze race | ||
[25 Aug 2008 6:43]
Vasil Dimov
The same race condition exists even in 4.1.22, so it has not been added recently. Probably it has always been there. Not discovered because it is unusual to run more than one ANALYZE TABLE for a given table at the same time.
[26 Aug 2008 16:59]
MySQL Verification Team
Thank you for the bug report.
[1 Oct 2009 12:58]
Vasil Dimov
Fixed by serializing all ANALYZE TABLE inside InnoDB. The serialization is done per server, not per table even though per table would suffice, but it would be too complicated.
[2 Nov 2009 11:33]
MySQL Verification Team
Hi Vasil! Is it possible this bug could cause a crash?
[2 Nov 2009 11:40]
Vasil Dimov
Shane, Potentially yes, I wouldn't bet that it will never ever crash due to this race.
[4 Nov 2009 9:26]
Bugs System
Pushed into 5.1.41 (revid:joro@sun.com-20091104092152-qz96bzlf2o1japwc) (version source revid:kristofer.pettersson@sun.com-20091103162305-08l4gkeuif2ozsoj) (merge vers: 5.1.41) (pib:13)
[11 Nov 2009 6:53]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091110093407-rw5g8dys2baqkt67) (version source revid:alik@sun.com-20091109080109-7dxapd5y5pxlu08w) (merge vers: 6.0.14-alpha) (pib:13)
[11 Nov 2009 7:01]
Bugs System
Pushed into 5.5.0-beta (revid:alik@sun.com-20091109115615-nuohp02h8mdrz8m2) (version source revid:svoj@sun.com-20091105122958-jyqjx9xus8v4e0yd) (merge vers: 5.5.0-beta) (pib:13)
[13 Nov 2009 2:20]
Paul DuBois
Noted in 5.1.41, 5.5.0, 6.0.14 changelogs. Simultaneous ANALYZE TABLE operations for an InnoDB tables could be subject to a race condition.
[18 Dec 2009 10:36]
Bugs System
Pushed into 5.1.41-ndb-7.1.0 (revid:jonas@mysql.com-20091218102229-64tk47xonu3dv6r6) (version source revid:jonas@mysql.com-20091218095730-26gwjidfsdw45dto) (merge vers: 5.1.41-ndb-7.1.0) (pib:15)
[18 Dec 2009 10:51]
Bugs System
Pushed into 5.1.41-ndb-6.2.19 (revid:jonas@mysql.com-20091218100224-vtzr0fahhsuhjsmt) (version source revid:jonas@mysql.com-20091217101452-qwzyaig50w74xmye) (merge vers: 5.1.41-ndb-6.2.19) (pib:15)
[18 Dec 2009 11:06]
Bugs System
Pushed into 5.1.41-ndb-6.3.31 (revid:jonas@mysql.com-20091218100616-75d9tek96o6ob6k0) (version source revid:jonas@mysql.com-20091217154335-290no45qdins5bwo) (merge vers: 5.1.41-ndb-6.3.31) (pib:15)
[18 Dec 2009 11:21]
Bugs System
Pushed into 5.1.41-ndb-7.0.11 (revid:jonas@mysql.com-20091218101303-ga32mrnr15jsa606) (version source revid:jonas@mysql.com-20091218064304-ezreonykd9f4kelk) (merge vers: 5.1.41-ndb-7.0.11) (pib:15)
[21 Apr 2010 21:15]
Mark Callaghan
Are you sure you have fixed all cases of concurrent calls to dict_update_statistics for the same table? There are several potential calls to ha_innobase::info using the HA_STATUS_TIME flag from code in sql_show.cc
[21 Apr 2010 21:32]
Mark Callaghan
Is the fix for this as simple as using analyze_mutex to guard the call to dict_update_statistics in ha_innobase::info
[21 Apr 2010 23:36]
Mark Callaghan
The simple fix is too simple as analyze_mutex is a global mutex. I don't want to remove the convoy from LOCK_open to create it on analyze_mutex.
[22 Apr 2010 6:00]
MySQL Verification Team
see bug #53046 for latest comments
[28 Apr 2010 10:32]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/106782 3429 Vasil Dimov 2010-04-28 Revert the fix of Bug#38996 Race condition in ANALYZE TABLE This is branches/zip@r6032 in SVN and _is part_ of revid:svn-v4:16c675df-0fcb-4bc9-8058-dcc011a37293:branches/zip:6113 in BZR. This is being reverted because now the code is serialized directly on index->stat_n_diff_key_vals[] as the fix for Bug#53046 dict_update_statistics_low can still be run concurrently on same table goes.
[5 May 2010 15:18]
Bugs System
Pushed into 5.1.47 (revid:joro@sun.com-20100505145753-ivlt4hclbrjy8eye) (version source revid:kristofer.pettersson@sun.com-20100503172109-f9hracq5pqsaomb1) (merge vers: 5.1.47) (pib:16)
[6 May 2010 16:02]
Paul DuBois
Push resulted from incorporation of InnoDB tree. No changes pertinent to this bug. Re-closing.
[28 May 2010 6:13]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100524190136-egaq7e8zgkwb9aqi) (version source revid:alik@sun.com-20100512070920-xgpmqeytp0gc183c) (pib:16)
[28 May 2010 6:41]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100524190941-nuudpx60if25wsvx) (version source revid:alik@sun.com-20100507093037-7cykrx1n73v0tetc) (merge vers: 6.0.14-alpha) (pib:16)
[28 May 2010 7:08]
Bugs System
Pushed into 5.5.5-m3 (revid:alik@sun.com-20100524185725-c8k5q7v60i5nix3t) (version source revid:alexey.kopytov@sun.com-20100507164602-8w09samq3mpvbxbn) (merge vers: 5.5.5-m3) (pib:16)
[29 May 2010 23:20]
Paul DuBois
Already fixed in 5.5.x, 6.0.x.
[15 Jun 2010 8:16]
Bugs System
Pushed into 5.5.5-m3 (revid:alik@sun.com-20100615080459-smuswd9ooeywcxuc) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (merge vers: 5.1.47) (pib:16)
[15 Jun 2010 8:33]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100615080558-cw01bzdqr1bdmmec) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (pib:16)
[17 Jun 2010 12:18]
Bugs System
Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:martin.skold@mysql.com-20100616204905-jxjg342w35ks9vfy) (merge vers: 5.1.47-ndb-7.0.16) (pib:16)
[17 Jun 2010 13:05]
Bugs System
Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100615090726-jotpykke96le59w5) (merge vers: 5.1.47-ndb-6.2.19) (pib:16)
[17 Jun 2010 13:46]
Bugs System
Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:martin.skold@mysql.com-20100616120453-jh7wr05z1vf7r8pm) (merge vers: 5.1.47-ndb-6.3.35) (pib:16)
Description: During ANALYZE TABLE the function btr_estimate_number_of_different_key_vals() is called and it modifies the array index->stat_n_diff_key_vals[]. This is not protected by any mutex and several ANALYZE TABLE can edit the array simultaneously. The backtrace is the following: #3 0x00000000007903fe in btr_estimate_number_of_different_key_vals (index=0x18160a8) at btr/btr0cur.c:3173 #4 0x00000000007b0e3d in dict_update_statistics_low (table=0x18150a8, has_dict_mutex=0) at dict/dict0dict.c:4024 #5 0x00000000007b0efa in dict_update_statistics (table=0x18150a8) at dict/dict0dict.c:4053 #6 0x000000000077fd96 in ha_innobase::info (this=0x252a010, flag=28) at handler/ha_innodb.cc:6597 #7 0x0000000000778cde in ha_innobase::analyze (this=0x252a010, thd=0x2283000, check_opt=0x22853a8) at handler/ha_innodb.cc:6798 #8 0x00000000006d344b in handler::ha_analyze (this=0x252a010, thd=0x2283000, check_opt=0x22853a8) at handler.cc:3079 #9 0x00000000006e936c in mysql_admin_table (thd=0x2283000, tables=0x25220d8, check_opt=0x22853a8, operator_name=0x9c9366 "analyze", lock_type=TL_READ_NO_INSERT, open_for_modify=true, no_warnings_for_error=false, extra_open_options=0, prepare_func=0, operator_func= {__pfn = 0x6d3410 <handler::ha_analyze(THD*, st_ha_check_opt*)>, __delta = 0}, view_operator_func=0) at sql_table.cc:4338 #10 0x00000000006ea049 in mysql_analyze_table (thd=0x2283000, tables=0x25220d8, check_opt=0x22853a8) at sql_table.cc:5008 #11 0x00000000005d1cdf in mysql_execute_command (thd=0x2283000) at sql_parse.cc:2799 #12 0x00000000005d7a7b in mysql_parse (thd=0x2283000, inBuf=0x2522010 "analyze table t", length=15, found_semicolon=0x7ffffebb5dd0) at sql_parse.cc:5656 #13 0x00000000005d86af in dispatch_command (command=COM_QUERY, thd=0x2283000, packet=0x2297001 "", packet_length=15) at sql_parse.cc:1137 #14 0x00000000005d9844 in do_command (thd=0x2283000) at sql_parse.cc:794 This should be serialized somewhere. How to repeat: Could be tricky. btr_estimate_number_of_different_key_vals() is called not only during ANALYZE TABLE but also when the table is opened. I stepped in gdb until btr_estimate_number_of_different_key_vals() was called from ANALYZE TABLE (the above backtrace), executed "call sleep(100)" from gdb and then executed another ANALYZE TABLE from a second mysql client. Suggested fix: Serialize in InnoDB or in MySQL layer.