| Bug #791 | At binlog rotation, INSERTs may not find their way into the binlog | ||
|---|---|---|---|
| Submitted: | 4 Jul 2003 9:39 | Modified: | 14 Jul 2003 3:59 |
| Reporter: | Guilhem Bichot | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server | Severity: | S1 (Critical) |
| Version: | 4.0 | OS: | Any (all) |
| Assigned to: | CPU Architecture: | Any | |
[11 Jul 2003 5:29]
Guilhem Bichot
Thank you for your bug report. This issue has been fixed in the latest
development tree for that product. You can find more information about
accessing our development trees at
http://www.mysql.com/doc/en/Installing_source_tree.html
fixed in ChangeSet 1.1514
[14 Jul 2003 2:21]
Guilhem Bichot
Re-opened in the 4.0 bk tree after ChangeSet@1.1512.1.13, 2003-07-14 10:12:05+03:00. Can be reproduced by adding a "sleep" in new_file() (see the initial bug report).
[14 Jul 2003 3:59]
Michael Widenius
Thank you for your bug report. This issue has been fixed in the latest
development tree for that product. You can find more information about
accessing our development trees at
http://www.mysql.com/doc/en/Installing_source_tree.html
The fix will appear in 4.0.14
[15 Mar 2011 4:44]
Wei Zhang
Hello My name is Wei, I am a grad student at Univ of Wisconsin, Madison. I am interested in studying some "intermediate patch"(aka the incomplete patch that didn't quite fix the bug at the first try) for a current research project. Therefore, I am specially interested in how to get the one that is mentioned as "ChangeSet@1.1512.1.13" for this MySQL 791 bug. However when I am trying to follow the link http://www.mysql.com/doc/en/Installing_source_tree.html, it seems that I cannot download anything from that website(I suspect that link ceases to function). I am wondering could you give me some more hints about how to get the MySQL version mentioned as "ChangeSet@1.1512.1.13". Thanks a lot! Wei
[15 Mar 2011 8:20]
Wei Zhang
Hello again, I did a little search myself, and realized that in order to check out the corresponding ChangeSet, I would need to use BitKeeper. I couldn't find bk-client.shar as many links have suggested, however I did get a bk-client2.0, which roughly works. However it seems that bk://mysql.bkbits.net/ is not quite working, as I couldnt check out any version of mysql(such as 5.1, 5.2 or 4.0) I understand that MySQL has moved from BitKeeper to Bazaar for the purpose of version control. I am curious if Bazaar will host all the Changesets previously handled by BitKeeper? Thanks! Wei
[15 Mar 2011 10:21]
Guilhem Bichot
Hi Wei. Yes, all bitkeeper history has been preserved in the bazaar trees. The "incomplete" fix in the bzr revision having this "ID": sp1r-guilhem@mysql.com-20030711122644-30671 Monty reverted this fix in: sp1r-monty@mashka.mysql.fi-20030714071205-35331 and did another fix in: sp1r-monty@narttu.mysql.fi-20030714115926-35380 As you want to see the incomplete fix: bzr branch lp:mysql-server -r 'sp1r-guilhem@mysql.com-20030711122644-30671' incompl_fix (all on a single line). This will download a version of MySQL with the incomplete fix and put it in directory "incompl_fix". If you want to see the changes introduced by this incomplete fix, do cd incompl_fix bzr diff -c 'sp1r-guilhem@mysql.com-20030711122644-30671'
[16 Mar 2011 8:52]
Wei Zhang
Hi Guilhem, Thank you very much for the prompt reply? I followed your directions and checked out the corresponding version. However, I have two questions after looking at the patch: (1)If I understand this incomplete fix correctly: it mainly adds locks around close() and open() in the new_file() function; and for the is_open(1) function called by mysql_insert() in sql_insert.cc, it also uses lock to protect the checking on log_type. Then it seems that the bug(atomicity violation) is solved by this fix:because even the new_file() function and is_open(1) are called simultaneously; due to the atomicity provided by this lock, is_open(1) will eventually see the log as not closed(thanks to the open() function). So I am curious why this patch didn't solve this problem ? (2) I was trying to build this incomplete fix so that I can play around with it, however from the checkout version, I only got the source code but not the related Makefile.in , configure files etc. I have tried to copy those files around, but it seems there are still files missing. I am wondering where can I get a version for this incomplete patch that is equipped with all the necessary files to build from scratch. Thank you very much! Wei
[16 Mar 2011 14:47]
Guilhem Bichot
1) yes the "incomplete" fix was fully correct as far as concurrency is concerned. But because it adds mutex locks to frequently called functions like is_open(), Monty thought it would slow down the system, so he did a different fix. 2) yes, "Makefile.in" and "configure" are missing. They have to be generated, from "Makefile.am" and "configure.in". Try to run BUILD/compile-pentium. If you want more control, try BUILD/autorun.sh ./configure your_options make This is a code tree from 2003; I'm not sure it will build on your machine; if you have a recent gcc compiler, this compiler may not accept constructs which we used in 2003. Good luck!
[17 Mar 2011 0:12]
Wei Zhang
Thank you very much Guilhem! It clears many of my doubts regarding the bug. I will look into how to generate the configuration files,etc following your suggestions as well. Thanks again! Wei
Description: In new_file() (log.cc) I changed code to be like this: close(); + if (save_log_type == LOG_BIN) + { + printf("after close, before open; I wait for 20 seconds\n"); + sleep(20); + printf("sleep finished, opening\n"); + } open(old_name, save_log_type, new_name_ptr, index_file_name, io_cache_type, no_auto_events, max_size); new_file() is called when FLUSH LOGS is issued, or when the binlog exceeds max_binlog_size. Here we sleep between close() and open(), so for 20 seconds the log is marked "closed", i.e. is_open() is false. While in sql_insert we have: if (mysql_bin_log.is_open()) { Query_log_event qinfo(thd, thd->query, thd->query_length, log_delayed); if (mysql_bin_log.write(&qinfo) && transactional_table) error=1; } So any INSERT issued during the 20 seconds silently does not go into the binlog. Indeed, during the sleep I issued INSERT queries and 16 of them are not in the binlog when I finally SHOW BINLOG EVENTS in the old and new binlog. This race condition is just very very serious and critical! How to repeat: Not repeatable except by changing the code or using a debugger to add breakpoints. Suggested fix: is_open() should take locks before reading log_type. I will fix it before 4.0.14 goes out.