Bug #47698 | backup.backup_logs fails under valgrind | ||
---|---|---|---|
Submitted: | 29 Sep 2009 6:59 | Modified: | 20 Feb 2010 19:07 |
Reporter: | Alexander Nozdrin | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | Tests: Backup | Severity: | S3 (Non-critical) |
Version: | 6.0 | OS: | Any |
Assigned to: | Chuck Bell | CPU Architecture: | Any |
Tags: | experimental, pb2, sporadic, test failure |
[29 Sep 2009 6:59]
Alexander Nozdrin
[29 Sep 2009 6:59]
Alexander Nozdrin
The test case has been marked experimental.
[10 Nov 2009 15:23]
Chuck Bell
The test contains an arbitrary time calculation that has no basis. In this case, the test is comparing the start and stop times by a factor of '5'. We must either remove the arbitrary calculation or introduce a means to delay backup so that we can guarantee time will elapse. To do so, one must use a time delay mechanism like sleep or real_sleep while backup is being run. However, it is know that these mechanisms are not reliable and therefore one cannot predict exactly how long a process will 'sleep' and therefore cannot be made predictable. The arbitrary calculation is line 296: --echo We calculate the timedifference between backup start time and stop --echo time. If this difference is '0', then backup start time and stop time --echo are same. SELECT timediff(@stop_backup, @start_backup) > 5; The benefit of doing this check is dubious. This test is stating that the stop time of a backup operation is at least 5 seconds more than the start time. But that is not reliable. If run on a fast system then it is possible a simple backup can run in less than 5 seconds. If this happens, the result will be 0 (as recorded in the result file). But if run on a very slow machine (like valgrind on PB2) it could be much more than 5 seconds and the result will be 1. But the real problem is the test statement verifies nothing and has no meaning for quality given the limited predictability of execution on slower machines. Furthermore, the comment that goes with the test statement is wrong. The test statement is not checking for start and stop to be = 0. Based on this, the following is the solution for this bug. Solution -------- Remove the arbitrary time calculation from the test. It is not needed and serves no purpose. Note: Only 1 reviewer is needed since this patch will change on test-related files.
[10 Nov 2009 15:25]
Chuck Bell
There are more arbitrary time calculations in lines 307-311. However, these do not (as yet) pose a problem. These will be left unchanged for now.
[10 Nov 2009 17:24]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/89982 2889 Chuck Bell 2009-11-10 BUG#47698: backup.backup_logs fails under valgrind The backup_logs test contains a test statement designed to ensure backup completes in less than 5 seconds. This goal is arbitrary and serves no purpose given the environment these tests execute. There could be significant delays in processing for virtual machines under heavy load thereby making any execution timing unpredictable. This patch removes the arbitrary timing test statement. @ mysql-test/collections/default.experimental Removed test from experimental list. @ mysql-test/suite/backup/r/backup_logs.result Corrected result file. @ mysql-test/suite/backup/t/backup_logs.test Removed arbitrary and inaccurate test statement.
[20 Feb 2010 9:19]
Bugs System
Pushed into 6.0.14-alpha (revid:ingo.struewing@sun.com-20100218152520-s4v1ld76bif06eqn) (version source revid:ingo.struewing@sun.com-20100119103538-wtp5alpz4p2jayl5) (merge vers: 6.0.14-alpha) (pib:16)
[20 Feb 2010 19:07]
Paul DuBois
Changes to test cases. No changelog entry needed.