Bug #38313 Tests will fail if time does not increase reasonably (NTP would help)
Submitted: 23 Jul 2008 15:11 Modified: 14 Jan 13:18
Reporter: Joerg Bruehe Email Updates:
Status: Won't fix Impact on me:
None 
Category:Tests Severity:S3 (Non-critical)
Version:any OS:Any
Assigned to: Bjørn Munch CPU Architecture:Any
Tags: time
Triage: D5 (Feature request)

[23 Jul 2008 15:11] Joerg Bruehe
Description:
This is *no* bug in either the tests or in our software,
this is a defect in the environment we use to run the tests.

Several of our tests check times before and after a command,
or sleep for some time,
or in some other way depend on real ("wallclock") time.

They will fail if time does not increase.

This assumption will be violated if time is ever running backward.

While we try to use NTP on our machines (which ensures time is increasing),
there are some build environments where this is not possible.

One example is RedHat 3 running in a VM, there may be more (ask IIT).

Possible symptom:
=====
main.func_misc                 [ fail ]

--- /PATH/mysql-test/r/func_misc.result
+++ /PATH/mysql-test/r/func_misc.reject
@@ -135,7 +135,7 @@
 timediff(b, a) >= '00:00:03'
 1
 1
-1
+0
 drop table t2;
 drop table t1;
 set global query_cache_size=default;
=====

How to repeat:
Run tests in any environment which does not use NTP
and set time back during test run.

One example is our RedHat 3 installation in a VM:
NTP does not work there,
the hardware clock deviates too much so some time sync is needed,
the machine is set up to run "ntpdate" in some tight loop (cron job)
which means the time will be set backwards if the hardware clock is too fast.

Suggested fix:
We need a machine time that is reasonably in sync across all our build hosts, because the autotools will fail if files have timestamps in the future
(and other negative effects, too).

The best solution is to use NTP, as we do on most hosts.

"ntpdate" in a loop is an alternative, but it may cause some backward jumps of time.

Could we get a "ntpdate" with a setuid-bit (for root) which we call at the start of each build job, so we ensure time is current (for the unpack) but have no risk of it being set back during the test run ?
(Unless there are parallel builds, but even then the risk is extremely low.)

This would avoid the loop with "ntpdate" calls, which sooner or later will hit us during test runs.
[14 Jan 13:18] Bjørn Munch
This problem has not been for a very long time (AFAIK) so closing.

In any case, this is not a product defect but an environment issue, so should be filed internally as such if still a problem.