Re: [PATCH] Make some commit hashes in tests reproducible

[Jeff King <peff@peff.net>]

On Tue, Jul 07, 2020 at 02:35:57PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > That's using the same start point as test_tick, though really it could
> > be anything. I've intentionally _not_ called test_tick at the beginning
> > of each script, because that would throw off all of the scripts that do
> > use it by one tick (whereas the first test_tick will overwrite these
> > values).
> 
> Yup, I think that is a sensible approach.

OK, I'll see if I can polish it into a series.

> Another thing we could do is under DEVELOPER=YesPlease we can set
> timestamps you just added here to some random time.
> 
> The ones that do care about reproducibility guarantee by using
> test_tick would not be affected, and those that were happy with the
> current time should be happy with such a random timestamp.  Or we
> could just drop what this patch does only under DEVELOPER=YesPlease
> which amounts to be the same as setting random time.

That should be easy to do. I wonder if it's really worth the trouble to
maintain two parallel worlds. I.e., one of the goals would be to stop
worrying about this non-determinism. If we keep it as a requirement,
then we might as well do so all the time (and making it random under
DEVELOPER is effectively keeping it as a requirement, since we expect
all tests to pass under that flag).

I'm also skeptical how often we use system times anyway, because _any_
use of test_commit or test_tick in a script is enough to make all of the
subsequent commands deterministic. I'd be more inclined to let a
particular script say "I'm interested in random times". But then, I'd
think such a script would be better written to trigger its interesting
cases with a well-crafted set of deterministic times.

-Peff