mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <>
To: Christian Couder <>
Cc: git <>, "Junio C Hamano" <>,
	"Thomas Rast" <>,
	"Ævar Arnfjörð Bjarmason" <>,
	"Christian Couder" <>
Subject: Re: [PATCH v1 0/4] Teach 'run' perf script to read config files
Date: Fri, 14 Jul 2017 04:05:44 -0400	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Fri, Jul 14, 2017 at 08:27:53AM +0200, Christian Couder wrote:

> The whole thing seems really complex to me though. And this makes me
> think that people might want to specify different GIT-BUILD-OPTIONS
> and config.mak files to be used when running perf tests, so that the
> results from perf tests can easily be reproduced later even if they
> have changed their build options in their development Git repo in the
> meantime.

I agree with the complexity. The general idea is that your currently
built HEAD is a snapshot in time of options. But that doesn't have to be
so, and laying out the options in a config file does seem like an

There is another implicit dependency, though: the set of (and exact
content of) the tests depends on your HEAD, too. So if I do:

  git checkout v2.5.0
  cd t/perf
  ./run v2.0.0 v2.1.0

I might get different results if I replace "v2.5.0" in the first command
with some other version, because the content of the tests will be
different. I'm not sure how to account for that in storing results. Most
of the time the version of the tests you ran is not going to be
interesting. But it can be a source of confusing discrepancies if a test
subtly changed between two runs. It probably happens infrequently enough
that it's not worth worrying about.

> So perhaps the config file should make it possible to specify a
> directory where all the build files (GIT-BUILD-OPTIONS, config.mak,
> config.mak.autogen and config.status) that should be used should be
> taken. And then it could also let people change some variables to
> override what is in those files which is needed to run perf tests with
> different parameters.

That sounds reasonable. I think you could ditch GIT-BUILD-OPTIONS
entirely. It's only needed to pull in GIT_PERF variables that would be
better served by being in the config in the first place.


  reply	other threads:[~2017-07-14  8:05 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-07-13  6:50 [PATCH v1 0/4] Teach 'run' perf script to read config files Christian Couder
2017-07-13  6:50 ` [PATCH v1 1/4] perf/run: add '--config' option to the 'run' script Christian Couder
2017-07-13  6:50 ` [PATCH v1 2/4] perf/run: add get_var_from_env_or_config() Christian Couder
2017-07-13  6:50 ` [PATCH v1 3/4] perf/run: add GIT_PERF_DIRS_OR_REVS Christian Couder
2017-07-13  6:50 ` [PATCH v1 4/4] perf/run: add calls to get_var_from_env_or_config() Christian Couder
2017-07-13 16:58 ` [PATCH v1 0/4] Teach 'run' perf script to read config files Jeff King
2017-07-13 18:29   ` Junio C Hamano
2017-07-13 18:40     ` Jeff King
2017-07-13 19:21       ` Junio C Hamano
2017-07-13 19:45       ` Christian Couder
2017-07-13 18:57   ` Christian Couder
2017-07-13 20:55     ` Jeff King
2017-07-14  6:27       ` Christian Couder
2017-07-14  8:05         ` Jeff King [this message]
2017-07-26 15:58         ` Christian Couder
2017-07-26 16:54           ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).