mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Christian Couder <>
To: Jeff King <>
Cc: "Junio C Hamano" <>, git <>,
	"Thomas Rast" <>,
	"Ævar Arnfjörð Bjarmason" <>,
	"Christian Couder" <>
Subject: Re: [PATCH v1 0/4] Teach 'run' perf script to read config files
Date: Thu, 13 Jul 2017 21:45:22 +0200	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Thu, Jul 13, 2017 at 8:40 PM, Jeff King <> wrote:
> On Thu, Jul 13, 2017 at 11:29:10AM -0700, Junio C Hamano wrote:
>> > So then I think your config file primarily becomes about defining the
>> > properties of each run. I'm not sure if it would look like what you're
>> > starting on here or not.
>> Yeah, I suspect that the final shape that defines the matrix might
>> have to become quite a bit different.
> I think it would help if the perf code was split better into three
> distinct bits:
>   1. A data-store capable of storing the run tuples along with their
>      outcomes for each test.
>   2. A "run" front-end that runs various profiles (based on config,
>      command-line options, etc) and writes the results to the data
>      store.
>   3. A flexible viewer which can slice and dice the contents of the data
>      store according to different parameters.
> We're almost there now. The "run" script actually does store results,
> and you can view them via "" without actually re-running the
> tests. But the data store only indexes on one property: the tree that
> was tested (and all of the other properties are ignored totally; you can
> get some quite confusing results if you do a "./run" using say git.git
> as your test repo, and then a followup with "linux.git").

Yeah I agree, but if possible I'd like to avoid working on the three
different parts at the same time.

I haven't thought much about how to improve the data store yet.
I may have to look at that soon though.

> I have to imagine that somebody else has written such a system already
> that we could reuse.  I don't know of one off-hand, but this is also not
> an area where I've spent a lot of time.

Actually about the viewer AEvar suggested having something like and which seem to be made using

So unless something else is suggested, I plan to make it possible to
import the results of the perf tests into codespeed, but I haven't
looked at that much yet.

> We're sort of drifting off topic from Christian's patches here. But if
> we did have a third-party system, I suspect the interesting work would
> be setting up profiles for the "run" tool to kick off. And we might be
> stuck in such a case using whatever format the tool prefers. So having a
> sense of what the final solution looks like might help us know whether
> it makes sense to introduce a custom config format here.

I don't think we should completely switch to a third-party system for
Though it would simplify my work if we decide to do that.

I think people might want different viewers, so we should just make
sure that we can easily massage the results from the run script, so
that it will be easy to provide them as input to many different

So we are pretty free to decide how we specify which tests should be
performed on which revision, and I think a config file is the best

  parent reply	other threads:[~2017-07-13 19:45 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-07-13  6:50 [PATCH v1 0/4] Teach 'run' perf script to read config files Christian Couder
2017-07-13  6:50 ` [PATCH v1 1/4] perf/run: add '--config' option to the 'run' script Christian Couder
2017-07-13  6:50 ` [PATCH v1 2/4] perf/run: add get_var_from_env_or_config() Christian Couder
2017-07-13  6:50 ` [PATCH v1 3/4] perf/run: add GIT_PERF_DIRS_OR_REVS Christian Couder
2017-07-13  6:50 ` [PATCH v1 4/4] perf/run: add calls to get_var_from_env_or_config() Christian Couder
2017-07-13 16:58 ` [PATCH v1 0/4] Teach 'run' perf script to read config files Jeff King
2017-07-13 18:29   ` Junio C Hamano
2017-07-13 18:40     ` Jeff King
2017-07-13 19:21       ` Junio C Hamano
2017-07-13 19:45       ` Christian Couder [this message]
2017-07-13 18:57   ` Christian Couder
2017-07-13 20:55     ` Jeff King
2017-07-14  6:27       ` Christian Couder
2017-07-14  8:05         ` Jeff King
2017-07-26 15:58         ` Christian Couder
2017-07-26 16:54           ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).