git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Jonathan Nieder <jrnieder@gmail.com>
To: "SZEDER Gábor" <szeder.dev@gmail.com>
Cc: Emily Shaffer <emilyshaffer@google.com>,
	Jeff King <peff@peff.net>,
	Jonathan Tan <jonathantanmy@google.com>,
	git@vger.kernel.org
Subject: Re: Git in Outreachy December 2019?
Date: Mon, 16 Sep 2019 16:13:49 -0700	[thread overview]
Message-ID: <20190916231348.GB67467@google.com> (raw)
In-Reply-To: <20190916214452.GC6190@szeder.dev>

SZEDER Gábor wrote:
> On Mon, Sep 16, 2019 at 11:42:08AM -0700, Emily Shaffer wrote:

>>  - try and make progress towards running many tests from a single test
>>    file in parallel - maybe this is too big, I'm not sure if we know how
>>    many of our tests are order-dependent within a file for now...
>
> Forget it, too many (most?) of them are order-dependent.

Hm, I remember a conversation about this with Thomas Rast a while ago.
It seemed possible at the time.

Most tests use "setup" or "set up" in the names of test assertions
that are required by later tests.  It's very helpful for debugging and
maintenance to be able to skip or reorder some tests, so I've been
able to rely on this a bit.  Of course there's no automated checking
in place for that, so there are plenty of test scripts that are
exceptions to it.

If we introduce a test_setup helper, then we would not have to rely on
convention any more.  A test_setup test assertion would represent a
"barrier" that all later tests in the file can rely on.  We could
introduce some automated checking that these semantics are respected,
and then we get a maintainability improvement in every test script
that uses test_setup.  (In scripts without any test_setup, treat all
test assertions as barriers since they haven't been vetted.)

With such automated tests in place, we can then try updating all tests
that say "setup" or "set up" to use test_setup and see what fails.

Some other tests cannot run in parallel for other reasons (e.g. HTTP
tests).  These can be declared as such, and then we have the ability
to run arbitrary individual tests in parallel.

Most of the time in a test run involves multiple test scripts running
in parallel already, so this isn't a huge win for the time to complete
a normal test run.  It helps more with expensive runs like --valgrind.

Thanks,
Jonathan

  reply	other threads:[~2019-09-16 23:13 UTC|newest]

Thread overview: 63+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-27  5:17 Jeff King
2019-08-31  7:58 ` Christian Couder
2019-08-31 19:44   ` Olga Telezhnaya
2019-09-04 19:41 ` Jeff King
2019-09-05  7:24   ` Christian Couder
2019-09-05 19:39   ` Emily Shaffer
2019-09-06 11:55     ` Carlo Arenas
2019-09-07  6:39       ` Jeff King
2019-09-07 10:13         ` Carlo Arenas
2019-09-07  6:36     ` Jeff King
2019-09-08 14:56   ` Pratyush Yadav
2019-09-09 17:00     ` Jeff King
2019-09-23 18:07   ` SZEDER Gábor
2019-09-26  9:47     ` SZEDER Gábor
2019-09-26 19:32       ` Johannes Schindelin
2019-09-26 21:54         ` SZEDER Gábor
2019-09-26 11:42     ` Johannes Schindelin
2019-09-13 20:03 ` Jonathan Tan
2019-09-13 20:51   ` Jeff King
2019-09-16 18:42     ` Emily Shaffer
2019-09-16 21:33       ` Eric Wong
2019-09-16 21:44       ` SZEDER Gábor
2019-09-16 23:13         ` Jonathan Nieder [this message]
2019-09-17  0:59           ` Jeff King
2019-09-17 11:23       ` Johannes Schindelin
2019-09-17 12:02         ` SZEDER Gábor
2019-09-23 12:47           ` Johannes Schindelin
2019-09-23 16:58             ` SZEDER Gábor
2019-09-26 11:04               ` Johannes Schindelin
2019-09-26 13:28                 ` SZEDER Gábor
2019-09-26 19:39                   ` Johannes Schindelin
2019-09-26 21:44                     ` SZEDER Gábor
2019-09-27 22:18                       ` Jeff King
2019-10-09 17:25                         ` SZEDER Gábor
2019-10-11  6:34                           ` Jeff King
2019-09-23 18:19             ` Jeff King
2019-09-24 14:30               ` Johannes Schindelin
2019-09-17 15:10         ` Christian Couder
2019-09-23 12:50           ` Johannes Schindelin
2019-09-23 19:30           ` Jeff King
2019-09-23 18:07         ` Jeff King
2019-09-24 14:25           ` Johannes Schindelin
2019-09-24 15:33             ` Jeff King
2019-09-28  3:56               ` Junio C Hamano
2019-09-24  0:55         ` Eric Wong
2019-09-26 12:45           ` Johannes Schindelin
2019-09-30  8:55             ` Eric Wong
2019-09-28  4:01           ` Junio C Hamano
2019-09-20 17:04     ` Jonathan Tan
2019-09-21  1:47       ` Emily Shaffer
2019-09-23 14:23         ` Christian Couder
2019-09-23 19:40         ` Jeff King
2019-09-23 22:29           ` Philip Oakley
2019-10-22 21:16         ` Emily Shaffer
2019-09-23 11:49       ` Christian Couder
2019-09-23 17:58         ` Jonathan Tan
2019-09-23 19:27           ` Jeff King
2019-09-23 20:48             ` Jonathan Tan
2019-09-23 19:15       ` Jeff King
2019-09-23 20:38         ` Jonathan Tan
2019-09-23 21:28           ` Jeff King
2019-09-24 17:07             ` Jonathan Tan
2019-09-26  7:09               ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190916231348.GB67467@google.com \
    --to=jrnieder@gmail.com \
    --cc=emilyshaffer@google.com \
    --cc=git@vger.kernel.org \
    --cc=jonathantanmy@google.com \
    --cc=peff@peff.net \
    --cc=szeder.dev@gmail.com \
    --subject='Re: Git in Outreachy December 2019?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).