From: Jeff King <email@example.com> To: Jonathan Nieder <firstname.lastname@example.org> Cc: "SZEDER Gábor" <email@example.com>, "Emily Shaffer" <firstname.lastname@example.org>, "Jonathan Tan" <email@example.com>, firstname.lastname@example.org Subject: Re: Git in Outreachy December 2019? Date: Mon, 16 Sep 2019 20:59:28 -0400 [thread overview] Message-ID: <20190917005928.GA27926@sigill.intra.peff.net> (raw) In-Reply-To: <20190916231348.GB67467@google.com> On Mon, Sep 16, 2019 at 04:13:49PM -0700, Jonathan Nieder wrote: > Most tests use "setup" or "set up" in the names of test assertions > that are required by later tests. It's very helpful for debugging and > maintenance to be able to skip or reorder some tests, so I've been > able to rely on this a bit. Of course there's no automated checking > in place for that, so there are plenty of test scripts that are > exceptions to it. > > If we introduce a test_setup helper, then we would not have to rely on > convention any more. A test_setup test assertion would represent a > "barrier" that all later tests in the file can rely on. We could > introduce some automated checking that these semantics are respected, > and then we get a maintainability improvement in every test script > that uses test_setup. (In scripts without any test_setup, treat all > test assertions as barriers since they haven't been vetted.) > > With such automated tests in place, we can then try updating all tests > that say "setup" or "set up" to use test_setup and see what fails. > > Some other tests cannot run in parallel for other reasons (e.g. HTTP > tests). These can be declared as such, and then we have the ability > to run arbitrary individual tests in parallel. This isn't quite the same, but couldn't we get most of the gain just by splitting the tests into more scripts? As you note, we already run those in parallel, so it increases the granularity of our parallelism. And you don't have to worry about skipping tests 1 through 18 if they're in another file; you just don't consider them at all. It also Just Works with things like HTTP, which choose ports under the assumption that the other tests are running simultaneously. It doesn't help with the case that test 1 does setup, and then tests 2, 3, and 4 are logically independent (and some could be skipped or not). If anybody is interested in splitting up scripts, the obvious ones to look at are the ones that take the longest (t9001 takes 55s on my system, though the whole suite runs in only 95s). Of course you can get most of the parallelism benefit by using "prove --state=slow,save", which ends up with lots of short scripts at the end (rather than one slow one chewing one CPU while the rest sit idle). > Most of the time in a test run involves multiple test scripts running > in parallel already, so this isn't a huge win for the time to complete > a normal test run. It helps more with expensive runs like --valgrind. Two easier suggestions than trying to make --valgrind faster: - use SANITIZE=address, which is way cheaper than valgrind (and catches more things!) - use --valgrind-only=17 to run everything else in "fast" mode, but check the test you care about -Peff
next prev parent reply other threads:[~2019-09-17 0:59 UTC|newest] Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-08-27 5:17 Jeff King 2019-08-31 7:58 ` Christian Couder 2019-08-31 19:44 ` Olga Telezhnaya 2019-09-04 19:41 ` Jeff King 2019-09-05 7:24 ` Christian Couder 2019-09-05 19:39 ` Emily Shaffer 2019-09-06 11:55 ` Carlo Arenas 2019-09-07 6:39 ` Jeff King 2019-09-07 10:13 ` Carlo Arenas 2019-09-07 6:36 ` Jeff King 2019-09-08 14:56 ` Pratyush Yadav 2019-09-09 17:00 ` Jeff King 2019-09-23 18:07 ` SZEDER Gábor 2019-09-26 9:47 ` SZEDER Gábor 2019-09-26 19:32 ` Johannes Schindelin 2019-09-26 21:54 ` SZEDER Gábor 2019-09-26 11:42 ` Johannes Schindelin 2019-09-13 20:03 ` Jonathan Tan 2019-09-13 20:51 ` Jeff King 2019-09-16 18:42 ` Emily Shaffer 2019-09-16 21:33 ` Eric Wong 2019-09-16 21:44 ` SZEDER Gábor 2019-09-16 23:13 ` Jonathan Nieder 2019-09-17 0:59 ` Jeff King [this message] 2019-09-17 11:23 ` Johannes Schindelin 2019-09-17 12:02 ` SZEDER Gábor 2019-09-23 12:47 ` Johannes Schindelin 2019-09-23 16:58 ` SZEDER Gábor 2019-09-26 11:04 ` Johannes Schindelin 2019-09-26 13:28 ` SZEDER Gábor 2019-09-26 19:39 ` Johannes Schindelin 2019-09-26 21:44 ` SZEDER Gábor 2019-09-27 22:18 ` Jeff King 2019-10-09 17:25 ` SZEDER Gábor 2019-10-11 6:34 ` Jeff King 2019-09-23 18:19 ` Jeff King 2019-09-24 14:30 ` Johannes Schindelin 2019-09-17 15:10 ` Christian Couder 2019-09-23 12:50 ` Johannes Schindelin 2019-09-23 19:30 ` Jeff King 2019-09-23 18:07 ` Jeff King 2019-09-24 14:25 ` Johannes Schindelin 2019-09-24 15:33 ` Jeff King 2019-09-28 3:56 ` Junio C Hamano 2019-09-24 0:55 ` Eric Wong 2019-09-26 12:45 ` Johannes Schindelin 2019-09-30 8:55 ` Eric Wong 2019-09-28 4:01 ` Junio C Hamano 2019-09-20 17:04 ` Jonathan Tan 2019-09-21 1:47 ` Emily Shaffer 2019-09-23 14:23 ` Christian Couder 2019-09-23 19:40 ` Jeff King 2019-09-23 22:29 ` Philip Oakley 2019-10-22 21:16 ` Emily Shaffer 2019-09-23 11:49 ` Christian Couder 2019-09-23 17:58 ` Jonathan Tan 2019-09-23 19:27 ` Jeff King 2019-09-23 20:48 ` Jonathan Tan 2019-09-23 19:15 ` Jeff King 2019-09-23 20:38 ` Jonathan Tan 2019-09-23 21:28 ` Jeff King 2019-09-24 17:07 ` Jonathan Tan 2019-09-26 7:09 ` Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190917005928.GA27926@sigill.intra.peff.net \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --subject='Re: Git in Outreachy December 2019?' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).