From: Jeff King <firstname.lastname@example.org> To: "SZEDER Gábor" <email@example.com> Cc: Johannes Schindelin <Johannes.Schindelin@gmx.de>, Emily Shaffer <firstname.lastname@example.org>, Jonathan Tan <email@example.com>, firstname.lastname@example.org Subject: Re: Git in Outreachy December 2019? Date: Fri, 11 Oct 2019 02:34:43 -0400 [thread overview] Message-ID: <20191011063443.GB25741@sigill.intra.peff.net> (raw) In-Reply-To: <20191009172551.GI29845@szeder.dev> On Wed, Oct 09, 2019 at 07:25:51PM +0200, SZEDER Gábor wrote: > > Probably they'd be easy enough to fix (and they're out of tree anyway), > > so I'm not really arguing against the escape hatch exactly. Mostly I'm > > just surprised that if I introduced 3 cases (out of probably a dozen > > scripts), I'm surprised that more contributors aren't accidentally doing > > so upstream. > > I see it a bit differently. Over a decade we gathered about > twenty-something such tests cases: that's about two cases per year. > You added three such cases in about a year and a half: that's two > cases per year. The numbers add up perfectly, you singlehandedly took > care of everything ;) Those cases are actually much older than that. I just didn't bother to clean them up until recently. So my rate is even lower :) > - Some shells do include file descriptor redirections in the trace > output, and it varies between implementations to which fd the > trace of the redirection goes. > > - 'ksh/ksh93' and NetBSD's /bin/sh send the trace of > redirections to the "wrong" fd, in the sense that e.g. the > trace of commands invoked in 'test_must_fail' goes to the > function's standard error, and checking its stderr with > 'test_cmp' would then fail. > > (But 'ksh/ksh93' doesn't really matter, because they don't > support the 'local' keyword, so they fail a bunch of tests > even without '-x' anyway.) > > I don't think we can do anything about these shells. Yeah, unless somebody is complaining, I don't know that it's worth worrying about too much. The test suite is certainly useful without being able to use "-x" on every single test run (you can still run it without "-x" obviously, or selectively use "-x" to debug a single test or script). So if it is only unreliable on a few tests on a few obscure shells, we can probably live with it until somebody demonstrates a real-world problem (e.g., that they're running automated CI on an obscure platform that is stuck with an old shell, and really want "-x --verbose-log" to get more verbose failures). > - We do call 'test_have_prereq' from within test cases as well, > notably from the 'test_i18ngrep', 'test_i18ncmp' and > 'test_ln_s_add' helper functions. In those cases all trace output > from 'test_have_prereq' is included in the test case's trace > output, which means that during the first invocation: > > - there is lots of distracting and confusing trace output, as > the script evaluating the prereq is passed around to a bunch > of functions. Yeah, I think this is probably an issue even with bash. > As far as 'test_i18ngrep' is concerned, which accounts for the > majority of 'test_have_prereq' invocations within test cases, I > don't understand why it uses 'test_have_prereq' in the first place > instead of checking the GIT_TEST_GETTEXT_POISON environment > variable; and 6cdccfce1e (i18n: make GETTEXT_POISON a runtime > option, 2018-11-08) doesn't give me any insight. I think it's just that checking the environment variable is non-trivial: we invoke env--helper to handle bool interpretation. So we'd prefer to cache the result (and not to run it at all if a test script doesn't use i18ngrep, though it's perhaps ubiquitous enough that we should just run it up front for every script). > I recall that some months ago we discussed the idea of how to > disable trace output from within test helper functions; that would > help with this 'test_have_prereq' issue as well, at least in case > of the more "common" shells. That might be worth doing, though IIRC it got kind of hairy. :) -Peff
next prev parent reply other threads:[~2019-10-11 6:34 UTC|newest] Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-08-27 5:17 Jeff King 2019-08-31 7:58 ` Christian Couder 2019-08-31 19:44 ` Olga Telezhnaya 2019-09-04 19:41 ` Jeff King 2019-09-05 7:24 ` Christian Couder 2019-09-05 19:39 ` Emily Shaffer 2019-09-06 11:55 ` Carlo Arenas 2019-09-07 6:39 ` Jeff King 2019-09-07 10:13 ` Carlo Arenas 2019-09-07 6:36 ` Jeff King 2019-09-08 14:56 ` Pratyush Yadav 2019-09-09 17:00 ` Jeff King 2019-09-23 18:07 ` SZEDER Gábor 2019-09-26 9:47 ` SZEDER Gábor 2019-09-26 19:32 ` Johannes Schindelin 2019-09-26 21:54 ` SZEDER Gábor 2019-09-26 11:42 ` Johannes Schindelin 2019-09-13 20:03 ` Jonathan Tan 2019-09-13 20:51 ` Jeff King 2019-09-16 18:42 ` Emily Shaffer 2019-09-16 21:33 ` Eric Wong 2019-09-16 21:44 ` SZEDER Gábor 2019-09-16 23:13 ` Jonathan Nieder 2019-09-17 0:59 ` Jeff King 2019-09-17 11:23 ` Johannes Schindelin 2019-09-17 12:02 ` SZEDER Gábor 2019-09-23 12:47 ` Johannes Schindelin 2019-09-23 16:58 ` SZEDER Gábor 2019-09-26 11:04 ` Johannes Schindelin 2019-09-26 13:28 ` SZEDER Gábor 2019-09-26 19:39 ` Johannes Schindelin 2019-09-26 21:44 ` SZEDER Gábor 2019-09-27 22:18 ` Jeff King 2019-10-09 17:25 ` SZEDER Gábor 2019-10-11 6:34 ` Jeff King [this message] 2019-09-23 18:19 ` Jeff King 2019-09-24 14:30 ` Johannes Schindelin 2019-09-17 15:10 ` Christian Couder 2019-09-23 12:50 ` Johannes Schindelin 2019-09-23 19:30 ` Jeff King 2019-09-23 18:07 ` Jeff King 2019-09-24 14:25 ` Johannes Schindelin 2019-09-24 15:33 ` Jeff King 2019-09-28 3:56 ` Junio C Hamano 2019-09-24 0:55 ` Eric Wong 2019-09-26 12:45 ` Johannes Schindelin 2019-09-30 8:55 ` Eric Wong 2019-09-28 4:01 ` Junio C Hamano 2019-09-20 17:04 ` Jonathan Tan 2019-09-21 1:47 ` Emily Shaffer 2019-09-23 14:23 ` Christian Couder 2019-09-23 19:40 ` Jeff King 2019-09-23 22:29 ` Philip Oakley 2019-10-22 21:16 ` Emily Shaffer 2019-09-23 11:49 ` Christian Couder 2019-09-23 17:58 ` Jonathan Tan 2019-09-23 19:27 ` Jeff King 2019-09-23 20:48 ` Jonathan Tan 2019-09-23 19:15 ` Jeff King 2019-09-23 20:38 ` Jonathan Tan 2019-09-23 21:28 ` Jeff King 2019-09-24 17:07 ` Jonathan Tan 2019-09-26 7:09 ` Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191011063443.GB25741@sigill.intra.peff.net \ --email@example.com \ --cc=Johannes.Schindelin@gmx.de \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: Git in Outreachy December 2019?' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).