Re: [PATCH] t7518: fix flaky grep invocation

[Elijah Newren <newren@gmail.com>]

On Fri, Oct 16, 2020 at 5:02 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > output, but on one automatic CI run I observed the following output:
> >
> > fatal: empty ident name (for <runner@fv-az128-670.gcliasfzo2nullsdbrimjtbyhg.cx.internal.cloudapp.net>) not allowed
> >
> > Note that 'null' appears as a substring of the domain name, found
> > within 'gcliasfzo2nullsdbrimjtbyhg'.  Tighten the test by searching for
> > "(null)" rather than "null".
>
> Why do we even need grep again?  If we were to segfault, wouldn't
> test_must_fail catch it for us?
>
> ... ahh, OK, your libc may allow NULL left in email and that is what
> we are trying to catch?

Yes, as mentioned in the commit message:

    ...was trying to make sure that
    the test with an empty ident did not segfault and did not result in
    glibc quiety translating a NULL pointer into a name of "(null)".

This was also taken from the comment in the file just above the code
in question (though it was just barely outside the context region).

> Honestly, I am not sure if it even makes sense to test it like this.
> The code that the test tries to protect against future breakages
> roughly look like this these days.
>
>         if (!email) {
>                 ... one way to assign non-NULL to email ...
>         }
>         if (!email) {
>                 ... do other things to assign non-NULL to email or die ...
>         }
>
>         if (want_name) {
>                 ... here we require email to be set because we show
>                 ... it in an error message
>         }
>
> The original problem was that the code had "if (want_name)" part
> first before email==NULL condition has been dealt with, and used
> email==NULL in one of the error messages.  The fix was to move the
> part that deals with email==NULL up, as it does not need any of the
> effects that happen in "if (want_name)" block.
>
> Now, what is this particular test protecting against?  We may again
> move the "if (want_name)" block up and trigger the error message in
> it that uses e-mail while having NULL in email again?  If so, would
> it make more sense to do something along this line, lose the "grep"
> and keep "test_must_fail"?  We expect and require at this point in
> the code (the patch is inside that "if (want_name)" block) email
> must have been set up already, so it would be a BUG() otherwise.
>
> Hmm?

It may come down to the angle you view it from; "the current code
doesn't have such a problem and can't within its newer design, so why
test it?" vs. "when you add a regression test at the time you make a
bugfix, the regression test should demonstrate the failure with the
old version of the code".  Without the grep, the test would not have
failed on the original old version of the code, at least not on some
platforms.
I'm guessing that's why Peff put it there, and still had a preference
to leave the grep in (see his comments on this issue at
https://lore.kernel.org/git/20201016200031.GA3355643@coredump.intra.peff.net/;
this is a case where it'd be nice if gitgitgadget accepted an
in-reply-to flag of some sort).

However, although I have a slight preference to fixing what grep is
searching for, my primary concern was with avoiding flakiness in the
test.  I'm happy if someone wants to make additional code changes or
tweak the testcase here so long as we avoid the flakiness.

>  ident.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git i/ident.c w/ident.c
> index 6aba4b5cb6..9b71563d95 100644
> --- i/ident.c
> +++ w/ident.c
> @@ -430,6 +430,8 @@ const char *fmt_ident(const char *name, const char *email,
>                         if (strict) {
>                                 if (using_default)
>                                         ident_env_hint(whose_ident);
> +                               if (!email)
> +                                       BUG("NULL email when we need to complain???");
>                                 die(_("empty ident name (for <%s>) not allowed"), email);
>                         }
>                         pw = xgetpwuid_self(NULL);