Re: [PATCH] t4204 is not sanitizer clean at all

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Thu, Dec 16 2021, Junio C Hamano wrote:

> Earlier we marked that this patch-id test is leak-sanitizer clean,
> but if we read the test script carefully, it is merely because we
> have too many invocations of commands in the "git log" family on the
> upstream side of the pipe, hiding breakages from them.
>
> Split the pipeline so that breakages from these commands can be
> caught (not limited to aborts due to leak-sanitizer) and unmark
> the script as not passing the test with leak-sanitizer in effect.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>
>  A quick grep tells me that tests 3302, 3303, 3305, 4020 and 6236
>  all use "git log" and still are marked as passing tests with
>  leak-sanitizer in effect.  I've taken a deep look at none of them,
>  but I suspect they share the same kind of breakage.

This change looks good to me.

FWIW this is not a mistake on my part, but something I'm perfectly aware
of. I don't consider it to be "brekage".

We have plenty of place in the test suite where we hide exit codes on
the LHS of a pipe, or where we call a function that doesn't &&-chain its
git invocations.

In those cases we can and usually will "succeed" under LSAN, because it
allows the program to emit its full output, and will abort() at the very
end.

I have an unsubmitted logging mode (using LSAN_OPTIONS=log_path=<path>)
where I log every one of these to test-results/*, there's a lot more of
these.

But in the meantime I think the best way forward is to gradually mark
the tests that pass with LSAN as passing, to ensure that we at least
don't have regressions in the meantime. Before this we'd at least check
the "git checkout" etc. for leaks.

If I made fixing all broken &&-chains or git on the LHS of a pipe a
prerequisite for marking as passing under under LSAN I'd end up with
something that's approximately the size of [1] and more (i.e. Eric's
upcoming patches to do that).

I don't see why we'd consider perfect the enemy of the good in these
cases. Yes we won't catch the successful exit of every single git
invocation, but our tests aren't doing that now, LSAN or not. But until
that's fixed we'll at least catch some, which helps our overall memory
leak regression coverage.

More importantly it makes it a lot easier to reason about future memory
leak patches, as we'll be able to get to a 1=1 mapping of tests that
pass, and those that are marked being known to pass. I'm using that
locally to fake-fail those that start passing unexpectedly that aren't
on the list, which then helps to inform the addition of "this test now
passes with no leaks".

1. https://lore.kernel.org/git/20211213063059.19424-1-sunshine@sunshineco.com/