Re: [PATCH 0/6] Add merge recursive testcases with undetected conflicts

[Elijah Newren <newren@gmail.com>]

On Mon, Jul 9, 2018 at 9:44 PM, Jeff King <peff@peff.net> wrote:
> On Mon, Jul 09, 2018 at 01:22:29PM -0700, Elijah Newren wrote:
>
>> Oh, I didn't know about test-lint.  Is there a place that documents
>> the various checks you run, so I can avoid slowing you down?  Ones I
>> know about:
>>
>> Already documented:
>>   * `make DEVELOPER=1` (from CodingGuidelines)
>>   * running tests (from SubmittingPatches)
>>
>> Stuff I've seen you mention in emails over time:
>>   * linux/scripts/checkpatch.pl
>>   * git grep -e '\<inline\>' --and --not -e 'static inline' -- \*.h
>>   * make -C t/ test-lint
>
> test-lint is supposed to be run automatically as part of "make test" (or
> "make prove"), unless you've specifically disabled it by setting
> TEST_LINT. And it does complain for me with your patches. If it doesn't
> for you, then we have a bug to fix. :)

Oh, this may be my bad.  Years ago someone pointed out that the
testsuite could be run under 'prove', which provided nicer output and
made sure to run the longest tests (e.g. the horrifically slow
t9001-send-email.sh) first.  So my test alias is:

   time prove -j7 --timer --state failed,slow,save t[0-9]*.sh ::
"--root=/dev/shm"

(with possibly different -j settings on different machines) and I just
stopped running make test.  Didn't learn about the 'make prove'
target, even though it's apparently now been there for nearly 8 years.

> I won't be surprised, though, if you just ran "./t6036" manually before
> sending, since your patches literally didn't touch any other files.

That may also be true, though I would have missed it even if I had
code changes due to not being aware of 'make prove'.

> In theory we could push some of the linting down into the test scripts
> themselves (some of it, like the &&-linter, is there already by
> necessity). But it might also end up annoying, since usually dropping
> down to manual single-test runs means you're trying to debug something,
> and extra linting processes could get in the way.
>
>> Are there others?
>
> I like:
>
>   make SANITIZE=address,undefined test
>
> though it's pretty heavy-weight (but not nearly as much as valgrind).
> You probably also need BLK_SHA1=Yes, since the default DC_SHA1 has some
> unaligned loads that make UBSan complain. We should maybe teach the
> Makefile to do that by default.
>
> I've also been playing with clang's scan-build. It _did_ find a real bug
> recently, but it has a bunch of false positives.
>
> Stefan runs Coverity against pu periodically. IIRC It's a pain to run
> yourself, but the shared results can be mailed to you, or you can poke
> around at https://scan.coverity.com/projects/git. That _also_ has a ton
> of false positives, but it's good about cataloguing them so the periodic
> email usually just mentions the new ones.

Cool, thanks for the pointers.