Re: [PATCH v3 6/7] t/Makefile: run unit tests alongside shell tests

[Jeff King <peff@peff.net>]

On Thu, Apr 11, 2024 at 11:44:09AM -0700, Josh Steadmon wrote:

> > An earlier step required that runs via "test-tool run-command" have
> > TEST_SHELL_PATH set correctly. So defaulting to /bin/sh here is
> > pointless, I'd think? This is used only for the in-Makefile "prove"
> > invocation, so running individual tests or even a manual "prove" outside
> > of the Makefile (where the user might not have set TEST_SHELL_PATH)
> > would not apply.
> 
> Actually, I think the "manual prove outside of the Makefile" situation
> is worth keeping this. I know I sometimes copy commands from Makefiles
> and run them manually when debugging issues, so it could be annoying for
> folks if we remove the default here.

Hmm, by "manually running prove" I meant running:

  prove t0001-init.sh

and so on. Or even "prove --state=failed" to re-run failed tests. But
neither of those would even use this script, because there's no --exec
option.

But it sounds like you mean literally cutting and pasting the "prove
--exec" line from the Makefile. That seems to me like a weird thing to
want to do, but OK, I'll try not to judge your workflow. ;)

But if you are worried about making debugging more confusing, it seems
like silently defaulting to /bin/sh might make things worse. It is not
necessarily what "make test" did, and complaining loudly might be more
helpful than trying to run with an alternate shell.

I don't feel too strongly about it, though. I'd generally just run
single tests as "./t0001-init.sh", which runs into the same issue. I've
been working on Git long enough that I know it is one of the possible
gotchas when a test failure does not reproduce. :)

-Peff