Re: [PATCH v1.1] lib-submodule-update: pass OVERWRITING_FAIL

[Denton Liu <liu.denton@gmail.com>]

Hi Junio,

On Thu, Apr 30, 2020 at 01:38:59PM -0700, Junio C Hamano wrote:
> Denton Liu <liu.denton@gmail.com> writes:
> 
> > We are using `test_must_fail $command`. However, $command is not
> > necessarily a git command; it could be a test helper function.
> >
> > In an effort to stop using test_must_fail with non-git commands, instead
> > of invoking `test_must_fail $command`, run
> > `OVERWRITING_FAIL=test_must_fail $command` instead.
> 
> This description alone does not make much sense to me as a reader.
> For "when OVERWRITING_FAIL environment variable is set to
> test_must_fail, something magical happens while running $command" to
> be true, $command (which may be "a git command", or "a test helper
> function") must be aware of, and must be told how to react to the
> OVERWRITING_FAIL environment variable.  For example,
> 
> 	test_must_fail test a = b
> 	test_must_fail git cat-file leaf HEAD
> 
> may succeed because "test a = b" and "git cat-file leaf HEAD" fail
> and test_must_fail would notice that these commands exited with
> non-zero status without crashing.
> 
> But how would the same happen for these commands,
> 
> 	OVERWRITING_FAIL=test_must_fail test a = b
> 	OVERWRITING_FAIL=test_must_fail git cat-file leaf HEAD
> 
> which is what the above paragraph tells me to write "instead of
> using test_must_fail".  There is something gravely missing from the
> description.  

Because of the previous patch in this series, helper functions are
called using test_submodule_switch_func() while git commands are invoked
using test_submodule_switch() and test_submodule_forced_switch(). As a
result, helper functions that are invoked must learn to handle
$OVERWRITING_FAIL appropriately. The latter two functions now feature
something like

	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"

Does this commit message increase the clarity?

	lib-submodule-update: pass OVERWRITING_FAIL

	We are using `test_must_fail $command`. However, $command is not
	necessarily a git command; it could be a test helper function.

	In an effort to stop using test_must_fail with non-git commands, instead
	of invoking `test_must_fail $command`, run
	`OVERWRITING_FAIL=test_must_fail $command` instead in
	test_submodule_switch_common().

	In the case where $command is a git command,
	test_submodule_switch_common() is called by one of
	test_submodule_switch() or test_submodule_forced_switch(). In those two
	functions, pass $command like this:

		test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"

	In the case where $command is a test helper function, increase the
	granularity of the test helper function by marking the git invocation
	which was meant to fail like this:

		$OVERWRITING_FAIL git checkout "$1"

	This is useful because currently, when we run a test helper function, we
	just mark the whole thing as `test_must_fail`. However, it's possible
	that the helper function might fail earlier or later than expected due
	to an introduced bug. If this happens, then the test case will still
	report as passing but it should really be marked as failing since it
	didn't actually display the intended behaviour.

	While we're at it, some helper functions have git commands piping into
	another git command. Break these pipes up into two separate invocations
	with a file buffer so that the return code of the first command is not
	lost.

	This patch can be better viewed with `--ignore-all-space`.

> Is it that $command is *NEVER* a 'git' command, but just a selected
> few helper functions know how to honor this convention?

With the changes made, it is now either a helper function or an eval of
a git command, as described above.

> If that is
> the case, perhaps can we teach these helper functions an *option* to
> expect a failure instead of expecting a success?

I don't think this is possible because $command is some arbitrary
command string, in particular due to the eval stuff. I suppose we could
write something like

	test_submodule_switch_common "f () {
			# handle potential --expect-fail
			git $command"
		} && f"

but I'm not sure if this would even work. I haven't tested it and I feel
like doing this would be far too unwieldy. Also, since
test_submodule_switch_common() uses $command as the test name, I don't
think it's feasible to take this approach.

> These are all
> speculations, because the above description is too vague as a
> starting point for clear thinking.
> 
> > +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
> > +			OVERWRITING_FAIL= &&
> 
> If we have given up the "single-shot environment export" for
> compatibility reasons (which is a sound decision to follow), we
> should make sure it is clear to our readers that we are not using
> that shell feature.  I.e.
> 
> 			export OVERWRITING_FAIL=test_must_fail &&
> 			$command replace_sub1_with_directory &&
> 			unset OVERWRITING_FAIL &&

Makes sense. I would drop the export, though, because $OVERWRITING_FAIL
should only be handled within the shell environment. We're never calling
any external commands that need to know about this variable.

On a tangent, I got a response[1] from the dash people about the
"single-shot environment export" propagating past a function. It seems
like in the most updated version of POSIX addresses this and it's up to
the implementers whether or not variables propagate past a function
invocation.

Thanks,

Denton

[1]: https://www.mail-archive.com/dash@vger.kernel.org/msg01925.html