[PATCH 0/2] some minor add--interactive fixes

[PATCH 0/2] some minor add--interactive fixes

[Jeff King]

There was a mini-mystery about the add--interactive tests on the
git-mentoring list this morning. The second patch here solves the
mystery. But while digging I also noticed a bug in the new C version,
which is fixed by the first patch.

  [1/2]: add-patch: fix inverted return code of repo_read_index()
  [2/2]: add--interactive.perl: specify --no-color explicitly

 add-patch.c                | 2 +-
 git-add--interactive.perl  | 2 +-
 t/t3701-add-interactive.sh | 8 ++++++++
 3 files changed, 10 insertions(+), 2 deletions(-)

-Peff

[PATCH 1/2] add-patch: fix inverted return code of repo_read_index()

[Jeff King]

After applying hunks to a file with "add -p", the C patch_update_file()
function tries to refresh the index (just like the perl version does).
We can only refresh the index if we're able to read it in, so we first
check the return value of repo_read_index(). But unlike many functions,
where "0" is success, that function is documented to return the number
of entries in the index.  Hence we should be checking for success with a
non-negative return value.

Neither the tests nor any users seem to have noticed this, probably due
to a combination of:

  - this affects only the C version, which is not yet the default

  - following it up with any porcelain command like "git diff" or "git
    commit" would refresh the index automatically.

But you can see the problem by running the plumbing "git diff-files"
immediately after "add -p" stages all hunks. Running the new test with
GIT_TEST_ADD_I_USE_BUILTIN=1 fails without the matching code change.

Signed-off-by: Jeff King <peff@peff.net>
---
 add-patch.c                | 2 +-
 t/t3701-add-interactive.sh | 7 +++++++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/add-patch.c b/add-patch.c
index 457b8c550e..9544437247 100644
--- a/add-patch.c
+++ b/add-patch.c
@@ -1641,7 +1641,7 @@ static int patch_update_file(struct add_p_state *s,
 					 NULL, 0, NULL, 0))
 				error(_("'git apply' failed"));
 		}
-		if (!repo_read_index(s->s.r))
+		if (repo_read_index(s->s.r) >= 0)
 			repo_refresh_and_write_index(s->s.r, REFRESH_QUIET, 0,
 						     1, NULL, NULL, NULL);
 	}
diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh
index fb73a847cb..67d1c36cef 100755
--- a/t/t3701-add-interactive.sh
+++ b/t/t3701-add-interactive.sh
@@ -569,6 +569,13 @@ test_expect_success 'patch mode ignores unmerged entries' '
 	diff_cmp expected diff
 '
 
+test_expect_success 'index is refreshed after applying patch' '
+	git reset --hard &&
+	echo content >test &&
+	printf y | git add -p &&
+	git diff-files --exit-code
+'
+
 test_expect_success 'diffs can be colorized' '
 	git reset --hard &&
 
-- 
2.28.0.889.g64d3ad7607

[PATCH 2/2] add--interactive.perl: specify --no-color explicitly

[Jeff King]

Our color tests of "git add -p" do something a bit different from how a
normal user would behave: we pretend there's a pager in use, so that Git
thinks it's OK to write color to a non-tty stdout.  This comes from
8539b46534 (t3701: avoid depending on the TTY prerequisite, 2019-12-06),
which allows us to avoid a lot of complicated mock-tty code.

However, those environment variables also make their way down to
sub-processes of add--interactive, including the "diff-files" we run to
generate the patches. As a result, it thinks it should output color,
too. So in t3701.50, for example, the machine-readable version of the
diff we get unexpectedly has color in it. We fail to parse it as a diff
and think there are zero hunks.

The test does still pass, though, because even with zero hunks we'll
dump the diff header (and we consider those unparseable bits to be part
of the header!), and so the output still has the expected color codes in
it. We don't notice that the command was totally broken and failed to
apply anything.

And in fact we're not really testing what we think we are about the
color, either. While add--interactive does correctly show the version we
got from running "diff-files --color", we'd also pass the test if we had
accidentally shown the machine-readable version, too, since it
(erroneously) has color codes in it.

One could argue that the test isn't very realistic; it's setting up this
"pretend there's a pager" situation to get around the tty restrictions
of the test environment. So one option would be to move back towards
using a real tty. But the behavior of add--interactive really is
user-visible here. If a user, for whatever reason, did run "git
--paginate add --patch" (perhaps because their pager is really a filter
or something), the command would totally fail to do anything useful.

Since we know that we don't want color in this output, let's just make
add--interactive more defensive, and say "--no-color" explicitly. It
doesn't hurt anything in the common case, but it fixes this odd case and
lets our test function properly again.

Note that the C builtin run_add_p() already passes --no-color, so it
doesn't need a similar fix. That will eventually replace this perl code
anyway, but the test change here will be valuable for ensuring that.

Signed-off-by: Jeff King <peff@peff.net>
---
Curiously, run_add_p() claims that --no-color is important to override
"diff.color = always", but in a quick test that doesn't seem to make any
difference. I didn't dig into whether it's possible to trigger or not
(either way, defensively using --no-color is a reasonable precaution).

 git-add--interactive.perl  | 2 +-
 t/t3701-add-interactive.sh | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/git-add--interactive.perl b/git-add--interactive.perl
index f36c0078ac..b6cdcfef61 100755
--- a/git-add--interactive.perl
+++ b/git-add--interactive.perl
@@ -714,7 +714,7 @@ sub parse_diff {
 	if (defined $patch_mode_revision) {
 		push @diff_cmd, get_diff_reference($patch_mode_revision);
 	}
-	my @diff = run_cmd_pipe("git", @diff_cmd, "--", $path);
+	my @diff = run_cmd_pipe("git", @diff_cmd, qw(--no-color --), $path);
 	my @colored = ();
 	if ($diff_use_color) {
 		my @display_cmd = ("git", @diff_cmd, qw(--color --), $path);
diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh
index 67d1c36cef..1590cf6b98 100755
--- a/t/t3701-add-interactive.sh
+++ b/t/t3701-add-interactive.sh
@@ -582,6 +582,7 @@ test_expect_success 'diffs can be colorized' '
 	echo content >test &&
 	printf y >y &&
 	force_color git add -p >output 2>&1 <y &&
+	git diff-files --exit-code &&
 
 	# We do not want to depend on the exact coloring scheme
 	# git uses for diffs, so just check that we saw some kind of color.
-- 
2.28.0.889.g64d3ad7607

Re: [PATCH 2/2] add--interactive.perl: specify --no-color explicitly

[Jeff King]

On Mon, Sep 07, 2020 at 04:17:39AM -0400, Jeff King wrote:

> Our color tests of "git add -p" do something a bit different from how a
> normal user would behave: we pretend there's a pager in use, so that Git
> thinks it's OK to write color to a non-tty stdout.  This comes from
> 8539b46534 (t3701: avoid depending on the TTY prerequisite, 2019-12-06),
> which allows us to avoid a lot of complicated mock-tty code.
> 
> However, those environment variables also make their way down to
> sub-processes of add--interactive, including the "diff-files" we run to
> generate the patches. As a result, it thinks it should output color,
> too. So in t3701.50, for example, the machine-readable version of the
> diff we get unexpectedly has color in it. We fail to parse it as a diff
> and think there are zero hunks.

By the way, this is an instance of a more generic bug, which is that:

  git -p my-script

will cause any sub-programs of git-my-script to think their stdout is
going to a pager, even if my-script redirects them to a file or another
pipe.

I had a solution long ago in:

  https://lore.kernel.org/git/20150810052353.GB15441@sigill.intra.peff.net/

but it raises a bunch of interesting portability questions. Since this
comes up so rarely, I never really pursued it further.

-Peff

Re: [PATCH 2/2] add--interactive.perl: specify --no-color explicitly

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> By the way, this is an instance of a more generic bug, which is that:
>
>   git -p my-script
>
> will cause any sub-programs of git-my-script to think their stdout is
> going to a pager, even if my-script redirects them to a file or another
> pipe.
>
> I had a solution long ago in:
>
>   https://lore.kernel.org/git/20150810052353.GB15441@sigill.intra.peff.net/
>
> but it raises a bunch of interesting portability questions. Since this
> comes up so rarely, I never really pursued it further.

That indeed is from long ago ;-) and I still kind of like it.  Is
the issue that some platforms do not even have inum of the "thing"
connected to a file descriptor?  I think a fallback code for such a
system can just return a constant pipe ID to anybody asks, so that
an exported GIT_PAGER_PIPE_ID would always match, which gives the
behaviour identical to what we currently do with GIT_PAGER_IN_USE,
and that would be one way to help those capable of giving useful IDs
while not harming others.

Thanks.

Re: [PATCH 2/2] add--interactive.perl: specify --no-color explicitly

[Johannes Schindelin]

Hi Peff,

ACK on both patches, just one clarification:

On Mon, 7 Sep 2020, Jeff King wrote:

> [...]
>
> One could argue that the test isn't very realistic; it's setting up this
> "pretend there's a pager" situation to get around the tty restrictions
> of the test environment. So one option would be to move back towards
> using a real tty. [...]

The main reason why we moved away from the "real" TTY is that the TTY we
simulate in `t/test-terminal.perl` is not actually real at all on Windows.
Instead, it is the Cygwin/MSYS2-emulated version of a PTY, and as a
consequence `git.exe` simply has no way of accessing it.

Which means that by moving "back towards using a real tty" we would lose
all test coverage on Windows, which is not a thought I like to entertain.

Ciao,
Dscho

Re: [PATCH 2/2] add--interactive.perl: specify --no-color explicitly

[Jeff King]

On Tue, Sep 08, 2020 at 08:51:31PM +0200, Johannes Schindelin wrote:

> > One could argue that the test isn't very realistic; it's setting up this
> > "pretend there's a pager" situation to get around the tty restrictions
> > of the test environment. So one option would be to move back towards
> > using a real tty. [...]
> 
> The main reason why we moved away from the "real" TTY is that the TTY we
> simulate in `t/test-terminal.perl` is not actually real at all on Windows.
> Instead, it is the Cygwin/MSYS2-emulated version of a PTY, and as a
> consequence `git.exe` simply has no way of accessing it.
> 
> Which means that by moving "back towards using a real tty" we would lose
> all test coverage on Windows, which is not a thought I like to entertain.

Yeah, regressing the improvements done by 8539b46534 (t3701: avoid
depending on the TTY prerequisite, 2019-12-06) was definitely part of my
motivation, but I guess I didn't spell that out completely.

-Peff