Bug or unexpected behaviour in git show <rev>:a\b

Bug or unexpected behaviour in git show <rev>:a\b

[David Burström]

Hi!

As part of some experiments on running git on Windows, I found
something unexpected. If I use e.g. bash to run the following in a
repository:

git show HEAD:a\\b

the command stalls for a short while and then exits with 0 and prints
nothing on stdout. I've verified this on git 2.25.0 and 2.21.0. I
would have expected either "fatal: Path 'a\b' does not exist in
'HEAD'" or something of that nature. Nothing in "man git revisions"
hint that backslash is treated specially. Is this a bug or is it an
undocumented feature?

Best regards,
  David Burström

Re: Bug or unexpected behaviour in git show <rev>:a\b

[Jeff King]

On Fri, Jan 24, 2020 at 01:45:28PM +0100, David Burström wrote:

> As part of some experiments on running git on Windows, I found
> something unexpected. If I use e.g. bash to run the following in a
> repository:
> 
> git show HEAD:a\\b
> 
> the command stalls for a short while and then exits with 0 and prints
> nothing on stdout. I've verified this on git 2.25.0 and 2.21.0. I
> would have expected either "fatal: Path 'a\b' does not exist in
> 'HEAD'" or something of that nature. Nothing in "man git revisions"
> hint that backslash is treated specially. Is this a bug or is it an
> undocumented feature?

Maybe both. :) This is an interesting case.

The arguments to git-show (like anything that invokes the revision
machinery, like git-log) can be _either_ an object identifier or a
pathspec. Since you didn't provide a "--" separator to disambiguate,
we'll try both.

For your case "HEAD:a\b", as well as the more mundane "HEAD:a-b",
get_oid() realizes they are not valid object identifiers (because the
files don't exist in HEAD^{tree}). So then we consider it as a pathspec.

In the case of "HEAD:a-b", no such file exists in the working tree
either, so we complain:

 $ git show HEAD:a-b
 fatal: Path 'a-b' does not exist in 'HEAD'

and here verify_filename(), which produces the error, is smart enough to
diagnose that the bogus argument looks a lot like a request for a
path-in-tree.

But since 28fcc0b71a (pathspec: avoid the need of "--" when wildcard is
used, 2015-05-02), we allow non-files as pathspecs if they use magic
glob meta-characters (to make it more convenient to use "*.c" or
similar). And "\" is such a meta-character. So we actually diff HEAD
using the pathspec "HEAD:a\b", which of course matches nothing.

You can defeat this by using "--". I.e.:

  [mundane name, dwim logic complains because file doesn't exist]
  $ git show HEAD:a-b
  fatal: Path 'a-b' does not exist in 'HEAD'

  [mundane name, explicit pathspec does diff with no matches]
  $ git show -- HEAD:a-b

  [mundane name, explicit revision complains that it can't be resolved]
  $ git show HEAD:a-b --
  fatal: bad revision 'HEAD:a-b'

  [exotic name, dwim considers it a pathspec due to metacharacter]
  $ git show HEAD:a\\b

  [exotic name, explicit pathspec does the same thing]
  $ git show -- HEAD:a\\b

  [exotic name, explicit revision is just like the mundane case]
  $ git show HEAD:a\\b --
  fatal: bad revision 'HEAD:a\b'

So everything is working as designed, or at least explainable. But I
think there is some room for improvement. A backslash that isn't
followed by a glob meta-character _is_ still a meta character (your
"a\b" would be globbing for "ab"). But it's useless enough that I think
it shouldn't be enough to trigger the "oh, you probably meant this as a
pathspec" DWIM rule.

We _could_ also say "even though this could technically be a pathspec
because of its metacharacter, it looks vaguely enough like a
path-in-tree revision that we shouldn't guess". That I'm less
comfortable with, just because it makes the heuristics even more
magical.

Also, unrelated to your problem, but I find it interesting in the output
above that "git show HEAD:foo --" produces a less useful error message
than "git show HEAD:foo" without the separator, even though the user has
given us even more information about their intent. I think the DWIM
verify_filename() has grown a lot more diagnosis code over the years
that could also be used in the other code path.

-Peff

Re: Bug or unexpected behaviour in git show <rev>:a\b

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> So everything is working as designed, or at least explainable. But I
> think there is some room for improvement. A backslash that isn't
> followed by a glob meta-character _is_ still a meta character (your
> "a\b" would be globbing for "ab"). But it's useless enough that I think
> it shouldn't be enough to trigger the "oh, you probably meant this as a
> pathspec" DWIM rule.

This sounds sensible.

> We _could_ also say "even though this could technically be a pathspec
> because of its metacharacter, it looks vaguely enough like a
> path-in-tree revision that we shouldn't guess". That I'm less
> comfortable with, just because it makes the heuristics even more
> magical.

Not just it becomes more magical, I am afraid that the code to
implement such a heuristics would be fragile and become a source of
unnecessary bugs.  Let's not go there.

I should learn to use "working as designed or at least explainable"
more often in my responses, by the way.  That's quite a useful and
good phrase ;-)

Re: Bug or unexpected behaviour in git show <rev>:a\b

[Jeff King]

On Fri, Jan 24, 2020 at 11:27:35AM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > So everything is working as designed, or at least explainable. But I
> > think there is some room for improvement. A backslash that isn't
> > followed by a glob meta-character _is_ still a meta character (your
> > "a\b" would be globbing for "ab"). But it's useless enough that I think
> > it shouldn't be enough to trigger the "oh, you probably meant this as a
> > pathspec" DWIM rule.
> 
> This sounds sensible.

OK, the patch I came up with is below.

> > We _could_ also say "even though this could technically be a pathspec
> > because of its metacharacter, it looks vaguely enough like a
> > path-in-tree revision that we shouldn't guess". That I'm less
> > comfortable with, just because it makes the heuristics even more
> > magical.
> 
> Not just it becomes more magical, I am afraid that the code to
> implement such a heuristics would be fragile and become a source of
> unnecessary bugs.  Let's not go there.

OK. It does mean that:

  git show HEAD:a*

will still quietly produce no output instead of saying "hey, there is no
a* in HEAD". But I think given the lack of bug reports over the years
that this case (and the backslash one I'm fixing) are probably
relatively rare.  The backslash one seems a lot more likely, just
because Windows folks may treat it like a path separator (I'm not sure
if that even works, considering its meaning in a glob, but certainly I
can imagine somebody doing so as an experiment and getting confused by
the result).

> I should learn to use "working as designed or at least explainable"
> more often in my responses, by the way.  That's quite a useful and
> good phrase ;-)

Perhaps that can be Git's motto. ;)

Anyway, here's the patch. Even though this is rare, I think it's worth
doing. The code is simple and I don't anticipate anybody complaining
about the tightening.

-- >8 --
Subject: verify_filename(): handle backslashes in "wildcards are pathspecs" rule

Commit 28fcc0b71a (pathspec: avoid the need of "--" when wildcard is
used, 2015-05-02) allowed:

  git rev-parse '*.c'

without the double-dash. But the rule it uses to check for wildcards
actually looks for any glob special. This is overly liberal, as it means
that a pattern that doesn't actually do any wildcard matching, like
"a\b", will be considered a pathspec.

If you do have such a file on disk, that's presumably what you wanted.
But if you don't, the results are confusing: rather than say "there's no
such path a\b", we'll quietly accept it as a pathspec which very likely
matches nothing (or at least not what you intended). Likewise, looking
for path "a\*b" doesn't expand the search at all; it would only find a
single entry, "a*b".

This commit switches the rule to trigger only when glob metacharacters
would expand the search, meaning both of those cases will now report an
error (you can still disambiguate using "--", of course; we're just
tightening the DWIM heuristic).

Note that we didn't test the original feature in 28fcc0b71a at all. So
this patch not only tests for these corner cases, but also adds a
regression test for the existing behavior.

Reported-by: David Burström <davidburstrom@spotify.com>
Signed-off-by: Jeff King <peff@peff.net>
---
 setup.c                        | 23 ++++++++++++++++++++---
 t/t1506-rev-parse-diagnosis.sh | 14 ++++++++++++++
 2 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/setup.c b/setup.c
index e2a479a64f..12228c0d9c 100644
--- a/setup.c
+++ b/setup.c
@@ -197,9 +197,26 @@ static void NORETURN die_verify_filename(struct repository *r,
  */
 static int looks_like_pathspec(const char *arg)
 {
-	/* anything with a wildcard character */
-	if (!no_wildcard(arg))
-		return 1;
+	const char *p;
+	int escaped = 0;
+
+	/*
+	 * Wildcard characters imply the user is looking to match pathspecs
+	 * that aren't in the filesystem. Note that this doesn't include
+	 * backslash even though it's a glob special; by itself it doesn't
+	 * cause any increase in the match. Likewise ignore backslash-escaped
+	 * wildcard characters.
+	 */
+	for (p = arg; *p; p++) {
+		if (escaped) {
+			escaped = 0;
+		} else if (is_glob_special(*p)) {
+			if (*p == '\\')
+				escaped = 1;
+			else
+				return 1;
+		}
+	}
 
 	/* long-form pathspec magic */
 	if (starts_with(arg, ":("))
diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
index 6d951ca015..8a75f37a11 100755
--- a/t/t1506-rev-parse-diagnosis.sh
+++ b/t/t1506-rev-parse-diagnosis.sh
@@ -222,4 +222,18 @@ test_expect_success 'reject Nth ancestor if N is too high' '
 	test_must_fail git rev-parse HEAD~100000000000000000000000000000000
 '
 
+test_expect_success 'pathspecs with wildcards are not ambiguous' '
+	echo "*.c" >expect &&
+	git rev-parse "*.c" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'backslash does not trigger wildcard rule' '
+	test_must_fail git rev-parse "foo\\bar"
+'
+
+test_expect_success 'escaped char does not trigger wildcard rule' '
+	test_must_fail git rev-parse "foo\\*bar"
+'
+
 test_done
-- 
2.25.0.421.gb74d19af79

Re: Bug or unexpected behaviour in git show <rev>:a\b

[David Burström]

On Sat, 25 Jan 2020 at 01:00, Jeff King <peff@peff.net> wrote:
> > > We _could_ also say "even though this could technically be a pathspec
> > > because of its metacharacter, it looks vaguely enough like a
> > > path-in-tree revision that we shouldn't guess". That I'm less
> > > comfortable with, just because it makes the heuristics even more
> > > magical.
> >
> > Not just it becomes more magical, I am afraid that the code to
> > implement such a heuristics would be fragile and become a source of
> > unnecessary bugs.  Let's not go there.
>
> OK. It does mean that:
>
>   git show HEAD:a*
>
> will still quietly produce no output instead of saying "hey, there is no
> a* in HEAD". But I think given the lack of bug reports over the years
> that this case (and the backslash one I'm fixing) are probably
> relatively rare.  The backslash one seems a lot more likely, just
> because Windows folks may treat it like a path separator (I'm not sure
> if that even works, considering its meaning in a glob, but certainly I
> can imagine somebody doing so as an experiment and getting confused by
> the result).

I'm normally in a Unix environment but needed to address a potential
issue in Windows, which indeed
got me confused about the results.

Upon experimenting in said Unix environment with what you outlined
above on making revision and pathspec explicit,
I figured that it becomes a little odd in the case where one would use
a glob meta character in a filename (why anybody would want
that except for academic reasons is beyond me), but it could still be
a source of bugs in tooling interacting with git:

Given a repository initialized with

$ git init && echo a > a && echo '*' > '*' && git add a '*' && git
commit -m "first" && git rm a '*' && git commit -m "second"

there's a bit of a discrepancy when using git show to dump the
contents per revision. In the beginning, everything is normal for
HEAD^ but for HEAD, it's a requirement to use disambiguation to avoid
believing that '*' was an empty file. At the same time, the
disambiguation causes the error message change its nature, for obvious
reasons:

$ git show 'HEAD^:a'
a
$ git show 'HEAD^:a' --
a
$ git show 'HEAD^:*' # presumably get_oid finds the object
*
$ git show 'HEAD^:*' --
*
$ git show 'HEAD:a'
fatal: Path 'a' does not exist in 'HEAD'
$ git show 'HEAD:a' --
fatal: bad revision 'HEAD:a'
$ git show 'HEAD:*'
$ git show 'HEAD:*' --
fatal: bad revision 'HEAD:*'

Indeed, it seems to be working as designed or at least explainably.
I'll make sure to add '--' in my tool integration to deal with this
corner case.

Re: Bug or unexpected behaviour in git show <rev>:a\b

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Subject: verify_filename(): handle backslashes in "wildcards are pathspecs" rule
>
> Commit 28fcc0b71a (pathspec: avoid the need of "--" when wildcard is
> used, 2015-05-02) allowed:
>
>   git rev-parse '*.c'
>
> without the double-dash. But the rule it uses to check for wildcards
> actually looks for any glob special. This is overly liberal, as it means
> that a pattern that doesn't actually do any wildcard matching, like
> "a\b", will be considered a pathspec.
>
> If you do have such a file on disk, that's presumably what you wanted.
> But if you don't, the results are confusing: rather than say "there's no
> such path a\b", we'll quietly accept it as a pathspec which very likely
> matches nothing (or at least not what you intended). Likewise, looking
> for path "a\*b" doesn't expand the search at all; it would only find a
> single entry, "a*b".
>
> This commit switches the rule to trigger only when glob metacharacters
> would expand the search, meaning both of those cases will now report an
> error (you can still disambiguate using "--", of course; we're just
> tightening the DWIM heuristic).

Makes sense.  Thanks.

Re: Bug or unexpected behaviour in git show <rev>:a\b

[Jeff King]

On Fri, Jan 24, 2020 at 02:01:19PM -0500, Jeff King wrote:

> Also, unrelated to your problem, but I find it interesting in the output
> above that "git show HEAD:foo --" produces a less useful error message
> than "git show HEAD:foo" without the separator, even though the user has
> given us even more information about their intent. I think the DWIM
> verify_filename() has grown a lot more diagnosis code over the years
> that could also be used in the other code path.

I took a look at this, but it gets ugly pretty quickly, so I gave up.
The problem is that all of the good errors are produced by get_oid()
with GET_OID_ONLY_TO_DIE, but the moment where we decide to produce the
bad error is sometimes removed from the get_oid() call. E.g., in
something like "a..b", we'd want to report on whichever of "a" or "b"
caused a problem, but at the point of reporting the error, we only have
the unparsed "a..b".

Certainly this can be overcome, but it seemed like a rabbit-hole that I
didn't feel like going down today. Anybody is welcome to give it a shot.
I suspect the whole thing would be simpler if get_oid() could optionally
return an error string to the caller, instead of forcing us to call it a
second time with the "only to die" flag.

Speaking of which, I did go part-way down a different rabbit-hole in
preparation, and ended up with these patches. So it seemed worth posting
them.

  [1/3]: t1400: avoid "test" string comparisons
  [2/3]: t1506: drop space after redirection operator
  [3/3]: sha1-name: mark get_oid() error messages for translation

 sha1-name.c                    | 30 +++++++++++------------
 t/t1400-update-ref.sh          | 36 ++++++++++++++++++----------
 t/t1506-rev-parse-diagnosis.sh | 44 +++++++++++++++++-----------------
 3 files changed, 61 insertions(+), 49 deletions(-)

-Peff

[PATCH 1/3] t1400: avoid "test" string comparisons

[Jeff King]

Using the shell "test" here is inflexible, because we can't easily swap
it out for an i18n-aware version like we can with test_cmp and
test_i18ncmp. And it's not even saving us any processes, since we have
to use "cat" to get the output. So let's switch to using test_cmp, which
has the added bonus that it will produce better output if there's a
failure.

Note that not all of the changed outputs here are candidates for
translation, but I've converted all of them for consistency and to
benefit from the better output.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t1400-update-ref.sh | 36 ++++++++++++++++++++++++------------
 1 file changed, 24 insertions(+), 12 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index b815cdd1b8..a86dd2fbd9 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -361,55 +361,67 @@ ld="Thu, 26 May 2005 18:43:00 -0500"
 test_expect_success 'Query "master@{May 25 2005}" (before history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{May 25 2005}" >o 2>e &&
-	test $C = $(cat o) &&
-	test "warning: Log for '\''master'\'' only goes back to $ed." = "$(cat e)"
+	echo "$C" >expect &&
+	test_cmp expect o &&
+	echo "warning: Log for '\''master'\'' only goes back to $ed." >expect &&
+	test_cmp expect e
 '
 test_expect_success 'Query master@{2005-05-25} (before history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify master@{2005-05-25} >o 2>e &&
-	test $C = $(cat o) &&
-	test "warning: Log for '\''master'\'' only goes back to $ed." = "$(cat e)"
+	echo "$C" >expect &&
+	test_cmp expect o &&
+	echo "warning: Log for '\''master'\'' only goes back to $ed." >expect &&
+	test_cmp expect e
 '
 test_expect_success 'Query "master@{May 26 2005 23:31:59}" (1 second before history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{May 26 2005 23:31:59}" >o 2>e &&
-	test $C = $(cat o) &&
-	test "warning: Log for '\''master'\'' only goes back to $ed." = "$(cat e)"
+	echo "$C" >expect &&
+	test_cmp expect o &&
+	echo "warning: Log for '\''master'\'' only goes back to $ed." >expect &&
+	test_cmp expect e
 '
 test_expect_success 'Query "master@{May 26 2005 23:32:00}" (exactly history start)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{May 26 2005 23:32:00}" >o 2>e &&
-	test $C = $(cat o) &&
+	echo "$C" >expect &&
+	test_cmp expect o &&
 	test_must_be_empty e
 '
 test_expect_success 'Query "master@{May 26 2005 23:32:30}" (first non-creation change)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{May 26 2005 23:32:30}" >o 2>e &&
-	test $A = $(cat o) &&
+	echo "$A" >expect &&
+	test_cmp expect o &&
 	test_must_be_empty e
 '
 test_expect_success 'Query "master@{2005-05-26 23:33:01}" (middle of history with gap)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{2005-05-26 23:33:01}" >o 2>e &&
-	test $B = $(cat o) &&
+	echo "$B" >expect &&
+	test_cmp expect o &&
 	test_i18ngrep -F "warning: log for ref $m has gap after $gd" e
 '
 test_expect_success 'Query "master@{2005-05-26 23:38:00}" (middle of history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{2005-05-26 23:38:00}" >o 2>e &&
-	test $Z = $(cat o) &&
+	echo "$Z" >expect &&
+	test_cmp expect o &&
 	test_must_be_empty e
 '
 test_expect_success 'Query "master@{2005-05-26 23:43:00}" (exact end of history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{2005-05-26 23:43:00}" >o 2>e &&
-	test $E = $(cat o) &&
+	echo "$E" >expect &&
+	test_cmp expect o &&
 	test_must_be_empty e
 '
 test_expect_success 'Query "master@{2005-05-28}" (past end of history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{2005-05-28}" >o 2>e &&
-	test $D = $(cat o) &&
+	echo "$D" >expect &&
+	test_cmp expect o &&
 	test_i18ngrep -F "warning: log for ref $m unexpectedly ended on $ld" e
 '
 
-- 
2.25.0.421.gb74d19af79

[PATCH 2/3] t1506: drop space after redirection operator

[Jeff King]

Some (but not all!) redirections in this file are spelled "2> error".
Let's switch them to our usual style.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t1506-rev-parse-diagnosis.sh | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
index 8a75f37a11..f49fc770d6 100755
--- a/t/t1506-rev-parse-diagnosis.sh
+++ b/t/t1506-rev-parse-diagnosis.sh
@@ -104,35 +104,35 @@ test_expect_success 'correct relative file objects (6)' '
 test_expect_success 'incorrect revision id' '
 	test_must_fail git rev-parse foobar:file.txt 2>error &&
 	grep "Invalid object name '"'"'foobar'"'"'." error &&
-	test_must_fail git rev-parse foobar 2> error &&
+	test_must_fail git rev-parse foobar 2>error &&
 	test_i18ngrep "unknown revision or path not in the working tree." error
 '
 
 test_expect_success 'incorrect file in sha1:path' '
-	test_must_fail git rev-parse HEAD:nothing.txt 2> error &&
+	test_must_fail git rev-parse HEAD:nothing.txt 2>error &&
 	grep "fatal: Path '"'"'nothing.txt'"'"' does not exist in '"'"'HEAD'"'"'" error &&
-	test_must_fail git rev-parse HEAD:index-only.txt 2> error &&
+	test_must_fail git rev-parse HEAD:index-only.txt 2>error &&
 	grep "fatal: Path '"'"'index-only.txt'"'"' exists on disk, but not in '"'"'HEAD'"'"'." error &&
 	(cd subdir &&
-	 test_must_fail git rev-parse HEAD:file2.txt 2> error &&
+	 test_must_fail git rev-parse HEAD:file2.txt 2>error &&
 	 test_did_you_mean HEAD subdir/ file2.txt exists )
 '
 
 test_expect_success 'incorrect file in :path and :N:path' '
-	test_must_fail git rev-parse :nothing.txt 2> error &&
+	test_must_fail git rev-parse :nothing.txt 2>error &&
 	grep "fatal: Path '"'"'nothing.txt'"'"' does not exist (neither on disk nor in the index)." error &&
-	test_must_fail git rev-parse :1:nothing.txt 2> error &&
+	test_must_fail git rev-parse :1:nothing.txt 2>error &&
 	grep "Path '"'"'nothing.txt'"'"' does not exist (neither on disk nor in the index)." error &&
-	test_must_fail git rev-parse :1:file.txt 2> error &&
+	test_must_fail git rev-parse :1:file.txt 2>error &&
 	test_did_you_mean ":0" "" file.txt "is in the index" "at stage 1" &&
 	(cd subdir &&
-	 test_must_fail git rev-parse :1:file.txt 2> error &&
+	 test_must_fail git rev-parse :1:file.txt 2>error &&
 	 test_did_you_mean ":0" "" file.txt "is in the index" "at stage 1" &&
-	 test_must_fail git rev-parse :file2.txt 2> error &&
+	 test_must_fail git rev-parse :file2.txt 2>error &&
 	 test_did_you_mean ":0" subdir/ file2.txt "is in the index" &&
-	 test_must_fail git rev-parse :2:file2.txt 2> error &&
+	 test_must_fail git rev-parse :2:file2.txt 2>error &&
 	 test_did_you_mean :0 subdir/ file2.txt "is in the index") &&
-	test_must_fail git rev-parse :disk-only.txt 2> error &&
+	test_must_fail git rev-parse :disk-only.txt 2>error &&
 	grep "fatal: Path '"'"'disk-only.txt'"'"' exists on disk, but not in the index." error
 '
 
-- 
2.25.0.421.gb74d19af79

[PATCH 3/3] sha1-name: mark get_oid() error messages for translation

[Jeff King]

There are several error messages in get_oid() and its children that are
clearly intended for humans, but aren't marked for translation. E.g.:

  $ git show :1:foo
  fatal: Path 'foo' is in the index, but not at stage 1.
  Did you mean ':0:foo'?

Let's mark these for translation. While we're at it, let's switch the
style to be more like our usual error messages: start with a lowercase
letter and omit a period at the end of the line.

This does mean that multi-line messages like the one above don't have
any punctuation between the two sentences. I solved that by adding a
"hint" marker like we'd see from advise(). So the result is:

  $ git show :1:foo
  fatal: path 'foo' is in the index, but not at stage 1
  hint: Did you mean ':0:foo'?

A few tests had to be switched to test_i18ngrep and test_i18ncmp. Since
we were touching them anyway, I also simplified the ones using i18ngrep
a bit for readability.

Signed-off-by: Jeff King <peff@peff.net>
---
I'm on the fence about whether the resulting message is clear enough.
Certainly we could just mark them for translation but leave the
capitalization and punctuation as-is.

I actually think _both_ of those lines could be considered hints. And
something like:

  hint: path 'foo' is in the index, but not at stage 1;
  hint: did you mean ':0:foo'?
  fatal: unable to resolve object name ':1:foo'

might be better still. But I suspect the text in that final "fatal"
could be better if done by the caller of get_oid(). So that takes me
back to my earlier suggestion that get_oid() should actually be
collecting this hints as it runs and making them available to the
caller. I punted on that for now, though.

 sha1-name.c                    | 30 +++++++++++++++---------------
 t/t1400-update-ref.sh          | 12 ++++++------
 t/t1506-rev-parse-diagnosis.sh | 22 +++++++++++-----------
 3 files changed, 32 insertions(+), 32 deletions(-)

diff --git a/sha1-name.c b/sha1-name.c
index 200eb373ad..75235cb490 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -919,15 +919,15 @@ static int get_oid_basic(struct repository *r, const char *str, int len,
 			}
 			if (at_time) {
 				if (!(flags & GET_OID_QUIETLY)) {
-					warning("Log for '%.*s' only goes "
-						"back to %s.", len, str,
+					warning(_("log for '%.*s' only goes back to %s"),
+						len, str,
 						show_date(co_time, co_tz, DATE_MODE(RFC2822)));
 				}
 			} else {
 				if (flags & GET_OID_QUIETLY) {
 					exit(128);
 				}
-				die("Log for '%.*s' only has %d entries.",
+				die(_("log for '%.*s' only has %d entries"),
 				    len, str, co_cnt);
 			}
 		}
@@ -1692,22 +1692,22 @@ static void diagnose_invalid_oid_path(struct repository *r,
 		prefix = "";
 
 	if (file_exists(filename))
-		die("Path '%s' exists on disk, but not in '%.*s'.",
+		die(_("path '%s' exists on disk, but not in '%.*s'"),
 		    filename, object_name_len, object_name);
 	if (is_missing_file_error(errno)) {
 		char *fullname = xstrfmt("%s%s", prefix, filename);
 
 		if (!get_tree_entry(r, tree_oid, fullname, &oid, &mode)) {
-			die("Path '%s' exists, but not '%s'.\n"
-			    "Did you mean '%.*s:%s' aka '%.*s:./%s'?",
+			die(_("path '%s' exists, but not '%s'\n"
+			    "hint: Did you mean '%.*s:%s' aka '%.*s:./%s'?"),
 			    fullname,
 			    filename,
 			    object_name_len, object_name,
 			    fullname,
 			    object_name_len, object_name,
 			    filename);
 		}
-		die("Path '%s' does not exist in '%.*s'",
+		die(_("path '%s' does not exist in '%.*s'"),
 		    filename, object_name_len, object_name);
 	}
 }
@@ -1735,8 +1735,8 @@ static void diagnose_invalid_index_path(struct repository *r,
 		ce = istate->cache[pos];
 		if (ce_namelen(ce) == namelen &&
 		    !memcmp(ce->name, filename, namelen))
-			die("Path '%s' is in the index, but not at stage %d.\n"
-			    "Did you mean ':%d:%s'?",
+			die(_("path '%s' is in the index, but not at stage %d\n"
+			    "hint: Did you mean ':%d:%s'?"),
 			    filename, stage,
 			    ce_stage(ce), filename);
 	}
@@ -1751,17 +1751,17 @@ static void diagnose_invalid_index_path(struct repository *r,
 		ce = istate->cache[pos];
 		if (ce_namelen(ce) == fullname.len &&
 		    !memcmp(ce->name, fullname.buf, fullname.len))
-			die("Path '%s' is in the index, but not '%s'.\n"
-			    "Did you mean ':%d:%s' aka ':%d:./%s'?",
+			die(_("path '%s' is in the index, but not '%s'\n"
+			    "hint: Did you mean ':%d:%s' aka ':%d:./%s'?"),
 			    fullname.buf, filename,
 			    ce_stage(ce), fullname.buf,
 			    ce_stage(ce), filename);
 	}
 
 	if (repo_file_exists(r, filename))
-		die("Path '%s' exists on disk, but not in the index.", filename);
+		die(_("path '%s' exists on disk, but not in the index"), filename);
 	if (is_missing_file_error(errno))
-		die("Path '%s' does not exist (neither on disk nor in the index).",
+		die(_("path '%s' does not exist (neither on disk nor in the index)"),
 		    filename);
 
 	strbuf_release(&fullname);
@@ -1774,7 +1774,7 @@ static char *resolve_relative_path(struct repository *r, const char *rel)
 		return NULL;
 
 	if (r != the_repository || !is_inside_work_tree())
-		die("relative path syntax can't be used outside working tree.");
+		die(_("relative path syntax can't be used outside working tree"));
 
 	/* die() inside prefix_path() if resolved path is outside worktree */
 	return prefix_path(startup_info->prefix,
@@ -1912,7 +1912,7 @@ static enum get_oid_result get_oid_with_context_1(struct repository *repo,
 			return ret;
 		} else {
 			if (only_to_die)
-				die("Invalid object name '%.*s'.", len, name);
+				die(_("invalid object name '%.*s'."), len, name);
 		}
 	}
 	return ret;
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index a86dd2fbd9..a6224ef65f 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -363,24 +363,24 @@ test_expect_success 'Query "master@{May 25 2005}" (before history)' '
 	git rev-parse --verify "master@{May 25 2005}" >o 2>e &&
 	echo "$C" >expect &&
 	test_cmp expect o &&
-	echo "warning: Log for '\''master'\'' only goes back to $ed." >expect &&
-	test_cmp expect e
+	echo "warning: log for '\''master'\'' only goes back to $ed" >expect &&
+	test_i18ncmp expect e
 '
 test_expect_success 'Query master@{2005-05-25} (before history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify master@{2005-05-25} >o 2>e &&
 	echo "$C" >expect &&
 	test_cmp expect o &&
-	echo "warning: Log for '\''master'\'' only goes back to $ed." >expect &&
-	test_cmp expect e
+	echo "warning: log for '\''master'\'' only goes back to $ed" >expect &&
+	test_i18ncmp expect e
 '
 test_expect_success 'Query "master@{May 26 2005 23:31:59}" (1 second before history)' '
 	test_when_finished "rm -f o e" &&
 	git rev-parse --verify "master@{May 26 2005 23:31:59}" >o 2>e &&
 	echo "$C" >expect &&
 	test_cmp expect o &&
-	echo "warning: Log for '\''master'\'' only goes back to $ed." >expect &&
-	test_cmp expect e
+	echo "warning: log for '\''master'\'' only goes back to $ed" >expect &&
+	test_i18ncmp expect e
 '
 test_expect_success 'Query "master@{May 26 2005 23:32:00}" (exactly history start)' '
 	test_when_finished "rm -f o e" &&
diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
index f49fc770d6..c2b5125c12 100755
--- a/t/t1506-rev-parse-diagnosis.sh
+++ b/t/t1506-rev-parse-diagnosis.sh
@@ -9,8 +9,8 @@ exec </dev/null
 test_did_you_mean ()
 {
 	cat >expected <<-EOF &&
-	fatal: Path '$2$3' $4, but not ${5:-$SQ$3$SQ}.
-	Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
+	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
+	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
 	EOF
 	test_cmp expected error
 }
@@ -103,26 +103,26 @@ test_expect_success 'correct relative file objects (6)' '
 
 test_expect_success 'incorrect revision id' '
 	test_must_fail git rev-parse foobar:file.txt 2>error &&
-	grep "Invalid object name '"'"'foobar'"'"'." error &&
+	test_i18ngrep "invalid object name .foobar." error &&
 	test_must_fail git rev-parse foobar 2>error &&
 	test_i18ngrep "unknown revision or path not in the working tree." error
 '
 
 test_expect_success 'incorrect file in sha1:path' '
 	test_must_fail git rev-parse HEAD:nothing.txt 2>error &&
-	grep "fatal: Path '"'"'nothing.txt'"'"' does not exist in '"'"'HEAD'"'"'" error &&
+	test_i18ngrep "path .nothing.txt. does not exist in .HEAD." error &&
 	test_must_fail git rev-parse HEAD:index-only.txt 2>error &&
-	grep "fatal: Path '"'"'index-only.txt'"'"' exists on disk, but not in '"'"'HEAD'"'"'." error &&
+	test_i18ngrep "path .index-only.txt. exists on disk, but not in .HEAD." error &&
 	(cd subdir &&
 	 test_must_fail git rev-parse HEAD:file2.txt 2>error &&
 	 test_did_you_mean HEAD subdir/ file2.txt exists )
 '
 
 test_expect_success 'incorrect file in :path and :N:path' '
 	test_must_fail git rev-parse :nothing.txt 2>error &&
-	grep "fatal: Path '"'"'nothing.txt'"'"' does not exist (neither on disk nor in the index)." error &&
+	test_i18ngrep "path .nothing.txt. does not exist (neither on disk nor in the index)" error &&
 	test_must_fail git rev-parse :1:nothing.txt 2>error &&
-	grep "Path '"'"'nothing.txt'"'"' does not exist (neither on disk nor in the index)." error &&
+	test_i18ngrep "path .nothing.txt. does not exist (neither on disk nor in the index)" error &&
 	test_must_fail git rev-parse :1:file.txt 2>error &&
 	test_did_you_mean ":0" "" file.txt "is in the index" "at stage 1" &&
 	(cd subdir &&
@@ -133,16 +133,16 @@ test_expect_success 'incorrect file in :path and :N:path' '
 	 test_must_fail git rev-parse :2:file2.txt 2>error &&
 	 test_did_you_mean :0 subdir/ file2.txt "is in the index") &&
 	test_must_fail git rev-parse :disk-only.txt 2>error &&
-	grep "fatal: Path '"'"'disk-only.txt'"'"' exists on disk, but not in the index." error
+	test_i18ngrep "path .disk-only.txt. exists on disk, but not in the index" error
 '
 
 test_expect_success 'invalid @{n} reference' '
 	test_must_fail git rev-parse master@{99999} >output 2>error &&
 	test_must_be_empty output &&
-	grep "fatal: Log for [^ ]* only has [0-9][0-9]* entries." error  &&
+	test_i18ngrep "log for [^ ]* only has [0-9][0-9]* entries" error  &&
 	test_must_fail git rev-parse --verify master@{99999} >output 2>error &&
 	test_must_be_empty output &&
-	grep "fatal: Log for [^ ]* only has [0-9][0-9]* entries." error
+	test_i18ngrep "log for [^ ]* only has [0-9][0-9]* entries" error
 '
 
 test_expect_success 'relative path not found' '
@@ -162,7 +162,7 @@ test_expect_success 'relative path outside worktree' '
 test_expect_success 'relative path when cwd is outside worktree' '
 	test_must_fail git --git-dir=.git --work-tree=subdir rev-parse HEAD:./file.txt >output 2>error &&
 	test_must_be_empty output &&
-	grep "relative path syntax can.t be used outside working tree." error
+	test_i18ngrep "relative path syntax can.t be used outside working tree" error
 '
 
 test_expect_success '<commit>:file correctly diagnosed after a pathname' '
-- 
2.25.0.421.gb74d19af79

Re: [PATCH 3/3] sha1-name: mark get_oid() error messages for translation

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> +			die(_("path '%s' exists, but not '%s'\n"
> +			    "hint: Did you mean '%.*s:%s' aka '%.*s:./%s'?"),
> ...

The above is meant to be localizable, but...

> diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
> index f49fc770d6..c2b5125c12 100755
> --- a/t/t1506-rev-parse-diagnosis.sh
> +++ b/t/t1506-rev-parse-diagnosis.sh
> @@ -9,8 +9,8 @@ exec </dev/null
>  test_did_you_mean ()
>  {
>  	cat >expected <<-EOF &&
> -	fatal: Path '$2$3' $4, but not ${5:-$SQ$3$SQ}.
> -	Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
> +	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
> +	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
>  	EOF
>  	test_cmp expected error

...this obviously would not allow GIT_TEST_GETTEXT_POISON test to
pass.  And ...

>  test_expect_success 'incorrect file in sha1:path' '
>  	test_must_fail git rev-parse HEAD:nothing.txt 2>error &&
> -	grep "fatal: Path '"'"'nothing.txt'"'"' does not exist in '"'"'HEAD'"'"'" error &&
> +	test_i18ngrep "path .nothing.txt. does not exist in .HEAD." error &&
>  	test_must_fail git rev-parse HEAD:index-only.txt 2>error &&
> -	grep "fatal: Path '"'"'index-only.txt'"'"' exists on disk, but not in '"'"'HEAD'"'"'." error &&
> +	test_i18ngrep "path .index-only.txt. exists on disk, but not in .HEAD." error &&
>  	(cd subdir &&
>  	 test_must_fail git rev-parse HEAD:file2.txt 2>error &&
>  	 test_did_you_mean HEAD subdir/ file2.txt exists )
>  '

... the last step executed in "subdir" does not seem to pass.

Re: [PATCH 3/3] sha1-name: mark get_oid() error messages for translation

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jeff King <peff@peff.net> writes:
>
>> +			die(_("path '%s' exists, but not '%s'\n"
>> +			    "hint: Did you mean '%.*s:%s' aka '%.*s:./%s'?"),
>> ...
>
> The above is meant to be localizable, but...
>
>> diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
>> index f49fc770d6..c2b5125c12 100755
>> --- a/t/t1506-rev-parse-diagnosis.sh
>> +++ b/t/t1506-rev-parse-diagnosis.sh
>> @@ -9,8 +9,8 @@ exec </dev/null
>>  test_did_you_mean ()
>>  {
>>  	cat >expected <<-EOF &&
>> -	fatal: Path '$2$3' $4, but not ${5:-$SQ$3$SQ}.
>> -	Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
>> +	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
>> +	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
>>  	EOF
>>  	test_cmp expected error
>
> ...this obviously would not allow GIT_TEST_GETTEXT_POISON test to
> pass.  And ...
>
>>  test_expect_success 'incorrect file in sha1:path' '
>>  	test_must_fail git rev-parse HEAD:nothing.txt 2>error &&
>> -	grep "fatal: Path '"'"'nothing.txt'"'"' does not exist in '"'"'HEAD'"'"'" error &&
>> +	test_i18ngrep "path .nothing.txt. does not exist in .HEAD." error &&
>>  	test_must_fail git rev-parse HEAD:index-only.txt 2>error &&
>> -	grep "fatal: Path '"'"'index-only.txt'"'"' exists on disk, but not in '"'"'HEAD'"'"'." error &&
>> +	test_i18ngrep "path .index-only.txt. exists on disk, but not in .HEAD." error &&
>>  	(cd subdir &&
>>  	 test_must_fail git rev-parse HEAD:file2.txt 2>error &&
>>  	 test_did_you_mean HEAD subdir/ file2.txt exists )
>>  '
>
> ... the last step executed in "subdir" does not seem to pass.

I'll queue this band-aid on top before making my last pushout for
the day.  Even with poisoned i18n/l10n, die(_(msg)) gives "fatal:"
prefix at the beginning, so that is what test_did_you_mean would
expect to see from a passing test under GIT_TEST_GETTEXT_POISON.

The other hunk is about a test that greps in "error".

 t/t1506-rev-parse-diagnosis.sh | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
index 2d26d2267e..f28a35c84e 100755
--- a/t/t1506-rev-parse-diagnosis.sh
+++ b/t/t1506-rev-parse-diagnosis.sh
@@ -8,11 +8,16 @@ exec </dev/null
 
 test_did_you_mean ()
 {
-	cat >expected <<-EOF &&
-	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
-	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
-	EOF
-	test_cmp expected error
+	if ! test_have_prereq C_LOCALE_OUTPUT
+	then
+		grep "^fatal: " error
+	else
+		cat >expected <<-EOF &&
+		fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
+		hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
+		EOF
+		test_cmp expected error
+	fi
 }
 
 HASH_file=
@@ -149,7 +154,7 @@ test_expect_success 'relative path not found' '
 	(
 		cd subdir &&
 		test_must_fail git rev-parse HEAD:./nonexistent.txt 2>error &&
-		grep subdir/nonexistent.txt error
+		test_i18ngrep subdir/nonexistent.txt error
 	)
 '
 

Re: [PATCH 3/3] sha1-name: mark get_oid() error messages for translation

[Jeff King]

On Wed, Jan 29, 2020 at 01:42:50PM -0800, Junio C Hamano wrote:

> >> -	fatal: Path '$2$3' $4, but not ${5:-$SQ$3$SQ}.
> >> -	Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
> >> +	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
> >> +	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
> >>  	EOF
> >>  	test_cmp expected error
> >
> > ...this obviously would not allow GIT_TEST_GETTEXT_POISON test to
> > pass.  And ...

Hrmph. I know I tested with GETTEXT_POISON, but you are obviously right
that this doesn't pass. I must have botched something in one of my
rebase passes at the end.

Thanks for catching it, but...

> I'll queue this band-aid on top before making my last pushout for
> the day.  Even with poisoned i18n/l10n, die(_(msg)) gives "fatal:"
> prefix at the beginning, so that is what test_did_you_mean would
> expect to see from a passing test under GIT_TEST_GETTEXT_POISON.
> 
> The other hunk is about a test that greps in "error".

I think we can do this much more simply, by just using i18ncmp:

diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
index c2b5125c12..62085a89e3 100755
--- a/t/t1506-rev-parse-diagnosis.sh
+++ b/t/t1506-rev-parse-diagnosis.sh
@@ -12,7 +12,7 @@ test_did_you_mean ()
 	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
 	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
 	EOF
-	test_cmp expected error
+	test_i18ncmp expected error
 }
 
 HASH_file=

(we'd still need the s/grep/test_i18ngrep/ in your second hunk).

-Peff

Re: [PATCH 3/3] sha1-name: mark get_oid() error messages for translation

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I think we can do this much more simply, by just using i18ncmp:
>
> diff --git a/t/t1506-rev-parse-diagnosis.sh b/t/t1506-rev-parse-diagnosis.sh
> index c2b5125c12..62085a89e3 100755
> --- a/t/t1506-rev-parse-diagnosis.sh
> +++ b/t/t1506-rev-parse-diagnosis.sh
> @@ -12,7 +12,7 @@ test_did_you_mean ()
>  	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
>  	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
>  	EOF
> -	test_cmp expected error
> +	test_i18ncmp expected error
>  }

Yeah, "what die(_(...)) under POISON says matches /^fatal:/"
probably is not worth checking and more importantly not something we
would want to rely on.

>  HASH_file=
>
> (we'd still need the s/grep/test_i18ngrep/ in your second hunk).

Yes.  The final amend to 3/3 has become like this.

$ git range-diff bc3f657f71..83252ba6f1 HEAD^..HEAD
1:  83252ba6f1 ! 1:  b0418303b1 sha1-name: mark get_oid() error messages for translation
    @@ t/t1506-rev-parse-diagnosis.sh: exec </dev/null
     +	fatal: path '$2$3' $4, but not ${5:-$SQ$3$SQ}
     +	hint: Did you mean '$1:$2$3'${2:+ aka $SQ$1:./$3$SQ}?
      	EOF
    - 	test_cmp expected error
    +-	test_cmp expected error
    ++	test_i18ncmp expected error
      }
    + 
    + HASH_file=
     @@ t/t1506-rev-parse-diagnosis.sh: test_expect_success 'correct relative file objects (6)' '
      
      test_expect_success 'incorrect revision id' '
    @@ t/t1506-rev-parse-diagnosis.sh: test_expect_success 'incorrect file in :path and
      '
      
      test_expect_success 'relative path not found' '
    + 	(
    + 		cd subdir &&
    + 		test_must_fail git rev-parse HEAD:./nonexistent.txt 2>error &&
    +-		grep subdir/nonexistent.txt error
    ++		test_i18ngrep subdir/nonexistent.txt error
    + 	)
    + '
    + 
     @@ t/t1506-rev-parse-diagnosis.sh: test_expect_success 'relative path outside worktree' '
      test_expect_success 'relative path when cwd is outside worktree' '
      	test_must_fail git --git-dir=.git --work-tree=subdir rev-parse HEAD:./file.txt >output 2>error &&

Re: [PATCH 3/3] sha1-name: mark get_oid() error messages for translation

[Jeff King]

On Thu, Jan 30, 2020 at 11:16:13AM -0800, Junio C Hamano wrote:

> Yes.  The final amend to 3/3 has become like this.
> [...]

Perfect, thanks for fixing my mistake.

-Peff