Expected behavior of "git check-ignore"...

Expected behavior of "git check-ignore"...

[John Szakmeister]

A StackOverflow user posted a question about how to reliably check
whether a file would be ignored by "git add" and expected "git
check-ignore" to return results that matched git add's behavior.  It
turns out that it doesn't.  If there is a negation rule, we end up
returning that exclude and printing it and exiting with 0 (there are
some ignored files) even though the file has been marked to not be
ignored.

Is the expected behavior of "git check-ignore" to return 0 even if the
file is not ignore when a negation is present?

>>>>
git init .
echo 'foo/*' > .gitignore
echo '!foo/bar' > .gitignore
mkdir foo
touch foo/bar
git check-ignore foo/bar
<<<<

I expect the last command to return 1 (no files are ignored), but it
doesn't.  The StackOverflow user had the same expectation, and imagine
others do as well.  OTOH, it looks like the command is really meant to
be a debugging tool--to show me the line in a .gitignore associated
with this file, if there is one.  In which case, the behavior is
correct but the return code description is a bit misleading (0 means
the file is ignored, which isn't true here).

Thoughts?  It seems like this question was asked before several years
ago but didn't get a response.

Thanks!

-John

PS The SO question is here:
https://stackoverflow.com/questions/45210790/how-to-reliably-check-whether-a-file-is-ignored-by-git

Re: Expected behavior of "git check-ignore"...

[Philip Oakley]

From: "John Szakmeister" <john@szakmeister.net>
Sent: Thursday, July 20, 2017 11:37 AM
>A StackOverflow user posted a question about how to reliably check
> whether a file would be ignored by "git add" and expected "git
> check-ignore" to return results that matched git add's behavior.  It
> turns out that it doesn't.  If there is a negation rule, we end up
> returning that exclude and printing it and exiting with 0 (there are
> some ignored files) even though the file has been marked to not be
> ignored.
>
> Is the expected behavior of "git check-ignore" to return 0 even if the
> file is not ignore when a negation is present?

I'm testing this on..
$ git --version

git version 2.10.0.windows.1


>
>>>>>
> git init .
> echo 'foo/*' > .gitignore
> echo '!foo/bar' > .gitignore

Is this missing the >> append to get the full two line .gitignore?
adding in a `cat .gitignore` would help check.


> mkdir foo
> touch foo/bar
I don't think you need these. It's the given pathnames that are checked, not 
the file system content.

> git check-ignore foo/bar

Does this need the `-q` option to set the exit status?

echo $? # to display the status.


> <<<<
>
> I expect the last command to return 1 (no files are ignored), but it
> doesn't.  The StackOverflow user had the same expectation, and imagine
> others do as well.  OTOH, it looks like the command is really meant to
> be a debugging tool--to show me the line in a .gitignore associated
> with this file, if there is one.  In which case, the behavior is
> correct but the return code description is a bit misleading (0 means
> the file is ignored, which isn't true here).

Maybe the logic isn't that clear? Maybe it is simply detecting if any one of 
the ignore lines is active, and doesn't reset the status for a negation?

I appear to get the same response as yourself, but I haven't spent much time 
on it - I'm clearing a backlog of work at the moment.

I also tried the -v -n options, and if I swap the ignore lines around it 
still says line 2 is the one that ignores.
It gets more interesting if two paths are given `foo/bar foo/baz`, to see 
which line picks up which pathname (and with the swapped ignore lines).

Is there a test for this in the test suite?

>
> Thoughts?  It seems like this question was asked before several years
> ago but didn't get a response.
>
> Thanks!
>
> -John
>
> PS The SO question is here:
> https://stackoverflow.com/questions/45210790/how-to-reliably-check-whether-a-file-is-ignored-by-git

--
Philip 

Re: Expected behavior of "git check-ignore"...

[John Szakmeister]

On Sun, Jul 23, 2017 at 12:33 PM, Philip Oakley <philipoakley@iee.org> wrote:
[snip]
>>
>>>>>>
>> git init .
>> echo 'foo/*' > .gitignore
>> echo '!foo/bar' > .gitignore
>
>
> Is this missing the >> append to get the full two line .gitignore?
> adding in a `cat .gitignore` would help check.

Yes, sorry about that.

>
>> mkdir foo
>> touch foo/bar
>
> I don't think you need these. It's the given pathnames that are checked, not
> the file system content.

It was there so you could see that `git status` ignores foo/bar
(though that wasn't part of the little script).

>> git check-ignore foo/bar
>
> Does this need the `-q` option to set the exit status?

No, it's always set.

> echo $? # to display the status.

Sure.  So, to recap the update reproduction recipe would be:

>>>>
git init .
echo 'foo/*' > .gitignore
echo '!foo/bar' >> .gitignore
mkdir foo
touch foo/bar
git status # foo/ shows as untracked because bar is present
git check-ignore foo/bar
echo $? # show the exit status
<<<<

It seems like it should print "1", but it prints "0".

>> I expect the last command to return 1 (no files are ignored), but it
>> doesn't.  The StackOverflow user had the same expectation, and imagine
>> others do as well.  OTOH, it looks like the command is really meant to
>> be a debugging tool--to show me the line in a .gitignore associated
>> with this file, if there is one.  In which case, the behavior is
>> correct but the return code description is a bit misleading (0 means
>> the file is ignored, which isn't true here).
>
>
> Maybe the logic isn't that clear? Maybe it is simply detecting if any one of
> the ignore lines is active, and doesn't reset the status for a negation?
>
> I appear to get the same response as yourself, but I haven't spent much time
> on it - I'm clearing a backlog of work at the moment.

Correct, it appears that if any line in the ignore matches, then it
exits with 0.  So it's not that it's ignored, but that there is a
matching line in an ignore file somewhere.  I can see the logic in
this if it's meant to be a debugging tools, especially combined with
-v.  Simply changing it does affect quite a few tests, but I'm not
sure that it was intentional for negation to be treated this way.

> I also tried the -v -n options, and if I swap the ignore lines around it
> still says line 2 is the one that ignores.
> It gets more interesting if two paths are given `foo/bar foo/baz`, to see
> which line picks up which pathname (and with the swapped ignore lines).
>
> Is there a test for this in the test suite?

There are several.  But line 427, test_expect_success_multi 'nested
include', is one that I think is pretty direct about testing this.  I
imagine what happened is that gitignores used to contain only things
you wanted to ignore and when the ability to negate came along the
semantics of this was never changed--and possibly for good reason.
I'm just wondering if it should change, or if the documentation should
be updated to reflect how it actually behaves (the file may not be
ignored, but a line is present in a gitignore that affects its
status).  The behavior is definitely a little unexpected as it stands,
given the documentation though.

Thanks for taking a look Philip!

-John

Re: Expected behavior of "git check-ignore"...

[Junio C Hamano]

John Szakmeister <john@szakmeister.net> writes:

> Correct, it appears that if any line in the ignore matches, then it
> exits with 0.  So it's not that it's ignored, but that there is a
> matching line in an ignore file somewhere.  I can see the logic in
> this if it's meant to be a debugging tools, especially combined with
> -v.  Simply changing it does affect quite a few tests, but I'm not
> sure that it was intentional for negation to be treated this way.

I am reasonably sure that the command started its life as a pure
debugging aid.  

The treatment of the negation _might_ impose conflicting goals to
its purpose as a debugging aid---a user who debugs his .gitignore
file would want to know what causes a thing that wants to be ignored
is not or vice versa, and use of the exit status to indicate if it
is ignored may not mesh well with its goal as a debugging aid, but I
didn't think about the potential issues deeply myself while writing
this response.  As you mentioned, use of (or not using) "-v" could
be used as a sign to see which behaviour the end-user expects, I
guess.

Re: Expected behavior of "git check-ignore"...

[John Szakmeister]

On Mon, Jul 24, 2017 at 3:23 PM, Junio C Hamano <gitster@pobox.com> wrote:
[snip]
> I am reasonably sure that the command started its life as a pure
> debugging aid.
>
> The treatment of the negation _might_ impose conflicting goals to
> its purpose as a debugging aid---a user who debugs his .gitignore
> file would want to know what causes a thing that wants to be ignored
> is not or vice versa, and use of the exit status to indicate if it
> is ignored may not mesh well with its goal as a debugging aid, but I
> didn't think about the potential issues deeply myself while writing
> this response.  As you mentioned, use of (or not using) "-v" could
> be used as a sign to see which behaviour the end-user expects, I
> guess.

Is there another way of checking to see if a file is ignored?  If so,
maybe we could suggest that instead.  Perhaps using `git status
--porcelain --ignored` and examining the output?  I'm not sure how
well that would work with directories.

Thanks for the insight Junio.  I'm going to let the exit status thing
drop for now.  You don't seem like it's a good thing to do, and I'm
not particularly fond of having it behave two different ways based on
`-v` being present.

-John

Re: Expected behavior of "git check-ignore"...

[Junio C Hamano]

John Szakmeister <john@szakmeister.net> writes:

> On Mon, Jul 24, 2017 at 3:23 PM, Junio C Hamano <gitster@pobox.com> wrote:
> [snip]
>> I am reasonably sure that the command started its life as a pure
>> debugging aid.
>>
>> The treatment of the negation _might_ impose conflicting goals to
>> its purpose as a debugging aid---a user who debugs his .gitignore
>> file would want to know what causes a thing that wants to be ignored
>> is not or vice versa, and use of the exit status to indicate if it
>> is ignored may not mesh well with its goal as a debugging aid, but I
>> didn't think about the potential issues deeply myself while writing
>> this response.  As you mentioned, use of (or not using) "-v" could
>> be used as a sign to see which behaviour the end-user expects, I
>> guess.
>
> Is there another way of checking to see if a file is ignored?  If so,...

Maybe I sounded like waffling, but I do think "check-ignore" when
used as an end-user tool should be that command, to get a preview of
what would happen if you gave the path to "git add".  

I was merely giving a possible explanation why it may not behave
like so in the current code, i.e. those who used it for debugging
their .gitignore files may have felt that the current way to handle
negation were more convenient during their debugging session.

But I think there is a way out to satisfy both groups of people.

What if we (re)define that "-v" is a way to ask "which entry, if
any, decides the final fate of this path?" question, and that is a
sign that the user is using it to debug their .gitignore?  And we
use the exit status to mean "Yeah, there is an explicit entry that
decides the fate of the path" in that case, which is what the
current behaviour seems to be---the command exits with non-zero
status only when there is nothing that matches in the exclude
mechanism (which makes the final fate of the path to be 'not
ignored').

And we interpret the lack of "-v" as a signal that the user wants to
learn the fate of a given path via the exit status of the command,
which will "fix" the exit code to match the expectation in your
initial message in this thread.

Would that work well?

Re: Expected behavior of "git check-ignore"...

[Philip Oakley]

From: "Junio C Hamano" <gitster@pobox.com>
Sent: Thursday, July 27, 2017 6:05 PM
> John Szakmeister <john@szakmeister.net> writes:
>
>> On Mon, Jul 24, 2017 at 3:23 PM, Junio C Hamano <gitster@pobox.com> 
>> wrote:
>> [snip]
>>> I am reasonably sure that the command started its life as a pure
>>> debugging aid.
>>>
>>> The treatment of the negation _might_ impose conflicting goals to
>>> its purpose as a debugging aid---a user who debugs his .gitignore
>>> file would want to know what causes a thing that wants to be ignored
>>> is not or vice versa, and use of the exit status to indicate if it
>>> is ignored may not mesh well with its goal as a debugging aid, but I
>>> didn't think about the potential issues deeply myself while writing
>>> this response.  As you mentioned, use of (or not using) "-v" could
>>> be used as a sign to see which behaviour the end-user expects, I
>>> guess.
>>
>> Is there another way of checking to see if a file is ignored?  If so,...
>
> Maybe I sounded like waffling, but I do think "check-ignore" when
> used as an end-user tool should be that command, to get a preview of
> what would happen if you gave the path to "git add".
>
> I was merely giving a possible explanation why it may not behave
> like so in the current code, i.e. those who used it for debugging
> their .gitignore files may have felt that the current way to handle
> negation were more convenient during their debugging session.
>
> But I think there is a way out to satisfy both groups of people.
>
> What if we (re)define that "-v" is a way to ask "which entry, if
> any, decides the final fate of this path?" question, and that is a
> sign that the user is using it to debug their .gitignore?  And we
> use the exit status to mean "Yeah, there is an explicit entry that
> decides the fate of the path" in that case, which is what the
> current behaviour seems to be---the command exits with non-zero
> status only when there is nothing that matches in the exclude
> mechanism (which makes the final fate of the path to be 'not
> ignored').
>
> And we interpret the lack of "-v" as a signal that the user wants to
> learn the fate of a given path via the exit status of the command,
> which will "fix" the exit code to match the expectation in your
> initial message in this thread.
>
> Would that work well?

The old answer on StackOverflow 
https://stackoverflow.com/questions/12144633/which-gitignore-rule-is-ignoring-my-file 
has some of the older gory detail.

I also see that one of the changes was reverted (Revert "Merge branch 
'nd/exclusion-regression-fix', 5cee3493, 18 Mar 2016 
https://github.com/git/git/commit/5cee349370bd2dce48d0d653ab4ce99bb79a3415#diff-b9ad88b4882e66b5be4541046b713b51). 
It doesn't look like any follow up happened to the check-ignore docs.

To me, the return status for the plain vanilla case should always work 
correctly. So that is, given one path, using the current index, we know if 
it is, or is not, ignored.

The verbose output for debugging should be able to be varied at will (e.g. 
re-inclusion can show all operative lines in the debug output). It shouldn't 
be a backward compatibility issue.

Maybe that, verbosely, for each path given, the summary status is also given 
now that re-inclusion should be possible, which would not have been required 
before.
--
Philip