ls-files bug report

ls-files bug report

[christian w]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

git ls-files --others 'some/path/*.some.extension'

What did you expect to happen? (Expected behavior)

only list files under some/path/ that had .some.extension

What happened instead? (Actual behavior)

It listed those files and also all untracked folders underneath some/path/

What's different between what you expected and what actually happened?

The folders did not match the grep pattern but were still listed.

Anything else you want to add:

Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.28.0
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Darwin 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 18 21:21:34
PDT 2020; root:xnu-4570.71.82.5~1/RELEASE_X86_64 x86_64
compiler info: clang: 10.0.0 (clang-1000.11.45.5)
libc info: no libc information available
$SHELL (typically, interactive shell): /bin/bash


[Enabled Hooks]
pre-commit
commit-msg
post-commit
post-checkout
post-merge
pre-push
post-rewrite

Re: ls-files bug report

[Kyle Meyer]

christian w writes:

> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
>
> What did you do before the bug happened? (Steps to reproduce your issue)
>
> git ls-files --others 'some/path/*.some.extension'
>
> What did you expect to happen? (Expected behavior)
>
> only list files under some/path/ that had .some.extension
>
> What happened instead? (Actual behavior)
>
> It listed those files and also all untracked folders underneath some/path/

I tried to follow your description, and I don't see this on my end.  In
a fresh repository:

    $ mkdir -p some/path/d0
    $ mkdir -p some/path/d1
    $ touch some/path/f0 some/path/d0/f0 some/path/f1.some.extension
    
    $ git ls-files --others
    some/path/d0/f0
    some/path/f0
    some/path/f1.some.extension
    
    $ git ls-files --others 'some/path/*.some.extension'
    some/path/f1.some.extension
    
    $ git version
    git version 2.28.0

Could you provide a more detailed recipe to reproduce the issue?

> [System Info]
> git version:
> git version 2.28.0

Re: ls-files bug report

[christian w]

Sorry for the insufficient reproduction steps.

I managed to figure out it has something to do with git repos within
git repos. It happened to us because of go packages that are inside
our repo. These are the reproduction steps that work for me
consistently using Go version go1.14.4 darwin/amd64. This is the repo
that gets cloned into src/golang.org/x/debug by the go get command:
https://github.com/golang/debug/tree/66ec140f2f72d15dc6133502edd2bb7238b1740c

$ mkdir testdir
$ cd testdir
$ git init .
$ GOPATH=$(pwd) go get -u golang.org/x/debug/ || true
$ git ls-files --others '*.txt'#
src/golang.org/x/debug/

On Wed, Aug 5, 2020 at 7:59 PM Kyle Meyer <kyle@kyleam.com> wrote:
>
> christian w writes:
>
> > Thank you for filling out a Git bug report!
> > Please answer the following questions to help us understand your issue.
> >
> > What did you do before the bug happened? (Steps to reproduce your issue)
> >
> > git ls-files --others 'some/path/*.some.extension'
> >
> > What did you expect to happen? (Expected behavior)
> >
> > only list files under some/path/ that had .some.extension
> >
> > What happened instead? (Actual behavior)
> >
> > It listed those files and also all untracked folders underneath some/path/
>
> I tried to follow your description, and I don't see this on my end.  In
> a fresh repository:
>
>     $ mkdir -p some/path/d0
>     $ mkdir -p some/path/d1
>     $ touch some/path/f0 some/path/d0/f0 some/path/f1.some.extension
>
>     $ git ls-files --others
>     some/path/d0/f0
>     some/path/f0
>     some/path/f1.some.extension
>
>     $ git ls-files --others 'some/path/*.some.extension'
>     some/path/f1.some.extension
>
>     $ git version
>     git version 2.28.0
>
> Could you provide a more detailed recipe to reproduce the issue?
>
> > [System Info]
> > git version:
> > git version 2.28.0

Re: ls-files bug report

[Kyle Meyer]

christian w writes:

> Sorry for the insufficient reproduction steps.
>
> I managed to figure out it has something to do with git repos within
> git repos. It happened to us because of go packages that are inside
> our repo. These are the reproduction steps that work for me
> consistently using Go version go1.14.4 darwin/amd64. This is the repo
> that gets cloned into src/golang.org/x/debug by the go get command:
> https://github.com/golang/debug/tree/66ec140f2f72d15dc6133502edd2bb7238b1740c
>
> $ mkdir testdir
> $ cd testdir
> $ git init .
> $ GOPATH=$(pwd) go get -u golang.org/x/debug/ || true
> $ git ls-files --others '*.txt'#
> src/golang.org/x/debug/

Okay, removing Go from the equation:

     $ # in fresh repo
     $ mkdir a
     $ git init a/d0
     $ touch a/f0.txt
     $ git ls-files --others 'a/*.txt'
     a/d0/
     a/f0.txt

It looks like the spurious a/d0/ entry doesn't show up until v2.27.0, in
particular 95c11ecc73 (Fix error-prone fill_directory() API; make it
only return matches, 2020-04-01).  Adding the author to the Cc.  (Sorry,
Elijah, I recall your frustration with dir.c from a previous ls-files
issue that I reported [1]).

[1] https://lore.kernel.org/git/CABPp-BFG3FkTkC=L1v97LUksndkOmCN8ZhNJh5eoNdquE7v9DA@mail.gmail.com/

Re: ls-files bug report

[Elijah Newren]

On Wed, Aug 5, 2020 at 6:54 PM Kyle Meyer <kyle@kyleam.com> wrote:
>
> christian w writes:
>
> > Sorry for the insufficient reproduction steps.
> >
> > I managed to figure out it has something to do with git repos within
> > git repos. It happened to us because of go packages that are inside
> > our repo. These are the reproduction steps that work for me
> > consistently using Go version go1.14.4 darwin/amd64. This is the repo
> > that gets cloned into src/golang.org/x/debug by the go get command:
> > https://github.com/golang/debug/tree/66ec140f2f72d15dc6133502edd2bb7238b1740c
> >
> > $ mkdir testdir
> > $ cd testdir
> > $ git init .
> > $ GOPATH=$(pwd) go get -u golang.org/x/debug/ || true
> > $ git ls-files --others '*.txt'#
> > src/golang.org/x/debug/
>
> Okay, removing Go from the equation:
>
>      $ # in fresh repo
>      $ mkdir a
>      $ git init a/d0
>      $ touch a/f0.txt
>      $ git ls-files --others 'a/*.txt'
>      a/d0/
>      a/f0.txt
>
> It looks like the spurious a/d0/ entry doesn't show up until v2.27.0, in
> particular 95c11ecc73 (Fix error-prone fill_directory() API; make it
> only return matches, 2020-04-01).  Adding the author to the Cc.  (Sorry,
> Elijah, I recall your frustration with dir.c from a previous ls-files
> issue that I reported [1]).

Thanks for digging in and coming up with a smaller list of steps to
reproduce.  You are right that I dread reports that touch dir.c -- if
I would have just ignored the first report back in March of 2018,
there's a really, really long list of problems I could have avoided...

I'll try to take a look in the next week.

>
> [1] https://lore.kernel.org/git/CABPp-BFG3FkTkC=L1v97LUksndkOmCN8ZhNJh5eoNdquE7v9DA@mail.gmail.com/

Re: ls-files bug report

[Elijah Newren]

On Thu, Aug 6, 2020 at 9:10 PM Elijah Newren <newren@gmail.com> wrote:
>
> On Wed, Aug 5, 2020 at 6:54 PM Kyle Meyer <kyle@kyleam.com> wrote:
> >
> > christian w writes:
> >
> > > Sorry for the insufficient reproduction steps.
> > >
> > > I managed to figure out it has something to do with git repos within
> > > git repos. It happened to us because of go packages that are inside
> > > our repo. These are the reproduction steps that work for me
> > > consistently using Go version go1.14.4 darwin/amd64. This is the repo
> > > that gets cloned into src/golang.org/x/debug by the go get command:
> > > https://github.com/golang/debug/tree/66ec140f2f72d15dc6133502edd2bb7238b1740c
> > >
> > > $ mkdir testdir
> > > $ cd testdir
> > > $ git init .
> > > $ GOPATH=$(pwd) go get -u golang.org/x/debug/ || true
> > > $ git ls-files --others '*.txt'#
> > > src/golang.org/x/debug/
> >
> > Okay, removing Go from the equation:
> >
> >      $ # in fresh repo
> >      $ mkdir a
> >      $ git init a/d0
> >      $ touch a/f0.txt
> >      $ git ls-files --others 'a/*.txt'
> >      a/d0/
> >      a/f0.txt
> >
> > It looks like the spurious a/d0/ entry doesn't show up until v2.27.0, in
> > particular 95c11ecc73 (Fix error-prone fill_directory() API; make it
> > only return matches, 2020-04-01).  Adding the author to the Cc.  (Sorry,
> > Elijah, I recall your frustration with dir.c from a previous ls-files
> > issue that I reported [1]).
>
> Thanks for digging in and coming up with a smaller list of steps to
> reproduce.  You are right that I dread reports that touch dir.c -- if
> I would have just ignored the first report back in March of 2018,
> there's a really, really long list of problems I could have avoided...
>
> I'll try to take a look in the next week.

Should be solved by this fix:
https://lore.kernel.org/git/11a286b071ca8a6b96f4fba6658e9bafb9314be4.1597216356.git.gitgitgadget@gmail.com/

Re: ls-files bug report

[christian w]

that's great to hear, thanks for jumping on this so quickly!

On Wed, Aug 12, 2020 at 10:50 AM Elijah Newren <newren@gmail.com> wrote:
>
> On Thu, Aug 6, 2020 at 9:10 PM Elijah Newren <newren@gmail.com> wrote:
> >
> > On Wed, Aug 5, 2020 at 6:54 PM Kyle Meyer <kyle@kyleam.com> wrote:
> > >
> > > christian w writes:
> > >
> > > > Sorry for the insufficient reproduction steps.
> > > >
> > > > I managed to figure out it has something to do with git repos within
> > > > git repos. It happened to us because of go packages that are inside
> > > > our repo. These are the reproduction steps that work for me
> > > > consistently using Go version go1.14.4 darwin/amd64. This is the repo
> > > > that gets cloned into src/golang.org/x/debug by the go get command:
> > > > https://github.com/golang/debug/tree/66ec140f2f72d15dc6133502edd2bb7238b1740c
> > > >
> > > > $ mkdir testdir
> > > > $ cd testdir
> > > > $ git init .
> > > > $ GOPATH=$(pwd) go get -u golang.org/x/debug/ || true
> > > > $ git ls-files --others '*.txt'#
> > > > src/golang.org/x/debug/
> > >
> > > Okay, removing Go from the equation:
> > >
> > >      $ # in fresh repo
> > >      $ mkdir a
> > >      $ git init a/d0
> > >      $ touch a/f0.txt
> > >      $ git ls-files --others 'a/*.txt'
> > >      a/d0/
> > >      a/f0.txt
> > >
> > > It looks like the spurious a/d0/ entry doesn't show up until v2.27.0, in
> > > particular 95c11ecc73 (Fix error-prone fill_directory() API; make it
> > > only return matches, 2020-04-01).  Adding the author to the Cc.  (Sorry,
> > > Elijah, I recall your frustration with dir.c from a previous ls-files
> > > issue that I reported [1]).
> >
> > Thanks for digging in and coming up with a smaller list of steps to
> > reproduce.  You are right that I dread reports that touch dir.c -- if
> > I would have just ignored the first report back in March of 2018,
> > there's a really, really long list of problems I could have avoided...
> >
> > I'll try to take a look in the next week.
>
> Should be solved by this fix:
> https://lore.kernel.org/git/11a286b071ca8a6b96f4fba6658e9bafb9314be4.1597216356.git.gitgitgadget@gmail.com/