Re: git stash --include-untracked walks ignored directories

[Brian Malehorn <bmalehorn@gmail.com>]

Ah, my original message was a bit misleading. I believe git slows down
based on the number of directories, not the number of files. Here's an
updated version of your script that creates a lot of directories
instead of a lot of files:

#!/bin/bash

rm -rf stupid
git init -q stupid
cd stupid

echo ignored >.gitignore
seq 1 10 >numbers-tracked
git add numbers-tracked .gitignore
git commit -q -m initial

seq 11 20 >>numbers-tracked
seq 21 30 >numbers-untracked

mkdir ignored
cd ignored
for i in $(seq 1 50); do
  for j in $(seq 1 1000); do
    echo "$i/$j"
  done
done | xargs mkdir -p
cd ..

echo "Number of directories in ignored before: $(find ignored -type d | wc -l)"
time git stash --include-untracked
git --version
echo "Number of directories in ignored after: $(find ignored -type d | wc -l)"

----

I got it to reproduce with the versions I have handy:

$ ./repro.sh
Number of directories in ignored before: 50051
Saved working directory and index state WIP on master: 0dabddf initial

real    0m0.023s
user    0m0.012s
sys     0m0.010s
git version 2.25.1
Number of directories in ignored after: 50051


$./repro.sh
Number of directories in ignored before: 50051
Saved working directory and index state WIP on master: 175d4ce initial

real    0m0.619s
user    0m0.157s
sys     0m0.452s
git version 2.27.0.83.g0313f36c6e
Number of directories in ignored after: 50051

On Tue, Jun 9, 2020 at 8:21 PM Elijah Newren <newren@gmail.com> wrote:
>
> On Tue, Jun 9, 2020 at 1:39 PM Brian Malehorn <bmalehorn@gmail.com> wrote:
> >
> > Hi,
> >
> > Not sure if this is the right place to send this, but I'm here to
> > report a performance regression with git stash --include-untracked.
> >
> > Here's a quick way to reproduce:
> >
> > 1. make a directory with a lot of ignored files
> >
> > $ find ignored -type f | wc -l
> >    50000
> >
> > $ cat .gitignore
> > ignored
> >
> > 2. touch foo
> >
> > 3. time git stash --include-untracked
> >
> > git version 2.26.0:
> > real    0m0.094s
> >
> > git version 2.27.0.83.g0313f36c6e:
> > real    0m1.913s
> >
> > This is a much bigger pain point on my work repo, which has 1.4
> > million ignored files(!). As you can imagine it takes a long time to
> > run git stash. While it might be valid to question why anyone would
> > need that many files for any purpose, the bottom line is that I told
> > git to ignore this directory, and it didn't ignore it.
> >
> > In the meantime I've reverted to 2.26.0 which doesn't have this
> > performance regression. Let me know if you want any other information
> > related to this issue.
> >
> > Thanks,
> > Brian
>
> I seem to be missing some important step to reproduce; what else is
> needed?  Here's what I see:
>
> <Set path to use git-2.26.0>
> $ ./repro.sh
> Number of files in ignored before: 50000
> Saved working directory and index state WIP on master: e2b0471 initial
>
> real 0m0.029s
> user 0m0.014s
> sys 0m0.014s
> git version 2.26.0
> Number of files in ignored after: 50000
>
> <Set path to use git-2.27.0>
> $ ./repro.sh
> Number of files in ignored before: 50000
> Saved working directory and index state WIP on master: 5c596b8 initial
>
> real 0m0.052s
> user 0m0.014s
> sys 0m0.034s
> git version 2.27.0
> Number of files in ignored after: 50000
>
>
> Where repro.sh is:
>
> #!/bin/bash
>
> rm -rf stupid
> git init -q stupid
> cd stupid
>
> echo ignored >.gitignore
> seq 1 10 >numbers-tracked
> git add numbers-tracked .gitignore
> git commit -q -m initial
>
> seq 11 20 >>numbers-tracked
> seq 21 30 >numbers-untracked
>
> mkdir ignored
> cd ignored
> for i in $(seq 1 50000); do >$i; done
> cd ..
>
> echo "Number of files in ignored before: $(find ignored -type f | wc -l)"
> time git stash --include-untracked
> git --version
> echo "Number of files in ignored after: $(find ignored -type f | wc -l)"