Re: git slow unless piped to cat

["Uwe Kleine-König" <u.kleine-koenig@pengutronix.de>]

On Tue, Aug 27, 2019 at 11:44:07AM +0200, SZEDER Gábor wrote:
> On Tue, Aug 27, 2019 at 11:12:06AM +0200, Philipp Zabel wrote:
> > On Tue, 2019-08-27 at 10:56 +0200, Uwe Kleine-König wrote:
> > > On Tue, Aug 27, 2019 at 10:41:11AM +0200, SZEDER Gábor wrote:
> > > > On Tue, Aug 27, 2019 at 10:15:59AM +0200, Uwe Kleine-König wrote:
> > > > > I have a problem here with git being slow in some situations.
> > > > > Using git 2.23.0 (from Debian) the effect is:
> > > > > 
> > > > > ukl@dude.ptx:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time git show v5.2
> > > > > tag v5.2
> > > > > ...
> > > > > 
> > > > > real	0m12.727s
> > > > > user	0m0.300s
> > > > > sys	0m0.371s
> > > > > 
> > > > > But to get the actual data isn't the problem:
> > > > > 
> > > > > ukl@dude.ptx:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time git show v5.2 | cat
> > > > > tag v5.2
> > > > > ...
> > > > > 
> > > > > real	0m0.764s
> > > > > user	0m0.014s
> > > > > sys	0m0.020s
> > > > > 
> > > > 
> > > > How does 'git --no-pager show v5.2' perform?  If it's as fast as the
> > > > case piping the output to cat, then look into what pager and pager
> > > > options you use.
> > > 
> > > 	ukl@dude.ptx:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time git --no-pager show v5.2
> > > 	tag v5.2
> > > 	...
> > > 
> > > 	real	0m13.225s
> > > 	user	0m0.355s
> > > 	sys	0m0.336s
> > > 
> > > So this doesn't seem to be the problem. Also the local configuration
> > > can be ruled out:
> > > 
> > > 	ukl@dude.ptx:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time env GIT_CONFIG_NOSYSTEM=1 HOME=/nonexistant XDG_CONFIG_HOME=/nonexistant git --no-pager show --no-color v5.2
> > > 	tag v5.2
> > > 	...
> > > 
> > > 	real	0m13.587s
> > > 	user	0m0.335s
> > > 	sys	0m0.336s
> > > 
> > > Thanks
> > > Uwe
> > 
> > Have you checked strace output? I see a directory walk through .git/refs
> > and .git/packed-refs if the output is not redirected.
> 
> That's a good point, 'git show/log' show decorations (refs pointing to
> any commits shown) when the output is a terminal:
> 
>   $ git show v5.2
>   [...]
>   commit 0ecfebd2b52404ae0c54a878c872bb93363ada36 (HEAD -> master, tag: v5.2)
>   [...]
>   $ git show v5.2 |cat
>   [...]
>   commit 0ecfebd2b52404ae0c54a878c872bb93363ada36
>   [...]

> And indeed, if you have a lot of refs and cold cache, then that might
> account for a couple of seconds difference.  So, how does 'git show
> --no-decorate v5.2' perform, and if it performs well, then how many
> refs do you have ('git for-each-ref |wc -l')?

	$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time env GIT_CONFIG_NOSYSTEM=1 HOME=/nonexistant XDG_CONFIG_HOME=/nonexistant git --no-pager show --no-color --no-decorate v5.2
	...

	real	0m1.041s
	user	0m0.000s
	sys	0m0.021s

	$ git for-each-ref |wc -l
	10013

So this is indeed the problem.

I'm a bit surprised that the default for --decorate depends on the
output being a terminal.

Thanks for your help, I will think about what I want to do. Just using
--no-decorate will work, but isn't nice either. Will test if just
throwing away all those tags from linux-next will make this already
better.

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |