Re: how to speed up "git log"?

[Linus Torvalds <torvalds@linux-foundation.org>]

On Mon, 12 Feb 2007, Bruno Haible wrote:

> Hello Johannes,
> 
> Thanks for the helpful answer.
> 
> > Yes, because there were only 147 commits which changed the file. But git 
> > looked at all commits to find that.
> 
> Ouch.

This should become a FAQ.

Git simply DOES NOT HAVE per-file history. And having it is actually a 
BUG in other systems.

Not having per-file history is what allows git to do

	git log directory-or-file-set

ratehr than being able to track just one file. You can't do it sanely 
with per-file history (because to tie the per-file histories back 
together in a logical sequence, you need the global history to sort it 
again!)

So:

 - git is "slow" on single-file things, because such things DON'T EVEN 
   EXIST in git!

   When you do "git log <path-limiter>", itreally always ends up being a 
   full git log. 

 - but this is fundamentally what allows you to track multiple directories 
   well. It's what makes things like "gitk drivers/scsi/" actually work, 
   where you really can see the history for a random *collection* of 
   files. Nobody else can do it, afaik, and git just considers a single 
   filename to be a case of the "random collection of files".

The example I gave to corecode was to do

	gitk builtin-rev-list.c
	gitk builtin-rev-parse.c
	gitk builtin-rev-parse.c builtin-rev-list.c

adn realize that doing the history for two files together is NOT AT ALL 
EQUIVALENT to doing the history for those files individually and stitching 
it together.

(The reason the above is a great example is that both of the files alone 
have a very simple linear history, but when you look at the *combined* 
history you actually see concurrent development, and merges: you see 
merge commits that simply don't "exist" when only looking at the history 
of one of them separately).

> Is there some other concept or command that git offers? I'm in the situation
> where I know that 'tr' in coreutils version 5.2.1 had a certain bug and
> version 6.4 does not have the bug, and I want to review all commits that
> are relevant to this. I know that the only changes in tr.c are relevant
> for this, and I'm interested in a display of the minimum amount of relevant
> commit messages. If "git log" is not the right command for this question,
> which command is it?

Do

	git log v5.2.1..v6.4 -- tr.c

(or whatever your tag-names for releases are) where you can limit the log 
generation cost by giving the beginning commit. But yeah, it *will* look 
at the whole history in between, so if there is a long long history 
between v5.2.1 and v6.4, you'll still end up using reasonable amounts of 
CPU.

> > Probably the mmap() problem. Does it go away when you use git 1.5.0-rc4?
> 
> No, it became even worse: git-1.5.0-rc4 is twice as slow as git-1.4.4 for
> this command:
>   git-1.4.4: 25 seconds real time, 24 seconds of CPU time (12 user, 12 system)
>   git-1.5.0: 50 seconds real time, 39 seconds of CPU time (20 user, 19 system)

That's an interesting fact in itself. Do you have the repo available 
somewhere?

Yes, some of the operations can be improved upon by not wasting quite so 
much time uncompressing stuff, so we could at least help this a bit. But 
that's a long-term thing. The slowdown is bad, and that probably has some 
simple explanation.

		Linus