git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Alex Bennée" <kernel-hacker@bennee.com>
To: John Keeping <john@keeping.me.uk>
Cc: git@vger.kernel.org
Subject: Re: Poor performance of git describe in big repos
Date: Thu, 30 May 2013 13:29:02 +0100	[thread overview]
Message-ID: <CAJ-05NM9EhikDBP0izqWrnLbZW6RcHq_cH-20YTE08SZw5fjqA@mail.gmail.com> (raw)
In-Reply-To: <20130530114808.GD17475@serenity.lan>

The repo is a fairly hairy one as it includes two historically
un-related but content related repos which I'm the process of
cherry-picking stuff across.

11:58 ajb@sloy/x86_64 [work.git] >git count-objects -v
count: 493
size: 4572
in-pack: 399307
packs: 1
size-pack: 1930755
prune-packable: 0
garbage: 0
size-garbage: 0

This was after a repack which did have slight negative effect on
performance. The pack file is:

13:27 ajb@sloy/x86_64 [work.git] >ls -lh ./.git/objects/pack/*
-r--r--r-- 1 ajb cvs  11M May 30 11:56
./.git/objects/pack/pack-a9ba133a6f25ffa74c3c407e09ab030f8745b201.idx
-r--r--r-- 1 ajb cvs 1.9G May 30 11:56
./.git/objects/pack/pack-a9ba133a6f25ffa74c3c407e09ab030f8745b201.pack

I ran perf on it and the top items in the report where:

 41.58%   git  libcrypto.so.1.0.0  [.] 0x6ae73
 33.96%   git  libz.so.1.2.3.4     [.] 0xe0ec
 10.39%   git  libz.so.1.2.3.4     [.] adler32
  2.03%   git  [kernel.kallsyms]   [k] clear_page_c

So I'm guessing it's spending a lot of non-cache efficient time
un-packing and processing the deltas?

--
Alex.

On 30 May 2013 12:48, John Keeping <john@keeping.me.uk> wrote:
> On Thu, May 30, 2013 at 11:38:32AM +0100, Alex Bennée wrote:
>> One factor might be the size of my repo (.git is around 2.4G). Could
>> this just be due to computational cost of searching through large
>> packs to walk the commit chain? Is there any way to make this easier
>> for git to do?
>
> What does "git count-objects -v" say for your repository?
>
> You may find that performance improves if you repack with "git gc
> --aggressive".



-- 
Alex, homepage: http://www.bennee.com/~alex/

  reply	other threads:[~2013-05-30 12:29 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-05-30 10:38 Poor performance of git describe in big repos Alex Bennée
2013-05-30 11:33 ` Ramkumar Ramachandra
2013-05-30 13:09   ` Alex Bennée
2013-05-30 14:32     ` Ramkumar Ramachandra
2013-05-30 15:01       ` Alex Bennée
2013-05-30 15:17         ` Ramkumar Ramachandra
2013-05-30 15:33     ` Thomas Rast
2013-05-30 16:01       ` Alex Bennée
2013-05-30 16:21         ` Thomas Rast
2013-05-30 16:44           ` Thomas Rast
2013-05-30 19:01             ` Antoine Pelisse
2013-05-30 20:00             ` [PATCH 1/2] sha1_file: silence sha1_loose_object_info Thomas Rast
2013-05-30 20:00               ` [PATCH 2/2] lookup_commit_reference_gently: do not read non-{tag,commit} Thomas Rast
2013-05-30 21:22                 ` Jeff King
2013-05-31  0:52                   ` Duy Nguyen
2013-05-31  8:08                   ` Thomas Rast
2013-05-31 16:00                     ` Jeff King
2013-05-31  6:43                 ` Ramkumar Ramachandra
2013-05-31  8:16                   ` Thomas Rast
2013-05-30 19:30           ` Poor performance of git describe in big repos John Keeping
2013-05-31  8:14             ` Alex Bennée
2013-05-31  8:24               ` Thomas Rast
2013-05-31  8:40                 ` Alex Bennée
2013-05-31  8:46                   ` Thomas Rast
2013-05-31  9:57                     ` Alex Bennée
2013-06-03  8:02                       ` Alex Bennée
2013-06-03 16:32                         ` Junio C Hamano
2013-06-03 17:48                           ` Junio C Hamano
2013-05-31 10:27                     ` Thomas Rast
2013-05-31 16:17                       ` Jeff King
2013-06-03  8:39                         ` Alex Bennée
2013-06-03 14:49                           ` Jeff King
2013-05-31  8:32               ` John Keeping
2013-05-31  8:49                 ` Alex Bennée
2013-05-31  8:59                   ` John Keeping
2013-05-30 11:48 ` John Keeping
2013-05-30 12:29   ` Alex Bennée [this message]
2013-05-30 13:20     ` Duy Nguyen
     [not found]       ` <CAJ-05NPacjAEC99Ntd9eMnTD9_PMMYFob-_tAx5CeSB79TkRSg@mail.gmail.com>
2013-05-30 13:45         ` Duy Nguyen
2013-05-30 14:02           ` Alex Bennée
2013-05-30 13:16   ` Alex Bennée

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAJ-05NM9EhikDBP0izqWrnLbZW6RcHq_cH-20YTE08SZw5fjqA@mail.gmail.com \
    --to=kernel-hacker@bennee.com \
    --cc=git@vger.kernel.org \
    --cc=john@keeping.me.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).