Re: Poor performance of git describe in big repos

["Alex Bennée" <kernel-hacker@bennee.com>]

On 30 May 2013 15:32, Ramkumar Ramachandra <artagnon@gmail.com> wrote:
> Alex Bennée wrote:
>> And through my "special" repo:
>>
>>  41.58%   git  libcrypto.so.1.0.0  [.] sha1_block_data_order_ssse3
>>  33.62%   git  libz.so.1.2.3.4     [.] inflate_fast
>>  10.39%   git  libz.so.1.2.3.4     [.] adler32
>>   2.03%   git  [kernel.kallsyms]   [k] clear_page_c
>>
>>  I'm not sure why libcrypto features so highly in the results
>
> While Duy churns on the delta chain, let me try to make a (rather
> crude) observation:
>
> What does it mean for libcrypto to be so high in your perf report?
> sha1_block_data_order is ultimately by object.c:parse_object.  While
> it indicates that deltas are taking a long time to apply (or are
> somehow not optimally organized for IO), I think it indicates either:
>
> 1. Your history is very deep and there are an unusually high number of
> deltas for each blob.  What are the total number of commits?

Well the history does en-compose about 10 years of product development
and has a high number of files in the repo (including about 3 copies of
the kernel - sans upstream history).

15:50 ajb@sloy/x86_64 [work.git] >time git log --pretty=oneline | wc -l
24648

real    0m0.434s
user    0m0.388s
sys     0m0.112s

Although it doesn't take too long to walk the whole mainline history
(obviously ignoring all the other branches).

15:52 ajb@sloy/x86_64 [work.git] >git count-objects -v -H
count: 581
size: 5.09 MiB
in-pack: 399307
packs: 1
size-pack: 1.49 GiB
prune-packable: 0
garbage: 0
size-garbage: 0 bytes

It is a pick repo. The gc --aggressive nearly took out my machine keeping
around 4gb resident for most of the half hour and using nearly 8gb of VM.

Of course most of the history is not needed for day to day stuff. Maybe
if I split the pack files up it wouldn't be quite such a strain to work
through them?

> 2. You have have huge (binary) files checked into your repository.  Do
> you?  If so, why isn't the code in streaming.c kicking in?

We do have some binary blobs in the repository (mainly DSP and FPGA images)
although not a huge number:

15:58 ajb@sloy/x86_64 [work.git] >time git log --pretty=oneline -- xxx
xxx/xxxxxx/*.out ./xxx/xxx/*.out ./xxx/xxxxxxx/*.out | wc -l
234

real    0m0.590s
user    0m0.552s
sys     0m0.040s

How can I tell if streaming is kicking in or now?


-- 
Alex, homepage: http://www.bennee.com/~alex/