From: Jeff King <peff@peff.net>
To: Uri Moszkowicz <uri@4refs.com>
Cc: git@vger.kernel.org
Subject: Re: Long clone time after "done."
Date: Thu, 8 Nov 2012 10:56:07 -0500 [thread overview]
Message-ID: <20121108155607.GD15560@sigill.intra.peff.net> (raw)
In-Reply-To: <CAMJd5ASL1UPfzUmfXqACuYuGhXQjJLqbsjq7dfpRpF4K-hZFtA@mail.gmail.com>
On Wed, Nov 07, 2012 at 11:32:37AM -0600, Uri Moszkowicz wrote:
> #4 parse_object (sha1=0xb0ee98
> "\017C\205Wj\001`\254\356\307Z\332\367\353\233.\375P}D") at
> object.c:212
> #5 0x00000000004ae9ec in handle_one_ref (path=0xb0eec0
> "refs/tags/<removed>", sha1=0xb0ee98
> "\017C\205Wj\001`\254\356\307Z\332\367\353\233.\375P}D", flags=2,
> cb_data=<optimized out>) at pack-refs.
>
> [...]
>
> It looks like handle_one_ref() is called for each ref and most result
> in a call to read_sha1_file().
Right. When generating the packed-refs file, we include the "peeled"
reference for a tag (i.e., the commit that a tag object points to). So
we have to actually read any tag objects to get the value.
The upload-pack program generates a similar list, and I recently added
some optimizations. This code path could benefit from some of them by
using "peel_ref" instead of hand-rolling the tag dereferencing. The main
optimization, though, is reusing peeled values that are already in
packed-refs; we would probably need some additional magic to reuse the
values from the source repository.
However:
> It only takes a second or so for each call but when you have thousands
> of them (one for each ref) it adds up.
I am more concerned that it takes a second to read each tag. Even in my
pathological tests for optimizing upload-pack, peeling 50,000 refs took
only half a second.
> Adding --single-branch --branch <branch> doesn't appear to help as
> it is implemented afterwards. I would like to debug this problem
> further but am not familiar enough with the implementation to know
> what the next step is. Can anyone offer some suggestions? I don't see
> why a clone should be dependent on an O(#refs) operations.
Does this patch help? In a sample repo with 5000 annotated tags, it
drops my local clone time from 0.20s to 0.11s. Which is a big percentage
speedup, but this code isn't taking a long time in the first place for
me.
---
diff --git a/pack-refs.c b/pack-refs.c
index f09a054..3344749 100644
--- a/pack-refs.c
+++ b/pack-refs.c
@@ -40,13 +40,9 @@ static int handle_one_ref(const char *path, const unsigned char *sha1,
fprintf(cb->refs_file, "%s %s\n", sha1_to_hex(sha1), path);
if (is_tag_ref) {
- struct object *o = parse_object(sha1);
- if (o->type == OBJ_TAG) {
- o = deref_tag(o, path, 0);
- if (o)
- fprintf(cb->refs_file, "^%s\n",
- sha1_to_hex(o->sha1));
- }
+ unsigned char peeled[20];
+ if (!peel_ref(path, peeled))
+ fprintf(cb->refs_file, "^%s\n", sha1_to_hex(peeled));
}
if ((cb->flags & PACK_REFS_PRUNE) && !do_not_prune(flags)) {
next prev parent reply other threads:[~2012-11-08 15:56 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-23 18:30 Long clone time after "done." Uri Moszkowicz
2012-10-24 4:29 ` Nguyen Thai Ngoc Duy
2012-10-24 17:14 ` Uri Moszkowicz
2012-11-07 17:32 ` Uri Moszkowicz
2012-11-08 15:56 ` Jeff King [this message]
2012-11-08 17:20 ` Uri Moszkowicz
2012-11-08 20:33 ` Jeff King
2012-11-08 21:49 ` Uri Moszkowicz
2012-11-08 22:11 ` Jeff King
2012-11-08 22:16 ` Uri Moszkowicz
2012-11-08 22:33 ` Jeff King
2012-11-08 23:35 ` Uri Moszkowicz
2012-11-26 18:06 ` Uri Moszkowicz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121108155607.GD15560@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=uri@4refs.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).