git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: Daniel Koverman <dkoverman@predictiveTechnologies.com>
Cc: Junio C Hamano <gitster@pobox.com>,
	"git@vger.kernel.org" <git@vger.kernel.org>,
	Nasser Grainawi <nasser@codeaurora.org>
Subject: Re: Why does send-pack call pack-objects for all remote refs?
Date: Mon, 14 Dec 2015 16:04:29 -0500	[thread overview]
Message-ID: <20151214210429.GC14788@sigill.intra.peff.net> (raw)
In-Reply-To: <d0a39b03e49d41e685cf61398c0d1102@EXCHANGE2U.uunet.arlington.PredictiveTechnologies.com>

On Mon, Dec 14, 2015 at 01:47:39PM +0000, Daniel Koverman wrote:

> > You might also try repacking with "git repack -adb", which will
> > build reachability bitmaps. Pack-objects can use them to compute
> > the set of required objects much faster.
> 
> Running "git repack -adb" caused my push time to incease by about 5x.
> I made some fresh clones and tried other options with repack, and
> consistently anything I tried with -b caused the push time to
> increase about 5x.
> 
> I don't know much about reachability bitmaps, but perhaps it is
> important to note that I timed the pushes after repacking on Git for
> Windows. My earlier timings were done on both Linux and Windows and I
> did not see a significant difference.

Hmm. I guess that makes sense. The bitmap we want is the set difference
between the objects we are sending, and the tips the other side has. If
we have a bitmap at each ref tip, that's very fast. But if you have a
very large number of refs, we don't make one for each ref, and it has to
fallback to walking to the nearest one (and it ends up worse than a
regular walk, because it's filling in the bitmap for each tree, rather
than just doing the "good enough" commit walk that we usually do).

I suspect there's room for improvement in the way we select commits to
store bitmaps for (so that the average walk is smaller). But it's rather
tricky; there's not a single constant to change to make it work better.

Thanks for trying out my suggestion.

-Peff

  reply	other threads:[~2015-12-14 21:14 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-12-07 21:02 Why does send-pack call pack-objects for all remote refs? Daniel Koverman
2015-12-07 22:41 ` Junio C Hamano
2015-12-07 22:57   ` Jeff King
2015-12-08 17:34     ` Daniel Koverman
2015-12-10  4:19       ` Jeff King
2015-12-12  4:15         ` Nasser Grainawi
2015-12-14 13:47         ` Daniel Koverman
2015-12-14 21:04           ` Jeff King [this message]
2015-12-14 22:31             ` Jonathan Nieder
2015-12-14 22:37               ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20151214210429.GC14788@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=dkoverman@predictiveTechnologies.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=nasser@codeaurora.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).