From: Kirill Smelkov <kirr@nexedi.com>
To: Jeff King <peff@peff.net>
Cc: Eric Sunshine <sunshine@sunshineco.com>,
Junio C Hamano <gitster@pobox.com>,
Jonathan Tan <jonathantanmy@google.com>,
Brandon Williams <bmwill@google.com>,
Takuto Ikuta <tikuta@chromium.org>,
Jeff Hostetler <jeffhost@microsoft.com>,
Johannes Schindelin <Johannes.Schindelin@gmx.de>,
Michael Haggerty <mhagger@alum.mit.edu>,
Git List <git@vger.kernel.org>
Subject: Re: [PATCH v2] fetch-pack: don't try to fetch peel values with --all
Date: Thu, 14 Jun 2018 05:29:42 +0000 [thread overview]
Message-ID: <20180614052938.GB2861@deco.navytux.spb.ru> (raw)
In-Reply-To: <20180613211302.GB15574@sigill.intra.peff.net>
On Wed, Jun 13, 2018 at 05:13:02PM -0400, Jeff King wrote:
> On Tue, Jun 12, 2018 at 06:54:17PM +0000, Kirill Smelkov wrote:
>
> > > If an extra connection isn't a problem, you might be better off with
> > > "git ls-remote", and then picking through the results for refs of
> > > interest, and then "git fetch-pack" to actually get the pack. That's how
> > > git-fetch worked when it was a shell script (e.g., see c3a200120d, the
> > > last shell version).
> >
> > Yes, this is what I ended up doing:
> >
> > https://lab.nexedi.com/kirr/git-backup/commit/899103bf
> >
> > but for another reason - to avoid repeating for every fetched repository
> > slow (in case of my "big" destination backup repository) quickfetch()
> > checking in every spawned `git fetch`: git-backup can build index of
> > objects we already have ourselves only once at startup, and then in
> > fetch, after checking lsremote output, consult that index, and if we see
> > we already have everything for an advertised reference - just avoid
> > giving it to fetch-pack to process. It turns out for many pulled
> > repositories there is usually no references changed at all and this way
> > fetch-pack can be skipped completely:
> >
> > https://lab.nexedi.com/kirr/git-backup/commit/3efed898
>
> Thanks for sharing that, it's an interesting case. I'd hope that
> git-fetch is smart enough not to even bother with quickfetch() if there
> are no refs to update. But if we have even one change to fetch, then
> yeah, in the general case it makes sense to me that you could do better
> by amortizing the scan of local objects across many operations.
Thanks for feedback. For the reference in case of git-backup `git fetch`
or `git fetch-pack` would have to always do quickfetch scan or equivalent,
because in case of backup repo there is only one reference in it - its
master - and references of backed up repositories do not have anything
representing them in backup.git/refs/ .
Kirill
prev parent reply other threads:[~2018-06-14 5:55 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-10 14:32 [PATCH] fetch-pack: demonstrate --all breakage when remote have tags to non-commit objects Kirill Smelkov
2018-06-11 4:20 ` Jeff King
2018-06-11 4:47 ` [PATCH] fetch-pack: don't try to fetch peeled values with --all Jeff King
2018-06-11 5:28 ` Eric Sunshine
2018-06-11 5:53 ` [PATCH v2] fetch-pack: don't try to fetch peel " Jeff King
2018-06-11 9:43 ` Kirill Smelkov
2018-06-12 9:48 ` Jeff King
2018-06-12 18:54 ` Kirill Smelkov
2018-06-13 11:18 ` [PATCH] fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits Kirill Smelkov
2018-06-13 17:42 ` Junio C Hamano
2018-06-13 18:43 ` Kirill Smelkov
2018-06-13 21:05 ` Jeff King
2018-06-13 23:11 ` Jeff King
2018-06-14 5:25 ` Kirill Smelkov
2018-06-14 16:07 ` Junio C Hamano
2018-06-14 17:51 ` Kirill Smelkov
2018-06-13 12:55 ` [PATCH] fetch-pack: demonstrate --all failure when remote is empty Kirill Smelkov
2018-06-13 17:13 ` Junio C Hamano
2018-06-13 18:21 ` Kirill Smelkov
2018-06-13 21:13 ` [PATCH v2] fetch-pack: don't try to fetch peel values with --all Jeff King
2018-06-14 5:29 ` Kirill Smelkov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180614052938.GB2861@deco.navytux.spb.ru \
--to=kirr@nexedi.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=bmwill@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jeffhost@microsoft.com \
--cc=jonathantanmy@google.com \
--cc=mhagger@alum.mit.edu \
--cc=peff@peff.net \
--cc=sunshine@sunshineco.com \
--cc=tikuta@chromium.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).