git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Patrick Marlier (pamarlie)" <pamarlie@cisco.com>
To: Jeff King <peff@peff.net>
Cc: Jonathan Tan <jonathantanmy@google.com>,
	"git@vger.kernel.org" <git@vger.kernel.org>
Subject: Re: [PATCH] send-pack: use OBJECT_INFO_QUICK to check negative objects
Date: Fri, 29 Nov 2019 09:22:22 +0000	[thread overview]
Message-ID: <CH2PR11MB4294A6C1AA71DF1A1721C94CCF460@CH2PR11MB4294.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20191127123211.GG22221@sigill.intra.peff.net>

> On Tue, Nov 19, 2019 at 01:12:51PM +0000, Patrick Marlier (pamarlie) wrote:
>> I am hitting a performance issue with "git push <remote> <refspec>".
>> The local repository has only few refs and the remote repository has a
>> lot of refs (1000+) with objects unknown to the local repository.
>> 
>> "git push" of only one refspec takes minutes to complete. A quick
>> analysis shows that a lot of time is spent in the client side.
>> A deeper analysis shows that the client receives the entire list of
>> refs on the remote, then the client is checking in its local
>> repository if the objects exist for all remote refs.

>Right, this is expected. The client send-pack feeds the list of remote
>objects (that it has) to pack-objects, which can then limit the size of
>the packfile it sends based on what the other side has.

>So the patch you showed (to skip refs that aren't part of the push)
>would miss many opportunities for a smaller push. E.g., imagine I create
>a new branch "topic" from the remote branch "master", and then try to
>push it up. If I do not look at which objects are in "master", I'll end
>up sending the whole history again, when I really just need to send the
>differences between the two.

Thanks Jeff. Indeed it makes sense.
(My very particular use-case would have a limited impact in this case)


>> Since the local repository has a only few refs, most of the objects
>> are unknown.
>>
>> This issue is particularly amplified because the local repository is
>> using many alternates. Indeed for each unknown object, git will try to
>> find in all alternates too.

> I think the patch below would help you.

I will give a try next week and I keep you posted on how it goes.

Thanks a lot for your input and the patch!

Have a nice day,
--
Pat

  reply	other threads:[~2019-11-29  9:22 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-19 13:12 Push a ref to a remote with many refs Patrick Marlier (pamarlie)
2019-11-25 16:22 ` Patrick Marlier (pamarlie)
2019-11-27 12:32 ` [PATCH] send-pack: use OBJECT_INFO_QUICK to check negative objects Jeff King
2019-11-29  9:22   ` Patrick Marlier (pamarlie) [this message]
2019-11-30 17:08   ` Junio C Hamano
2019-12-03 23:20     ` Jeff King
2019-12-04 20:53       ` Jonathan Tan
2019-12-04 21:37         ` Junio C Hamano
2019-12-04  3:55   ` Jonathan Nieder
2019-12-04  4:05     ` Jeff King
2019-12-10 16:16       ` Patrick Marlier (pamarlie)
2019-12-10 20:27         ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CH2PR11MB4294A6C1AA71DF1A1721C94CCF460@CH2PR11MB4294.namprd11.prod.outlook.com \
    --to=pamarlie@cisco.com \
    --cc=git@vger.kernel.org \
    --cc=jonathantanmy@google.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).