git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
* git doing large refetches
       [not found] <1731755010.6694064.1593107165458.ref@mail.yahoo.com>
@ 2020-06-25 17:46 ` Hin-Tak Leung
  2020-06-25 17:54   ` Pratyush Yadav
  0 siblings, 1 reply; 3+ messages in thread
From: Hin-Tak Leung @ 2020-06-25 17:46 UTC (permalink / raw)
  To: git, git

Hi,

Lately I noticed that git occasionally do very large re-fetches, despite the difference between local and remote being not large. For example, here are two aborted fetches (see how small the emumerating / counting /compressing are, compared to the receiving object number):

$ git fetch torvalds
remote: Enumerating objects: 19374, done.
remote: Counting objects: 100% (19374/19374), done.
remote: Compressing objects: 100% (4016/4016), done.
^Cceiving objects:   2% (161673/7478285), 80.54 MiB | 2.78 MiB/s

$ git fetch sound
remote: Enumerating objects: 52009, done.
remote: Counting objects: 100% (52009/52009), done.
remote: Compressing objects: 100% (5480/5480), done.
^Cceiving objects:   1% (74819/7481898), 37.92 MiB | 1.98 MiB/s

I don't see any real pattern, as the last few times I fetch from either of those two are also quite small ( a few days ago).

$ git --version
git version 2.26.2

stanzas from my .git/config

[remote "torvalds"]
	url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
	fetch = +refs/heads/*:refs/remotes/torvalds/*
[remote "sound"]
	url = git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound.git
	fetch = +refs/heads/*:refs/remotes/sound/*

Another general issue is that, I used to have just a few small files under .git/objects/XX, with a few large ones under .git/objects/pack . But at the moment, ".git/objects/pack" is 4.4GB, while ".git/objects/" is 5.2GB .


Also, is there a "best practice" if I like to track multiple upstreams (and the convenience of it), while keeping the local repo / heads (the part that "I do", and requiring back-up, etc). I thought of splitting the two into a "mine" repo with "info/alternates" pointing to a pure-fetch-reference-upstream repo; but I am a bit worried about doing a fetch on "mine" before or without updating the "pure-fetch-reference-upstream" repo, in which case I would end up with duplicate objects ending up in mine, and also, would be vulnerable to upstream doing rebase, for example. Any thoughts?

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: git doing large refetches
  2020-06-25 17:46 ` git doing large refetches Hin-Tak Leung
@ 2020-06-25 17:54   ` Pratyush Yadav
  0 siblings, 0 replies; 3+ messages in thread
From: Pratyush Yadav @ 2020-06-25 17:54 UTC (permalink / raw)
  To: Hin-Tak Leung; +Cc: git

Hi,

On 25/06/20 05:46PM, Hin-Tak Leung wrote:
> Hi,
> 
> Lately I noticed that git occasionally do very large re-fetches, despite the difference between local and remote being not large. For example, here are two aborted fetches (see how small the emumerating / counting /compressing are, compared to the receiving object number):
> 
> $ git fetch torvalds
> remote: Enumerating objects: 19374, done.
> remote: Counting objects: 100% (19374/19374), done.
> remote: Compressing objects: 100% (4016/4016), done.
> ^Cceiving objects:   2% (161673/7478285), 80.54 MiB | 2.78 MiB/s
> 
> $ git fetch sound
> remote: Enumerating objects: 52009, done.
> remote: Counting objects: 100% (52009/52009), done.
> remote: Compressing objects: 100% (5480/5480), done.
> ^Cceiving objects:   1% (74819/7481898), 37.92 MiB | 1.98 MiB/s
> 
> I don't see any real pattern, as the last few times I fetch from either of those two are also quite small ( a few days ago).
> 
> $ git --version
> git version 2.26.2

See https://lore.kernel.org/git/20200422155047.GB91734@google.com/

This problem shouldn't be present in v2.27.*

-- 
Regards,
Pratyush Yadav

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: git doing large refetches
       [not found] <1406746652.7528852.1593187534410.ref@mail.yahoo.com>
@ 2020-06-26 16:05 ` Hin-Tak Leung
  0 siblings, 0 replies; 3+ messages in thread
From: Hin-Tak Leung @ 2020-06-26 16:05 UTC (permalink / raw)
  To: Pratyush Yadav; +Cc: git

On Thursday, 25 June 2020, 19:06:16 BST, Pratyush Yadav <me@yadavpratyush.com> wrote:

> Hi,

> On 25/06/20 05:46PM, Hin-Tak Leung wrote:
> > Hi,
> > 
> > Lately I noticed that git occasionally do very large re-fetches, despite the difference between local and remote being not large. For example, here are two aborted fetches (see how small the emumerating / counting /compressing are, compared to the receiving object number):
> > 
> > $ git fetch torvalds
> > remote: Enumerating objects: 19374, done.
> > remote: Counting objects: 100% (19374/19374), done.
> > remote: Compressing objects: 100% (4016/4016), done.
> > ^Cceiving objects:  2% (161673/7478285), 80.54 MiB | 2.78 MiB/s
> > 
> > $ git fetch sound
> > remote: Enumerating objects: 52009, done.
> > remote: Counting objects: 100% (52009/52009), done.
> > remote: Compressing objects: 100% (5480/5480), done.
> > ^Cceiving objects:  1% (74819/7481898), 37.92 MiB | 1.98 MiB/s
> > 
> > I don't see any real pattern, as the last few times I fetch from either of those two are also quite small ( a few days ago).
> > 
> > $ git --version
> > git version 2.26.2

> See https://lore.kernel.org/git/20200422155047.GB91734@google.com/

> This problem shouldn't be present in v2.27.*

Thanks a lot! Glad to know it is already fixed!

Cheers,

Hin-Tak

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <1731755010.6694064.1593107165458.ref@mail.yahoo.com>
2020-06-25 17:46 ` git doing large refetches Hin-Tak Leung
2020-06-25 17:54   ` Pratyush Yadav
     [not found] <1406746652.7528852.1593187534410.ref@mail.yahoo.com>
2020-06-26 16:05 ` Hin-Tak Leung

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git