git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
From: Takuto Ikuta <tikuta@chromium.org>
To: Junio C Hamano <gitster@pobox.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH v3] fetch-pack.c: use oidset to check existence of loose object
Date: Wed, 14 Mar 2018 06:26:46 +0000
Message-ID: <CALNjmMqqa1i0=ekqQfNZ5DmA+h7mOGzZcF-RB1A6+Rm8vw80pQ@mail.gmail.com> (raw)
In-Reply-To: <xmqqo9jr3lrt.fsf@gitster-ct.c.googlers.com>

2018年3月14日(水) 2:53 Junio C Hamano <gitster@pobox.com>:

> Takuto Ikuta <tikuta@chromium.org> writes:

> >> During fetch, everything_local() tries to mark common part by
> >> walking the refs the other side advertised upon the first contact,
> >> so it is correct that the number of checks is not reduced in a fetch
> >> that does not fetch many refs, but the number of remote-tracking refs
> >> you have has no effect, so I doubt such a rephrasing would make the
> >> description more understandable.  "When fetching from a repository
> >> with large number of refs" is probably what you want to say, no?
> >
> > For refs existing in local repository, everything_local looks to be
able to find
> > corresponding object from packed or loose objects. And if it exists,
> > I think, cost of lstat(2) is relatively smaller than other operations.
> > But for remote refs, everything_local fails to find it from packed
> > object (this check is fast)
> > and it tries to find loose object by using lstat(2), and this fails and
slow.
> > My patch is to skip this lstat(2) to non-existing ref objects for
repositories
> > having large number of remote refs.

> This still does not make sense to me, and I suspect that I am
> misreading you.  In what sense are you using the word "repository"
> and "remote refs"?


Yes, I think I did not understand the terms correctly.
I meant "repository" for repository in remote servers, and "remote refs" for
refs shown by `git ls-remote`.

> Imagine this situation.  I have a local repository A, I fetch from a
> remote repository B but in my repository A, I do *not* use
> remote-tracking refs to remember what the last values of refs at
> repository B.  Now when I try to fetch a single ref from B into A,
> many refs B advertises point at objects A has never heard about, and
> that triggers many lstat(2) that yields ENOENT that is slow.  Your
> patch is to optimize this so that we learn these objects do not
> exist locally without running many lstat(2) to objects and helps
> repositories (like my repository A) when fetching from a repository
> with large number of refs (like the repository B).  It does not
> matter how many "remote refs" receiving repository (e.g. my A) has,
> and it does not matter how many "remote refs" sending repository
> (e.g. my B) has---whether it is refs/remotes/origin/foo
> (i.e. "remote") or refs/heads/foo (i.e. "local"), a ref at B that
> points at an object that is missing at A will cause the same
> lstat(2) at A without your change.

> That is why I think "When fetching from a repository with large
> number of refs" is what you meant, not "fetching into a repository
> with large number of remote refs" nor "fetching from a repository
> with large number of remote refs".


Thank you for explanation.
And yes, that is exactly I want to do in my patch.
Fixed description in v6.

Thanks.

  reply index

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-03-08 12:06 [PATCH] " Takuto Ikuta
2018-03-08 17:19 ` René Scharfe
2018-03-09 13:42   ` Takuto Ikuta
2018-03-08 18:42 ` Junio C Hamano
2018-03-09 13:11   ` [PATCH v2 0/1] " Takuto Ikuta
2018-03-09 13:11     ` [PATCH v2 1/1] " Takuto Ikuta
2018-03-09 13:26       ` [PATCH v3] " Takuto Ikuta
2018-03-09 19:54         ` Junio C Hamano
2018-03-10 13:19           ` Takuto Ikuta
2018-03-13 17:53             ` Junio C Hamano
2018-03-14  6:26               ` Takuto Ikuta [this message]
2018-03-10 12:34         ` [PATCH v4] " Takuto Ikuta
2018-03-10 12:46           ` [PATCH v5] " Takuto Ikuta
2018-03-13 19:04             ` Junio C Hamano
2018-03-14  6:05           ` [PATCH v6] " Takuto Ikuta
2018-03-14  6:32             ` [PATCH v7] " Takuto Ikuta
2018-03-09 14:12   ` [PATCH] " Takuto Ikuta
2018-03-09 18:00     ` Junio C Hamano
2018-03-09 19:41       ` Junio C Hamano
2018-03-13 15:30   ` [PATCH] sha1_file: restore OBJECT_INFO_QUICK functionality Jonathan Tan

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALNjmMqqa1i0=ekqQfNZ5DmA+h7mOGzZcF-RB1A6+Rm8vw80pQ@mail.gmail.com' \
    --to=tikuta@chromium.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox