From: Glen Choo <chooglen@google.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: git@vger.kernel.org, Jonathan Tan <jonathantanmy@google.com>
Subject: Re: [PATCH v3 2/3] builtin/fetch: skip unnecessary tasks when using --negotiate-only
Date: Wed, 22 Dec 2021 12:27:15 -0800 [thread overview]
Message-ID: <kl6l4k70z8po.fsf@chooglen-macbookpro.roam.corp.google.com> (raw)
In-Reply-To: <xmqqpmpojv48.fsf@gitster.g>
Junio C Hamano <gitster@pobox.com> writes:
> Glen Choo <chooglen@google.com> writes:
>
>> I would have come to same conclusion if I agreed that we should recurse
>> into submodules even if no objects are fetched. When I first wrote this
>> patch, I was convinced that "no new objects" implies "no need to update
>> submodules" (see my response at [1]), but I'm not sure any more and I'd
>> like to check my understanding.
>
> For example, there is a "everything_local()" optimization in the
> fetch-pack codepath where the following steps happen:
>
> (1) we look at the current value of the remote refs we are fetching
> from their ls_refs output (let's call it "new tips");
>
> (2) we notice that all of these objects happen to exist in our
> object store.
>
> (3) we make sure that we do not see any "missing links" if we run a
> reachability traversal that starts from these objects and our
> existing refs, and stops when the traversal intersect.
>
> When the last step finds that all objects necessary to point at
> these "new tips" with our refs safely, then we have no reason to
> perform physical transfer of objects. Yet, we'd update our refs
> to the "new tips".
>
> This can happen in a number of ways.
>
> Imagine that you have a clone of https://github.com/git/git/ for
> only its 'main' branch (i.e. a single-branch clone). If you then
> say "git fetch origin maint:maint", we'll learn that the tip of
> their 'maint' branch points at a commit, we look into our object
> store, find that there is no missing object to reach from it to the
> part of the object graph that is reachable from our refs (i.e. my
> 'maint' is always an ancestor of my 'main'), and we find that there
> is no reason to transfer any object. Yet we will carete a new ref
> and point at the commit.
>
> Or if you did "git branch -d" locally, making objects unreachable in
> your object store, and then fetch from your upstream, which had fast
> forwarded to the contents of the branch you just deleted.
>
> Or they rewound and rebuilt their branches since you fetched the
> last time, and then they realized their mistake and now their refs
> point at a commit that you have already seen but are different from
> what your remote-tracking branches point at now.
>
> Or you are using Derrick's "prefetch" (in "git maintenance run") and
> a background process already downloaded the objects needed for the
> branch you are fetching in the past.
>
> Depending on what happened when these objects were pre-fetched, such
> a real fetch that did not have to perform an object transfer may
> likely to need to adjust things in the submodule repository.
> "prefetch" is designed not to disrupt and to be invisible to the
> normal operation as much as possible, so I would expect that it
> won't do any priming of the submodules based on what it prefetched
> for the superproject, for example.
Thanks for providing concrete examples; that's a lot more convincing
than my hypothetical.
> So in short, physical object transfer can be optimized out, even
> when the external world view, i.e. where in the history graph the
> refs point at, changes and makes it necessary to check in the
> submodule repositories.
Yes, you're right. I'll move the jump accordingly :)
next prev parent reply other threads:[~2021-12-22 20:27 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-07 19:29 [PATCH] builtin/fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2021-12-09 22:12 ` Jonathan Tan
2021-12-09 22:36 ` Glen Choo
2021-12-13 22:58 ` Jonathan Tan
2021-12-16 18:11 ` Glen Choo
2021-12-17 0:02 ` [PATCH v2] " Glen Choo
2021-12-17 23:35 ` Junio C Hamano
2021-12-20 19:37 ` Glen Choo
2021-12-20 19:56 ` Junio C Hamano
2021-12-20 20:54 ` Glen Choo
2021-12-20 22:12 ` Junio C Hamano
2021-12-21 0:18 ` Glen Choo
2021-12-21 23:07 ` Glen Choo
2021-12-22 0:11 ` [PATCH v3 0/3] " Glen Choo
2021-12-22 0:11 ` [PATCH v3 1/3] builtin/fetch: use goto cleanup in cmd_fetch() Glen Choo
2021-12-22 0:11 ` [PATCH v3 2/3] builtin/fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2021-12-22 6:42 ` Junio C Hamano
2021-12-22 17:28 ` Glen Choo
2021-12-22 19:29 ` Junio C Hamano
2021-12-22 20:27 ` Glen Choo [this message]
2021-12-22 0:11 ` [PATCH v3 3/3] builtin/fetch: die on --negotiate-only and --recurse-submodules Glen Choo
2021-12-22 6:46 ` Junio C Hamano
2021-12-23 19:08 ` Jonathan Tan
2022-01-13 0:44 ` [PATCH v4 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-13 0:44 ` [PATCH v4 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-13 0:45 ` [PATCH v4 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-13 0:45 ` [PATCH v4 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-13 1:16 ` Junio C Hamano
2022-01-18 18:54 ` [PATCH v5 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-18 18:54 ` [PATCH v5 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-18 18:54 ` [PATCH v5 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-18 18:54 ` [PATCH v5 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-18 22:05 ` Junio C Hamano
2022-01-18 23:41 ` Glen Choo
2022-01-19 0:26 ` Junio C Hamano
2022-01-19 0:00 ` [PATCH v6 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-19 0:00 ` [PATCH v6 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-19 0:00 ` [PATCH v6 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-19 0:00 ` [PATCH v6 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-20 2:38 ` Jiang Xin
2022-01-20 17:40 ` Glen Choo
2022-01-20 17:49 ` [PATCH v7 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-20 17:49 ` [PATCH v7 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-20 17:49 ` [PATCH v7 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-20 17:49 ` [PATCH v7 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-20 23:08 ` Junio C Hamano
2022-01-20 23:16 ` Glen Choo
2022-01-20 21:58 ` Re* [PATCH v7 0/3] fetch: skip unnecessary tasks when using --negotiate-only Junio C Hamano
2022-01-20 23:15 ` Glen Choo
2022-01-21 2:17 ` Jiang Xin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=kl6l4k70z8po.fsf@chooglen-macbookpro.roam.corp.google.com \
--to=chooglen@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jonathantanmy@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).