Re: [PATCH] fetch: do not look for submodule changes in unchanged refs

[Orgad Shaneh <orgads@gmail.com>]

Hi Junio,

Thanks for the detailed review. I posted a new commit message.

On Wed, Sep 2, 2020 at 11:26 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Orgad Shaneh via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Orgad Shaneh <orgads@gmail.com>
> >
> > This operation is very expensive, as it scans all the refs using
> > setup_revisions, which resolves each ref, including checking if it
> > is ambiguous, or if it is a file name etc.
>
> Nobody can tell what "This operation" is without looking at the
> patch/diff text.  Our commit message typically gives minimum
> explanation of the situation and the problem it tries to solve first
> to make it self sufficient.  And then we go on to order the code
> base to be in a better shape.  Something along the lines of ...
>
>     When fetching recursively with submodules, for each ref in the
>     superproject, we call check_for_new_submodule_commits() to
>     figure out X and Y for the object the ref was pointing at before
>     the fetch in the superproject, in order to ensure Z.  This is
>     expensive because of A, B and C, but it unnecessary if the fetch
>     in the superproject did not update the ref (i.e. the objects
>     that are required to exist in the submodule did not change).
>
>     Check if we are making any change to the ref, and skip the check
>     if we aren't.
>
> ... but I didn't fill the most important bits in the above, as by
> now you, as the person who encountered the issue and figured out a
> good way to solve it, would know what to fill the placeholders with
> far better than I would ;-)

That was very helpful. Thanks.

> [... snip ...]
> > diff --git a/builtin/fetch.c b/builtin/fetch.c
> > index 0f23dd4b8c..d3f922fc89 100644
> > --- a/builtin/fetch.c
> > +++ b/builtin/fetch.c
> > @@ -958,8 +958,10 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
> >                               ref->force = rm->peer_ref->force;
> >                       }
> >
> > -                     if (recurse_submodules != RECURSE_SUBMODULES_OFF)
> > +                     if (recurse_submodules != RECURSE_SUBMODULES_OFF &&
> > +                         (!rm->peer_ref || !oideq(&ref->old_oid, &ref->new_oid))) {
> >                               check_for_new_submodule_commits(&rm->old_oid);
> > +                     }
>
> The original before be76c212 fed ref->new_oid to the check
> function.  Now that we are using ref->{old,new}_oid in the
> condition, would it make more sense to pass ref->new_oid
> like we did before the commit, or is that an object that is
> different from rm->old_oid?

I think that was the whole point of this commit, to cover the case
of !rm->peer_ref, for newly fetched refs. On this case, ref is NULL.

> Thanks.
>
> >                       if (!strcmp(rm->name, "HEAD")) {
> >                               kind = "";
> >
> > base-commit: e19713638985533ce461db072b49112da5bd2042

- Orgad