Re: Bugreport - submodules are fetched twice in some cases

[Junio C Hamano <gitster@pobox.com>]

Glen Choo <chooglen@google.com> writes:

> And obviously, we aren't passing "--recurse-submodules=false", so there's
> good reason to believe that "--all" will fetch submodules R + 1 times.

Good find.

Given your recent work on enumerating the commits in the submodule
repository that are needed to complement "git fetch" made in the
superproject, the above finding raises an interesting question.

Imagine that we have two remotes for the current repository, and
this superproject uses one submodule.

When we run "git fetch --all --resurse-submodules", from one remote,
we may grab a range of history in the superproject that mentions
submodule commits C1 and C2 that are not in our clone of the
submodule, while the other remote gives a different range of history
in the superproject that mentions submodule commit C3 that we do not
have.

What should happen in our submodule?  In other words, how do we make
sure that we grab C1, C2 and C3?

Ideally, we probably would want to run a non-recursive fetch of the
superproject twice (i.e. once for each of the two remotes we have),
then traverse the superproject history to find that these three
commits are needed in the submodule, and run a single (possibly
recursive) fetch in the submodule and ask for C1, C2 and C3.  But I
am not sure if we are set up to do so.  Does the "parent" process
take a snapshot of our refs before spawning the two "child" fetches
for each remote when handling "fetch --all", so that we can later
figure out what superproject commits were obtained during the
fetches from these two remotes?  Without that information, we cannot
find out that C1, C2 and C3 are new in the submodule, so we cannot
implement the "fetch without recursion from each remote and then do
a single fetch in submodule to grab everything we need at once"
approach.

Provided if we have the "make sure everything needed in the
submodule is fetched by inspecting the range of commits we fetch for
a superproject" working correctly for a single remote, an
alternative approach is to run "git fetch --recurse-submodules" for
each remote separately, without the "parent" process doing anything
in the submodule (i.e. you earlier counted R+1 fetches, but instead,
we make R fetches in the submodule.  It is less than ideal but it
may be easier to implement).

Thoughts?