Re: Bugreport - submodules are fetched twice in some cases

[Junio C Hamano <gitster@pobox.com>]

Glen Choo <chooglen@google.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> A simple and concrete reproduction 
>>
>>     git init top
>>     cd top
>>     date >file1
>>     git init sub
>>     cd sub
>>     date >subfile1
>>     git add .
>>     git commit -m subinitial
>>     cd .. ;# back to top
>>     git submodule add ./sub sub
>>     git add file1
>>     git commit -m initial
>>     cd .. ;# out of top
>>     git clone --recurse-submodules top copy
>>     cd copy
>>     git config submodule.recurse true
>>     git config fetch.parallel 0
>>     GIT_TRACE2=$(pwd)/trace git fetch --all --prune --prune-tags
>>
>> This throws the three lines to the output.
>>
>> Fetching origin
>> Fetching submodule sub
>> Fetching submodule sub
>>
>> The two "Fetching submodule" messages are coming from two separate
>> calls to get_fetch_task_from_index(), and the trace does show that
>> the code is doing "git-upload-pack" three times (one for the top
>> level, twice for the same top/sub).  We can see it by grepping
>> for "git-upload-pack" in the resulting 'trace' file above.
>
>  
> Thanks for the reproduction recipe and findings, that'll be very helpful
> :)
>
>> Glen, as submodule.c::fetch_submodules() was created in your heavy
>> refactoring quite recently, I thought I'd redirect this report in
>> your direction, as I expect you'd be the most clueful in this area
>> ;-)
>
> Hm, this does look like something that I probably introduced. But even
> if it turns out to be older than that, I think I am the right person to
> fix it, yes.

It seems that ever since the introduction of the --prune-tags option
at v2.16.1-16-g97716d217c (fetch: add a --prune-tags option and
fetch.pruneTags config, 2018-02-09), we always behaved this way.

Without "--prune-tags" (but still with "--prune"), we can go even
older than that version, and v2.10.0 seems to fetch only once.

And the command keeps working that way all the way back to the
commit that starts honoring submodule.recurse configuration, at
v2.13.0-137-g58f4203e7d (builtin/fetch.c: respect
'submodule.recurse' option, 2017-05-31)

If we instead use "git fetch --recurse-submodules" with versions
of GIt older than that, we can go even older.  I saw v2.5.0 behaves
that way before I got tired and gave up.

So, we still would want to eventually get to it, but no rush.  This
is an old thing and not as urgent as fixing a recent regression.

FWIW, without "--all", we do not make an extra fetch at all, with
the current code or with code as ancient as v2.5.0

Thanks.