Re: Fetch on submodule update

[Robert Dailey <rcdailey.lists@gmail.com>]

On Thu, Aug 2, 2018 at 1:08 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> I think I misread this the first time.  I got distracted by your
> mention of the --remote option, but you mentioned you want to use the
> SHA-1 of the submodule listed, so that was silly of me.
>
> I think you'll find that "git fetch --no-recurse-submodules" and "git
> submodule update" do exactly what you want.  "git submodule update"
> does perform a fetch (unless you pass --no-fetch).
>
> Let me know how it goes. :)
>
> I'd still be interested in hearing more about the nature of the
> submodules involved --- maybe `submodule.fetchJobs` would help, or
> maybe this is a workflow where a tool that transparently fetches
> submodules on demand like
> https://gerrit.googlesource.com/gitfs/+/master/docs/design.md would be
> useful (I'm not recommending using slothfs for this today, since it's
> read-only, but it illustrates the idea).

Hi thanks for your response, sorry I am a bit late getting back with you.

Maybe my workflow is dated, because I'm still used to treating
submodules as distinctly separated and independent things. I realize
submodule recursion is becoming more inherent in many high level git
commands, but outside of git there are separation issues that make
this workflow doomed to be non-seamless. For example, pull requests
will never offer the same uniformity: You will still have 1 pull
request per submodule. There's also the issue of log audits: You
cannot use blame, log, bisect, or other "diagnostic" commands to
introspect into submodules "as if" they were subtree or something of
the like (i.e. truly part of the DAG). A more realistic example of one
of the common questions I still can't answer easily is: "How do you
determine which commit in a submodule made it into which release of
the software?" In the case where the parent repository has the
annotated tags (representing software release milestones), and the
submodule is just a common library (which does not have those tags and
has no release cycle). Anyway, none of these issues are particularly
related but they do contribute to the answer to your question
regarding my workflow and use cases. The list goes on but I hope you
get the idea.

Some of the more functional issues are performance related: I am aware
enough, at times, that I can save time (in both local operations and
network overhead) by skipping submodules. For example, if I know that
I'm merging mainline branches, I do not need to mess with the
submodules (I can fetch, merge, commit, push from the parent repo
without messing with the submodules. This saves me time). If
`fetchJobs` was also `updateJobs`, i.e. you could update submodules in
parallel too, that might make this less of an issue. Think of
repositories [like boost][1] that have (I think) over a hundred
sibling submodules: Fetching 8 in parallel *and* doing `submodule
update` in parallel 8 times might also speed things up. There's also
`git status`, that if it recurses into submodules, is also
significantly slow in the boost case (I'm not sure if it is
parallelized).

Again, none of this is particularly related, but just to give you more
context on the "why" for my ask. Sorry if I'm dragging this out too
far.

The TLDR is that I do prefer the manual control. Automatic would be
great if submodules were treated as integrated in a similar manner to
subtree, but it's not there. I wasn't aware that `submodule update`
did a fetch, because sometimes if I do that, I get errors saying SHA1
is not present (because the submodule did not get fetched). Granted I
haven't seen this in a while, so maybe the fetch on submodule update
is a newer feature. Do you know what triggers the fetch on update
without --remote? Is it the missing SHA1 that triggers it, or is it
fetching unconditionally?

Thanks for confirming it behaves as I already wanted. And as you can
tell, I'm also happy to further discuss motivation / use cases /
details related to overall usage of submodules if you'd like. I'm
happy to help however I can!

[1]: https://github.com/boostorg/boost