[PATCH 0/8] fetch --recurse-submodules: fetch unpopulated submodules

[Glen Choo <chooglen@google.com>]

= Background

When fetching submodule commits, "git fetch --recurse-submodules" only
considers populated submodules, and not all of the submodules in
$GIT_DIR/modules as one might expect. As a result, "git fetch
--recurse-submodules" behaves differently based on which commit is
checked out.

This can be a problem, for instance, if the user has a branch with
submodules and a branch without:

  # the submodules were initialized at some point in history..
  git checkout -b branch-with-submodules origin/branch-with-submodules
  git submodule update --init

  # later down the road..
  git checkout --recurse-submodules branch-without-submodules
  # no submodules are fetched!
  git fetch --recurse-submodules
  # if origin/branch-with-submodules has new submodule commits, this
  # checkout will fail because we never fetched the submodule
  git checkout --recurse-submodules branch-with-submodules

This series makes "git fetch" fetch the right submodules regardless of
which commit is checked out, as long as the submodule has already been
cloned. In particular, "git fetch" learns to:

1. read submodules from the relevant superproject commit instead of
   the file system
2. fetch all changed submodules, even if they are not populated

= Patch organization

- Patches 1-3 teach "git fetch" to read .gitmodules from the
  superproject commit.
- Patches 4-5 are quality-of-life improvements to the test suite that
  make it easier to write the tests in patch 7.
- Patch 6 separates the steps of "finding which submodules to fetch" and
  "fetching the submodules", making it easier to tell "git fetch" to
  fetch unpopulated submodules.
- Patch 7 teaches "git fetch" to fetch changed, unpopulated submodules
  in addition to populated submodules.
- Patch 8 is an optional bugfix + cleanup of the "git fetch" code that
  removes the last caller of the deprecated "add_submodule_odb()".

= Future work

Even with this series, there is no guarantee that "git fetch" will fetch
every necessary submodule commit because a superproject commit can
introduce new submodules, and since those submodules are not cloned, "git
fetch" cannot fetch those commits yet. This series should get us closer
to that goal because "git fetch" can read submodules from the
superproject commit, which is a necessary precursor to figuring out
whether to clone submodules from superproject commits.

Glen Choo (8):
  submodule: inline submodule_commits() into caller
  submodule: store new submodule commits oid_array in a struct
  submodule: make static functions read submodules from commits
  t5526: introduce test helper to assert on fetches
  t5526: use grep to assert on fetches
  submodule: extract get_fetch_task()
  fetch: fetch unpopulated, changed submodules
  submodule: fix bug and remove add_submodule_odb()

 Documentation/fetch-options.txt |  26 ++-
 Documentation/git-fetch.txt     |  10 +-
 submodule.c                     | 316 ++++++++++++++++----------
 submodule.h                     |   9 +-
 t/t5526-fetch-submodules.sh     | 386 ++++++++++++++++++++++++--------
 5 files changed, 524 insertions(+), 223 deletions(-)


base-commit: 679e3693aba0c17af60c031f7eef68f2296b8dad
-- 
2.33.GIT