mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Johannes Schindelin <>
Subject: [Summit topic] Submodules and how to make them worth using
Date: Thu, 21 Oct 2021 13:56:13 +0200 (CEST)	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

[-- Attachment #1: Type: text/plain, Size: 4921 bytes --]

This session was led by Emily Shaffer. Supporting cast: brian m. carlson,
Orgad Shaneh, Jonathan "jrnieder" Nieder, Jeff Hostetler, and Philip



    1. Internally at Google, a lot of use of “repo”

    2. Isn’t great, but not much alternative available

    3. Submodules are also not great, let’s make them better

    4. Some prior work: --recurse-submodules options

       1. I can run “git branch” with and without --recurse-submodules

    5. Being in recurse mode gives us a chance to be opinionated

    6. Don’t want to have a million options and create a lot of complexity

    7. Branches

       1. Superproject thinks “main” points to one set of states in submodules

       2. Submodules have “main” pointing elsewhere

       3. Which is right? The superproject is right, “git status” can show the

    8. Not trying to eliminate all complexity. There is some inherent
       complexity in stitching repositories together. But I want to make it

    9. For specifics, see the RFC linked to above

 2. brian: Interested in current status, what’s been implemented

    1. Emily: workflow git clone / git branch / git commit / git push, all
       using submodule.recurse, worked well

    2. Intern Mahi Kolla sent a patch to recurse by default once you’ve done a
       --recurse-submodules clone

    3. Ran demo for an internal team, feedback was positive

    4. Used a hacky remote helper to map “git push” to “git push origin
       HEAD:refs/for/main”, we have plans for not needing that :)

    5. Partial clone with submodules is close to done, is another important
       part of this

    6. Glen and Josh have done some work on branching + setting tracking info.
       That’s key for making recursive push work in an intuitive way, because
       the branch you want to push to in each submodule is not always the same

    7. I also pushed a series storing a path in each submodule’s git directory
       to its superproject’s git directory. Use that as another phase in config
       parsing, inherited-from-superproject config. That combines well with
       config-based hooks (thanks Ævar for the help with that)

    8. Next steps are around fast-forward merges and rebases

    9. Specifics are in the doc linked to

 3. Interaction with Gerrit

    1. Orgad: when you push to a submodule and superproject, at merge time the
       submodule commit changes, what do you do in the superproject to handle

    2. jrnieder: This comes up in any review flow, not just Gerrit --- ideally
       you’d want to review the superproject and submodule changes together as
       one unit. There’s some work happening in Gerrit on “multi-change

    3. What works today: Gerrit’s submodule subscription feature has the
       ability to update a superproject. If you have a set of submodule changes
       and a superproject change that are submitted together, then at submit
       time Gerrit will rewrite the superproject change to reflect what
       happened in the submodules.

    4. In the Android workflow the superproject only contains pointers to
       submodules so we don’t push changes for review to the superproject at
       all. So we handle this with submodule subscription.

    5. Emily: analogy to auto-generated merge commits

 4. Jeff Hostetler: back in 2014 Microsoft considered submodules, hit a can of

    1. Coordinating changes between submodule and superproject, this requires
       server-side locks to prevent edge cases

    2. Was hard enough that we abandoned it

    3. jrnieder: we’re viewing submodules as not a replacement for the
       monorepo, but as a separate thing for when components have an
       independent existence. Microsoft made the right choice by not using
       submodules artificially in the creation of the Windows monorepo.

 5. Jeff: do you want to support sub-sub-sub-submodules?

    1. Emily: we ruled that out.

    2. jrnieder: nested submodules already work well in Git, we’re not breaking

       1. Philip Oakley: good; if that changes, please make docs + config clear
          about it

    3. As a matter of project hygiene, we encourage people to put their
       submodules in the top-level directly. That way, you know what code
       you’re pulling in.

    4. That said, there are unusual use cases e.g. around a build that pulls
       together multiple versions of the full Android codebase. So we actually
       do take advantage of nested submodules for those niche cases

 6. Please read the design doc, and expect lotsa patches over the next 3-6

  parent reply	other threads:[~2021-10-21 11:56 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-21 11:55 Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20 Johannes Schindelin
2021-10-21 11:55 ` [Summit topic] Crazy (and not so crazy) ideas Johannes Schindelin
2021-10-21 12:30   ` Son Luong Ngoc
2021-10-26 20:14   ` scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas] Eric Wong
2021-10-30 19:58     ` Ævar Arnfjörð Bjarmason
2021-11-03  9:24       ` test suite speedups via some not-so-crazy ideas (was: scripting speedups[...]) Ævar Arnfjörð Bjarmason
2021-11-03 22:12         ` test suite speedups via some not-so-crazy ideas Junio C Hamano
2021-11-02 13:52     ` scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas] Johannes Schindelin
2021-10-21 11:55 ` [Summit topic] SHA-256 Updates Johannes Schindelin
2021-10-21 11:56 ` [Summit topic] Server-side merge/rebase: needs and wants? Johannes Schindelin
2021-10-22  3:06   ` Bagas Sanjaya
2021-10-22 10:01     ` Johannes Schindelin
2021-10-23 20:52       ` Ævar Arnfjörð Bjarmason
2021-11-08 18:21   ` Taylor Blau
2021-11-09  2:15     ` Ævar Arnfjörð Bjarmason
2021-11-30 10:06       ` Christian Couder
2021-10-21 11:56 ` Johannes Schindelin [this message]
2021-10-21 11:56 ` [Summit topic] Sparse checkout behavior and plans Johannes Schindelin
2021-10-21 11:56 ` [Summit topic] The state of getting a reftable backend working in git.git Johannes Schindelin
2021-10-25 19:00   ` Han-Wen Nienhuys
2021-10-25 22:09     ` Ævar Arnfjörð Bjarmason
2021-10-26  8:12       ` Han-Wen Nienhuys
2021-10-28 14:17         ` Philip Oakley
2021-10-26 15:51       ` Philip Oakley
2021-10-21 11:56 ` [Summit topic] Documentation (translations, FAQ updates, new user-focused, general improvements, etc.) Johannes Schindelin
2021-10-22 14:20   ` Jean-Noël Avila
2021-10-22 14:31     ` Ævar Arnfjörð Bjarmason
2021-10-27  7:02       ` Jean-Noël Avila
2021-10-27  8:50       ` Jeff King
2021-10-21 11:56 ` [Summit topic] Increasing diversity & inclusion (transition to `main`, etc) Johannes Schindelin
2021-10-21 12:55   ` Son Luong Ngoc
2021-10-22 10:02     ` vale check, was " Johannes Schindelin
2021-10-22 10:03       ` Johannes Schindelin
2021-10-21 11:57 ` [Summit topic] Improving Git UX Johannes Schindelin
2021-10-21 16:45   ` changing the experimental 'git switch' (was: [Summit topic] Improving Git UX) Ævar Arnfjörð Bjarmason
2021-10-21 23:03     ` changing the experimental 'git switch' Junio C Hamano
2021-10-22  3:33     ` changing the experimental 'git switch' (was: [Summit topic] Improving Git UX) Bagas Sanjaya
2021-10-22 14:04     ` martin
2021-10-22 14:24       ` Ævar Arnfjörð Bjarmason
2021-10-22 15:30         ` martin
2021-10-23  8:27           ` changing the experimental 'git switch' Sergey Organov
2021-10-22 21:54         ` Sergey Organov
2021-10-24  6:54       ` changing the experimental 'git switch' (was: [Summit topic] Improving Git UX) Martin
2021-10-24 20:27         ` changing the experimental 'git switch' Junio C Hamano
2021-10-25 12:48           ` Ævar Arnfjörð Bjarmason
2021-10-25 17:06             ` Junio C Hamano
2021-10-25 16:44     ` Sergey Organov
2021-10-25 22:23       ` Ævar Arnfjörð Bjarmason
2021-10-27 18:54         ` Sergey Organov
2021-10-21 11:57 ` [Summit topic] Improving reviewer quality of life (patchwork, subsystem lists?, etc) Johannes Schindelin
2021-10-21 13:41   ` Konstantin Ryabitsev
2021-10-22 22:06     ` Ævar Arnfjörð Bjarmason
2021-10-22  8:02 ` Missing notes, was Re: Notes from the Git Contributors' Summit 2021, virtual, Oct 19/20 Johannes Schindelin
2021-10-22  8:22   ` Johannes Schindelin
2021-10-22  8:30     ` Johannes Schindelin
2021-10-22  9:07       ` Johannes Schindelin
2021-10-22  9:44 ` Let's have public Git chalk talks, " Johannes Schindelin
2021-10-25 12:58   ` Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).