git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Emily Shaffer <emilyshaffer@google.com>
Cc: git@vger.kernel.org
Subject: Future structure of submodules and .git/, .git/modules/* organization
Date: Wed, 14 Apr 2021 12:32:17 +0200	[thread overview]
Message-ID: <87v98p17im.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <20210408233936.533342-1-emilyshaffer@google.com>


On Fri, Apr 09 2021, Emily Shaffer wrote:

> I'm hoping to work on some other submodule-centric stuff over the coming
> months, and it might end up being very useful to be able to tell "am I a
> submodule?" and "how do I talk to my superproject?" more generally - so
> I'm really open to figuring out a better way than this, if folks have
> ideas.
>
> Patch 1 is a small refactor that we can take or leave - I found
> "SCOPE_SUBMODULE" to be pretty ambiguous, especially since it seems to
> refer to configs from .gitmodules. Even though I decided that
> "superproject" was a better name than "submodule" I still wasn't super
> happy with the ambiguity. But we can drop it if folks don't want to
> rename.

This is less on your patch, and more on the larger work you're
suggesting, but the two are kind of related. Skip to the paragraph
starting with "But why" below for the relevance :)

I very much wish that we could eventually make the use of submodules
totally transparent, i.e. (taking the example of git.git):

 * You clone, and we just get objects from
   https://github.com/cr-marcstevens/sha1collisiondetection.git too

 * The fact that we have:

   160000 commit 855827c583bc30645ba427885caa40c5b81764d2  sha1collisiondetection

   Would become totally invisible to most users unless they run some
   gutsy ls-tree/files comand.

   We used to have a full git dir at sha1collisiondetection/.git and all
   the UX issues that entailed (e.g. switching to an old commit without
   the submodule).

   Now it's a stub and the actual repo is at
   .git/modules/sha1collisiondetection/, so we're kind of partially
   there.

 * I would think that the next (but big) logical step would be to use
   some combination of delta islands, upcoming sparse indexes etc. to
   actually share the object stores of the parent and submodule.

   Things like "git fsck" which now just punt on COMMIT would need to
   become smarter, but e.g. we could repack (or not, with islands)
   between parent and submodule.

I would think that this end goal makes more sense than the current
status quo of teaching every command that needs to e.g. grep the tree to
have a "--recurse-submodules". The distinction would be invisible to the
likes of "git-grep".

It would mean more complexity in e.g. "git commit", but we can imagine
if you wanted a cross-submodule commit it could do those commits
recursively, update parent COMMIT entries etc. (and even, optionally,
push out the submodule changes). That particular thing being so ad-hoc
is a *very* frequent pain point in submodule use.

But why am I talking about this here when all you're suggesting is
another config level?

Well, I think (but have not carefully thought about) that this
CONFIG_SCOPE_GITMODULES is probably a narrow net improvement now. If you
set most options in your .git/config to you that's the same logical
project, why shouldn't you get your diff setting or whatever because you
cd'd to a submodule "in the same project" (from the view of the user).

But I think that for a wider "improve submodules" effort it's worth
someone (and right now, that sounds like it's you) thinking about where
we're going with the feature. Maybe with some technical doc identifying
the most common pain points, what we propose (or could envision) doing
about them.

So e.g. in this case, having per-submodule config could be a step
forward, but it could also be one more step of walking in a circle.

I.e. don't think any user asked for or wanted to stitch together
multiple .git directories into one linked pseudo-checkout, that's
ultimately something we're exposing as an implementation detail. If we
no longer expose that implementation detail, would we be stuck
supporting what's ultimately a workaround feature?

None of that means we shouldn't have that one step forward that solves
real problems today.

But I think we should think about the end goal(s) sooner than
later. E.g. in your case, do you *really* want another config level, or
is it just the easiest way to get what you actually want, which is for a
"git config" in the submodule dir to perhaps consider its .git/config
and .git/modules/sha1collisiondetection/config as the same file for the
purposes of config parsing? Sans things like the remote URLs etc.

  parent reply	other threads:[~2021-04-14 10:32 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-08 23:39 [RFC PATCH 0/2] share a config between submodule and superproject Emily Shaffer
2021-04-08 23:39 ` [RFC PATCH 1/2] config: rename "submodule" scope to "gitmodules" Emily Shaffer
2021-04-08 23:39 ` [RFC PATCH 2/2] config: add 'config.superproject' file Emily Shaffer
2021-04-09 11:10   ` Philip Oakley
2021-04-13 18:05     ` Emily Shaffer
2021-04-09 14:35   ` Matheus Tavares Bernardino
2021-04-09 22:29     ` Junio C Hamano
2021-04-13 19:45       ` Emily Shaffer
2021-04-13 18:48     ` Emily Shaffer
2021-04-14 10:32 ` Ævar Arnfjörð Bjarmason [this message]
2021-04-15 21:25   ` Future structure of submodules and .git/, .git/modules/* organization Emily Shaffer
2021-04-15 21:41   ` Junio C Hamano
2021-04-23  0:15 ` [RFC PATCH v2 0/4] share a config between submodule and superproject Emily Shaffer
2021-04-23  0:15   ` [RFC PATCH v2 1/4] config: rename "submodule" scope to "gitmodules" Emily Shaffer
2021-04-23  9:46     ` Phillip Wood
2021-04-23  0:15   ` [RFC PATCH v2 2/4] t1510-repo-setup: don't use exact matching on traces Emily Shaffer
2021-04-23  9:59     ` Phillip Wood
2021-04-23  0:15   ` [RFC PATCH v2 3/4] t7006-pager.sh: more lenient trace checking Emily Shaffer
2021-04-23  9:54     ` Phillip Wood
2021-04-23 12:45       ` Phillip Wood
2021-04-23  0:15   ` [RFC PATCH v2 4/4] config: add 'config.superproject' file Emily Shaffer
2021-04-23 12:08     ` Johannes Schindelin
2021-06-19  0:31   ` [PATCH v3 0/2] share a config between submodule and superproject Emily Shaffer
2021-06-19  0:31     ` [PATCH v3 1/2] config: rename "submodule" scope to "gitmodules" Emily Shaffer
2021-06-19  0:31     ` [PATCH v3 2/2] config: add 'config.superproject' file Emily Shaffer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87v98p17im.fsf@evledraar.gmail.com \
    --to=avarab@gmail.com \
    --cc=emilyshaffer@google.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).