git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Antonio Ospite <ao2@ao2.it>
To: git@vger.kernel.org
Cc: Brandon Williams <bmwill@google.com>,
	Daniel Graña <dangra@gmail.com>,
	Jonathan Nieder <jrnieder@gmail.com>,
	Richard Hartmann <richih.mailinglist@gmail.com>,
	Stefan Beller <sbeller@google.com>, Antonio Ospite <ao2@ao2.it>
Subject: [RFC PATCH 00/10] Make submodules work if .gitmodules is not checked out
Date: Mon, 14 May 2018 12:58:13 +0200
Message-ID: <20180514105823.8378-1-ao2@ao2.it> (raw)

Hi,

vcsh[1] uses bare git repositories and detached work-trees to manage
*distinct* sets of configuration files directly into $HOME.

In this setup multiple repositories share the same directory (namely
$HOME) as their work dir, so the sets of checked out files would also
need to be *disjoint* to avoid collisions (examples of collisions can be
README or LICENSE files, or even .gitignore and .gitattributes).

Amongst vcsh users, a popular solution to this problem is to use sparse
checkouts, representing the *intersection* of all the repositories in
a common sparse-checkout file[2].

This works well but there are still limitations about the ability to use
submodules because git expects the .gitmodules file to be checked out.

The user (or vcsh itself) might learn to fully populate one repository
at a time when working with submodules but this is unhandy and would
introduce serialization even when it's not strictly needed like in the
case of _reading_ .gitmodules.

As a side note, git submodules have worked perfectly fine with detached
work-trees for some time[3,4,5] so extending them to also play nice with
sparse checkouts seems the next logical step to cover the vcsh use case.

This series teaches git to try and read the .gitmodules file from the
index (HEAD:.gitmodules) when it's not available in the work dir.

It does so by first providing an opaque way to access the submodules
configuration, and then extends the access mechanism behind the scenes.

Writing to .gitmodules still requires it to be checked out.

This series should be in line with what Stefan and Jonathan proposed;
although it's not perfect yet:

  - naming of functions can be improved,
  - code can be moved around to better places,
  - maybe some notes should be added to Documentation/git-submodule.txt,
  - my git terminology may still be a little off: do "work tree" and
    "work directory" mean the same thing?

the functionality is there and we should have a decent baseline to work
on.

The patchset is based on the current master (ccdcbd54c447), the
test-suite passes after each commit and there are some per-patch
annotations.

If anyone wanted to pick up and finish the work feel free to do so,
otherwise please comment and I'll try to address issues as time permits.

Thanks,
   Antonio

[1] https://github.com/RichiH/vcsh
[2] https://github.com/RichiH/vcsh/issues/120#issuecomment-387335765
[3] http://git.661346.n2.nabble.com/git-submodule-vs-GIT-WORK-TREE-td7562165.html
[4] http://git.661346.n2.nabble.com/PATCH-Solve-git-submodule-issues-with-detached-work-trees-td7563377.html
[5] https://github.com/git/git/commit/be8779f7ac9a3be9aa783df008d59082f4054f67

Antonio Ospite (10):
  config: make config_from_gitmodules generally useful
  submodule: factor out a config_gitmodules_set function
  t7411: be nicer to other tests and really clean things up
  submodule--helper: add a new 'config' subcommand
  submodule: use the 'submodule--helper config' command
  submodule--helper: add a '--stage' option to the 'config' sub command
  submodule: use 'submodule--helper config --stage' to stage .gitmodules
  t7506: cleanup .gitmodules properly before setting up new scenario
  submodule: support reading .gitmodules even when it's not checked out
  t7415: add new test about using HEAD:.gitmodules from the index

 builtin/fetch.c                        |   2 +-
 builtin/mv.c                           |   2 +
 builtin/rm.c                           |   7 +-
 builtin/submodule--helper.c            | 100 +++++++++++++++++++-
 cache.h                                |   1 +
 config.c                               |  26 ++++--
 config.h                               |  10 +-
 git-submodule.sh                       |  10 +-
 submodule-config.c                     |  16 +---
 submodule.c                            |  37 ++++++--
 submodule.h                            |   2 +
 t/t7411-submodule-config.sh            |  63 ++++++++++++-
 t/t7415-submodule-sparse-gitmodules.sh | 124 +++++++++++++++++++++++++
 t/t7506-status-submodule.sh            |   3 +-
 14 files changed, 357 insertions(+), 46 deletions(-)
 create mode 100755 t/t7415-submodule-sparse-gitmodules.sh

-- 
Antonio Ospite
https://ao2.it
https://twitter.com/ao2it

A: Because it messes up the order in which people normally read text.
   See http://en.wikipedia.org/wiki/Posting_style
Q: Why is top-posting such a bad thing?

             reply index

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-14 10:58 Antonio Ospite [this message]
2018-05-14 10:58 ` [RFC PATCH 01/10] config: make config_from_gitmodules generally useful Antonio Ospite
2018-05-14 18:19   ` Brandon Williams
2018-06-20 18:04     ` Antonio Ospite
2018-05-15  1:05   ` Stefan Beller
2018-06-20 18:06     ` Antonio Ospite
2018-06-20 19:10       ` Stefan Beller
2018-06-21 13:54         ` Antonio Ospite
2018-06-21 18:53           ` Stefan Beller
2018-05-14 10:58 ` [RFC PATCH 02/10] submodule: factor out a config_gitmodules_set function Antonio Ospite
2018-05-15  1:20   ` Stefan Beller
2018-06-20 18:41     ` Antonio Ospite
2018-05-14 10:58 ` [RFC PATCH 03/10] t7411: be nicer to other tests and really clean things up Antonio Ospite
2018-05-15  1:23   ` Stefan Beller
2018-06-20 21:16     ` Antonio Ospite
2018-05-14 10:58 ` [RFC PATCH 04/10] submodule--helper: add a new 'config' subcommand Antonio Ospite
2018-05-15  1:33   ` Stefan Beller
2018-06-20 21:32     ` Antonio Ospite
2018-05-14 10:58 ` [RFC PATCH 05/10] submodule: use the 'submodule--helper config' command Antonio Ospite
2018-05-14 10:58 ` [RFC PATCH 06/10] submodule--helper: add a '--stage' option to the 'config' sub command Antonio Ospite
2018-05-14 10:58 ` [RFC PATCH 07/10] submodule: use 'submodule--helper config --stage' to stage .gitmodules Antonio Ospite
2018-05-14 10:58 ` [RFC PATCH 08/10] t7506: cleanup .gitmodules properly before setting up new scenario Antonio Ospite
2018-05-14 10:58 ` [RFC PATCH 09/10] submodule: support reading .gitmodules even when it's not checked out Antonio Ospite
2018-05-15  1:45   ` Stefan Beller
2018-05-14 10:58 ` [RFC PATCH 10/10] t7415: add new test about using HEAD:.gitmodules from the index Antonio Ospite
2018-05-15  1:14 ` [RFC PATCH 00/10] Make submodules work if .gitmodules is not checked out Stefan Beller
2018-05-15  4:09 ` Junio C Hamano

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180514105823.8378-1-ao2@ao2.it \
    --to=ao2@ao2.it \
    --cc=bmwill@google.com \
    --cc=dangra@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=jrnieder@gmail.com \
    --cc=richih.mailinglist@gmail.com \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox