git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: Stefan Beller <sbeller@google.com>
Cc: jonathantanmy@google.com, git@vger.kernel.org
Subject: Re: [RFD] Long term plan with submodule refs?
Date: Thu, 09 Nov 2017 14:08:51 +0900
Message-ID: <xmqqbmkcjaxo.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <20171109001007.11894-1-sbeller@google.com>

Stefan Beller <sbeller@google.com> writes:

>> The relationship is indeed currently useful, but if the long term plan
>> is to strongly discourage detached submodule HEAD, then I would think
>> that these patches are in the wrong direction. (If the long term plan is
>> to end up supporting both detached and linked submodule HEAD, then these
>> patches are fine, of course.) So I think that the plan referenced in
>> Junio's email (that you linked above) still needs to be discussed.
>
> This email presents different approaches.
>
> Objective
> =========
> This document should summarize the current situation of Git submodules
> and start a discussion of where it can be headed long term.
> Show different ways in which submodule refs could evolve.
>
> Background
> ==========
> Submodules in Git are considered as an independet repository currently.
> This is okay for current workflows, such as utilizing a library that is
> rarely updated. Other workflows that require a tighter integration between
> submodule and superproject are possible, but cumbersome as there is an
> additional step that has to be performed, which is the update of the gitlink
> pointer in the superproject.

I do not think "rarely updaed" is an issue.

The problem is that we may want to make it easier to use a
superproject and its submodules as if the combined whole were a
single project, which currently is not easy, primarily because
submodules are separate entities with different set of branches that
can be checked out independently from what branch the superproject
is working on.

> Workflows
> =========
> * Obtaining a copy of the Superproject tightly coupled with submodules
>   solved via git clone --recurse-submodules=<pathspec>
> * Changing the submodule selection
>   solved via submodule.active flags
> * Changing the remote / Interacting with a different remote for all submodules
>   -> need to be solved, not core issue of this discussion
> * Syncing to the latest upstream
>   solved via git pull --recurse  
> * Working on a local feature in one submodule
>   -> How do refs work spanning superproject/submodule?
> * Working on a feature spanning multiple submodules
>   -> How do refs work spanning multiple repos?
> * Working on a bug fix (Changing the feature that you currently work on, branches)
>   -> How does switching branches in the superproject affect submodules

These are good starting points for copying such a combined whole to
your local machine and start working on it.  The more interesting,
important, and potentially difficult part is how the result of such
work is shared back to where you started from.  "push --recursive"
may be a simple phrase, but a sensible definition of how it should
work won't be that simple.

> Possible data models and workflow implications
> ==============================================
> In the following different data models are presented, which aid a submodule
> heavy workflow each giving pros and cons.
>
> Keep everything as is, superproject and submodule have their own refs
> ---------------------------------------------------------------------
> ...
> Cons:
>  * Current tools that manage multiple repositories (e.g. repo, git-slave)
>    have "branches in parallel", i.e. each repo has a branch of the same
>    name, instead of using a superproject to manage the state of all repos
>    involved. So users of such tools may be confused by submodules.
>  * when using a detached HEAD in the submodule, we may run into git-gc issues.

We should make detached HEAD safe against gc if it is not,
regardless of the use of submodules.  I thought it already was made
safe long time ago.

> Use replicate refs in submodules
> --------------------------------
> This approach will replicate the superproject refs into the submodule
> ref namespace, e.g. git-branch learns about --recurse-submodules, which
> creates a branch of a given name in all submodules. These (topic) branches
> should be kept in sync with the superproject
>
> Pros:
>  * This seemed intuitive to Gerrit users
>  * 'quick' to implement, most of the commands are already there,
>    just git-branch is needed to have the workflows mentioned above complete.
> Cons:
>  * What does "git checkout -b A B" mean? (special case: B == HEAD)

The command ran at which level?  In the superproject, or in a single
submodule?

>    Is the branch name replicated as a string into the submodule operation,
>    or do we dereference the superprojects gitlink and walk from there?

If they are "kept in sync with the superproject", then there should
be no difference between the two, so I do not see any room for
wondering about that.  In other words, if there is need to worry
about the differences between the above two, then it probably is
fundamentally impossible to keep these in sync, and a design that
assumes it is possible would have to expose glitches to the end-user
experience.

I do not know if glitches resulting from there would be so severe to
be show-stoppers, though.  It might be possible to paper them over.

> No submodule refstore at all
> ----------------------------
> Use refs and commits in the superproject to stitch submodule changes
> together. Disallow branches in the submodule. This is only restricted
> to the working tree inside the superproject, such that the output of git-branch
> changes depending whether the working tree is in- or outside the superproject
> working tree.

This would need enhancement for reachability code, but it feels the
cleanest from the philosophical standpoint---if you want to treat a
superproject and its submodules as if it were a single project,
ability to check out a branch in a submodule that does not match
that of the superproject would only get in the way of preserving the
illusion of "single project"-ness.

> New type of symbolic refs
> =========================
> A symbolic ref can currently only point at a ref or another symbolic ref.
> This proposal showcases different scenarios on how this could change in the
> future.
>
> HEAD pointing at the superprojects index
> ----------------------------------------

This looks to me a mere implementation detail for a (part of)
necessary component to realize the above "No submodule refstore".

> Superproject operations spanning index and worktree
>   E.g. git reset --mixed
> As the submodules HEAD is defined in the index, we would reset it to the
> version in the last commit. As --mixed promises to not touch the working tree,
> the submodules worktree would not be touched. git reset --mixed in the
> superproject is the same as --soft in the submodule.

I am not sure if you want to take these promises low-level "single
repository" plumbing operations make too literally.  "reset --mixed"
may promise not to touch the working tree, but it also promises not
to touch submodules at all.  If you are breaking the latter anyway,
it would make more sense not to be afraid of breaking the former if
it makes sense in the context of allowing the command to do more by
breaking the latter.


  parent reply index

Thread overview: 16+ messages in thread (expand / mbox.gz / Atom feed / [top])
2017-11-08 19:55 [RFC PATCH 0/4] git-status reports relation to superproject Stefan Beller
2017-11-08 19:55 ` [PATCH 1/4] remote, revision: factor out exclusive counting between two commits Stefan Beller
2017-11-08 19:55 ` [PATCH 2/4] submodule.c: factor start_ls_files_dot_dot out of get_superproject_working_tree Stefan Beller
2017-11-08 19:55 ` [PATCH 3/4] submodule.c: get superprojects gitlink value Stefan Beller
2017-11-08 19:55 ` [PATCH 4/4] git-status: report reference to superproject Stefan Beller
2017-11-08 22:36 ` [RFC PATCH 0/4] git-status reports relation " Jonathan Tan
2017-11-09  0:10   ` [RFD] Long term plan with submodule refs? Stefan Beller
2017-11-09  1:29     ` Jonathan Tan
2017-11-09  5:47       ` Junio C Hamano
2017-11-09  5:08     ` Junio C Hamano [this message]
2017-11-09 19:57       ` Stefan Beller
2017-11-09  6:54     ` Jacob Keller
2017-11-09 20:16       ` Stefan Beller
2017-11-10  3:37         ` Jacob Keller
2017-11-10 20:01           ` Stefan Beller
2017-11-11  5:25             ` Jacob Keller

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply to all the recipients using the --to, --cc,
  and --in-reply-to switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqbmkcjaxo.fsf@gitster.mtv.corp.google.com \
    --to=gitster@pobox.com \
    --cc=git@vger.kernel.org \
    --cc=jonathantanmy@google.com \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox