git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jonathan Tan <jonathantanmy@google.com>
To: Stefan Beller <sbeller@google.com>
Cc: git@vger.kernel.org
Subject: Re: [RFD] Long term plan with submodule refs?
Date: Wed, 8 Nov 2017 17:29:45 -0800	[thread overview]
Message-ID: <20171108172945.33c42a0e91b4ac494217b788@google.com> (raw)
In-Reply-To: <20171109001007.11894-1-sbeller@google.com>

On Wed,  8 Nov 2017 16:10:07 -0800
Stefan Beller <sbeller@google.com> wrote:

I thought of a possible alternative and how it would work.

> Possible data models and workflow implications
> ==============================================
> In the following different data models are presented, which aid a submodule
> heavy workflow each giving pros and cons.

What if, in the submodule, we have a new ref backend that mirrors the
superproject? When initializing the submodule, its original refs are not
cloned at all, but instead virtual refs are used.

Creation of brand-new refs is forbidden in the submodule.

When reading a ref in the submodule, if that ref is the current branch
in the superproject, read the corresponding gitlink entry in the index
(which may be dirty); otherwise read the gitlink in the tree of the tip
commit.

When updating a ref in the submodule, if that ref is the current branch
in the superproject, update the index; otherwise, create a commit on top
of the tip and update the ref to point to the new tip.

No synchronicity is enforced between superproject and submodule in terms
of HEAD, though: If a submodule is currently checked out to a branch,
and the gitlink for that branch is updated through whatever means, that
is equivalent to a "git reset --soft" in the submodule.

These rules seem straightforward to me (although I have been working
with Git for a while, so perhaps I'm not the best judge), and I think
leads to a good workflow, as discussed below.

> Workflows
> =========
> * Obtaining a copy of the Superproject tightly coupled with submodules
>   solved via git clone --recurse-submodules=<pathspec>
> * Changing the submodule selection
>   solved via submodule.active flags
> * Changing the remote / Interacting with a different remote for all submodules
>   -> need to be solved, not core issue of this discussion
> * Syncing to the latest upstream
>   solved via git pull --recurse  

(skipping the above, since they are either solved or not a core issue)

> * Working on a local feature in one submodule
>   -> How do refs work spanning superproject/submodule?

This is perhaps one weak point of my proposal - you can't work on a
submodule as if it were independent. You can checkout a branch and make
commits, but (i) they will automatically affect the superproject, and
(ii) the "origin/foo" etc. branches are those of the superproject. (But
if you checkout a detached HEAD, everything should still work.)

> * Working on a feature spanning multiple submodules
>   -> How do refs work spanning multiple repos?

The above rules allow the following workflow:
 - "checkout --recurse-submodules" the branch you want on the
   superproject
 - make whatever changes you want in each submodule
 - commit each individual submodule (which updates the index of the
   superproject), then commit the superproject (we can introduce a
   commit --recurse-submodules to make this more convenient)
 - a "push --recurse-submodules" can be implemented to push the
   superproject and its submodules independently (and the same refspec
   can be legitimately used both when pushing the superproject and when
   pushing a submodule, since the ref names are the same, and not by
   coincidence)

If the user insists on making changes on a non-current branch (i.e. by
creating commits in submodules then using "git update-ref" or
equivalent), possibly multiple commits would be created in the
superproject, but the user can still squash them later if desired.

> * Working on a bug fix (Changing the feature that you currently work on, branches)
>   -> How does switching branches in the superproject affect submodules

You will have to stash or commit your changes. (Which reminds me...GC in
the subproject will need to consult the revlog of the superproject too.)

> New type of symbolic refs
> =========================
> A symbolic ref can currently only point at a ref or another symbolic ref.
> This proposal showcases different scenarios on how this could change in the
> future.
> 
> HEAD pointing at the superprojects index
> ----------------------------------------

Assuming we don't need synchronicity, the existing HEAD format can be
retained. To clarify what happens during ref writes, I'll reuse the
scenarios Stefan described:

> Ref write operations driven by the submodule, affecting target ref
>   e.g. git commit, reset --hard, update-ref (in the submodule)
> 
> The HEAD stays the same, pointing at the superproject.
> The gitlink is changed to the target sha1, using
> 
>   git -C <superproject> update-index --add \
>       --cacheinfo 160000,$SHA1,<gitlink-path>
> 
> This will affect the superprojects index, such that then a commit in
> the superproject is needed.

In this proposal, the HEAD also stays the same (pointing at the branch).

Either the index is updated or a commit is needed. If a commit is
needed, it is automatically performed.

> Ref write operations driven by the superproject, changing the gitlink
>   e.g. git checkout <tree-ish>, git reset --hard (in the superproject)
> 
> This will change the gitlink in the superprojects index, such that the HEAD
> in the submodule changes, which would trigger an update of the
> submodules working tree.

The HEAD in the submodule is unchanged. If the value of a ref has
changed "from underneath", this is as if a "git reset --soft" was done.

> Superproject operations spanning index and worktree
>   E.g. git reset --mixed
> As the submodules HEAD is defined in the index, we would reset it to the
> version in the last commit. As --mixed promises to not touch the working tree,
> the submodules worktree would not be touched. git reset --mixed in the
> superproject is the same as --soft in the submodule.

Same.

> Consistency considerations (gc)
>   e.g. git gc --aggressive --prune=now
> 
> The repacking logic is already aware of a detached HEAD, such that
> using this new symref mechanism would not generate problems as long as
> we keep the HEAD attached to the superproject. However when commits/objects
> are created while the HEAD is attached to the superproject and then HEAD
> switches to a local branch, there are problems with the created objects
> as they seem unreachable now.
> 
> This problem is not new as a superproject may record submodule objects
> that are not reachable from any of the submodule branches. Such objects
> fall prey to overzealous packing in the submodule.

The scenario Stefan describes will work OK - if a commit is created
while the HEAD is pointing to a branch, then either the superproject's
index will be updated or commits will be created in the superproject.
When GC reads the list of refs in the submodule, the new submodule
commit will be included. (Remember that if the superproject's current
branch is "foo", "refs/heads/foo" in the submodule reflects the
superproject's index, so any changes to the index, though uncommitted,
will appear as a ref.)

The problem still exists (e.g. stashes in the superproject) but is
reduced, I think.

  reply	other threads:[~2017-11-09  1:29 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-11-08 19:55 [RFC PATCH 0/4] git-status reports relation to superproject Stefan Beller
2017-11-08 19:55 ` [PATCH 1/4] remote, revision: factor out exclusive counting between two commits Stefan Beller
2017-11-08 19:55 ` [PATCH 2/4] submodule.c: factor start_ls_files_dot_dot out of get_superproject_working_tree Stefan Beller
2017-11-08 19:55 ` [PATCH 3/4] submodule.c: get superprojects gitlink value Stefan Beller
2017-11-08 19:55 ` [PATCH 4/4] git-status: report reference to superproject Stefan Beller
2017-11-08 22:36 ` [RFC PATCH 0/4] git-status reports relation " Jonathan Tan
2017-11-09  0:10   ` [RFD] Long term plan with submodule refs? Stefan Beller
2017-11-09  1:29     ` Jonathan Tan [this message]
2017-11-09  5:47       ` Junio C Hamano
2017-11-09  5:08     ` Junio C Hamano
2017-11-09 19:57       ` Stefan Beller
2017-11-09  6:54     ` Jacob Keller
2017-11-09 20:16       ` Stefan Beller
2017-11-10  3:37         ` Jacob Keller
2017-11-10 20:01           ` Stefan Beller
2017-11-11  5:25             ` Jacob Keller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171108172945.33c42a0e91b4ac494217b788@google.com \
    --to=jonathantanmy@google.com \
    --cc=git@vger.kernel.org \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).