[PATCH] RFC: A new type of symbolic refs

[PATCH] RFC: A new type of symbolic refs

[Stefan Beller]

A symbolic ref can currently only point at (another symbolic) ref.
However are use cases for symbolic refs to point at other things
representing a commit-ish, such as gitlink entries in other
repositories.  In this use case we can use such a symbolic link
to strengthen the relationship between a submodule and a superproject.
Examples:

1) It makes it easier to explain when we recurse into submodules and
   to which sha1 the submodule is updated.

   Currently a submodule (or any repo) is either on a branch (i.e.
   HEAD is a symbolic ref) or is in 'detached HEAD' state (HEAD is
   a direct ref).
   For submodules it is wrong to be on its own branch if it wants to be
   controlled by the superproject as being on its own branch signals that
   the submodule is independent and can move the HEAD freely.
   Being in 'detached HEAD' state is the alternative to that and was
   chosen when git-submodule was implemented, but it is equally wrong;
   the lesser of two evils.

   Semantically the correct way to state a submodule is part of the
   superproject is by pointing its HEAD to the superproject.

   We do have "reset/checkout --recurse-submodules" now, but it is
   hard to explain what actually happens there (Which submodules are
   updated? All of them! -- But I want a subset only!)

   With this new mode of symbolic refs, any submodule that tracks the
   superproject, would 'automatically follow' the superproject as the
   submodules HEAD moves when the superproject changes.

2) "git -C submodule commit" would behave the same as it would on branch
   nowadays: It would make the commit in the submodule and then change
   the target of the symbolic ref which would be the index of the
   superproject! That implies that you do not need to 'git add' the
   submodule to the superproject, but have it done automatically.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h              |  2 ++
 refs/files-backend.c | 10 ++++++++++
 submodule.c          | 40 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 52 insertions(+)

diff --git a/cache.h b/cache.h
index 71fe092644..4f79d23202 100644
--- a/cache.h
+++ b/cache.h
@@ -2029,4 +2029,6 @@ void sleep_millisec(int millisec);
  */
 void safe_create_dir(const char *dir, int share);
 
+extern int read_external_symref(struct strbuf *from, struct strbuf *out);
+
 #endif /* CACHE_H */
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 0404f2c233..f56f7b87ce 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -713,6 +713,16 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 		goto out;
 	}
 
+	if (starts_with(buf, "repo:")) {
+		if (read_external_symref(&sb_contents, referent)) {
+			*type |= REF_ISBROKEN;
+			ret = -1;
+			goto out;
+		}
+		*type |= REF_ISSYMREF;
+		ret = 0;
+	}
+
 	/*
 	 * Please note that FETCH_HEAD has additional
 	 * data after the sha.
diff --git a/submodule.c b/submodule.c
index da2b484879..7297f90485 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2037,3 +2037,43 @@ int submodule_to_gitdir(struct strbuf *buf, const char *submodule)
 cleanup:
 	return ret;
 }
+
+int read_external_symref(struct strbuf *from, struct strbuf *out)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+	const char *repo, *gitlink;
+	int hint, code;
+	struct strbuf **split = strbuf_split(from, 0);
+	struct strbuf cmd_out = STRBUF_INIT;
+
+	if (!split[0] || !split[1])
+		return -1;
+
+	repo = split[0]->buf + 5; /* skip 'repo:' */
+	gitlink = split[1]->buf;
+
+	argv_array_pushl(&cp.args,
+			"ignored-first-arg",
+			"-C", repo,
+			"ls-tree", "-z", "HEAD", "--", gitlink, NULL);
+
+	/*
+	 * 17 accounts for '160000 commit ',
+	 * the \t before path and trailing \0.
+	 */
+	hint = 17 + GIT_SHA1_HEXSZ + split[1]->len;
+	code = capture_command(&cp, &cmd_out, hint);
+
+	strbuf_release(split[0]);
+	strbuf_release(split[1]);
+
+	if (!code) {
+		strbuf_reset(out);
+		strbuf_add(out, cmd_out.buf + strlen("160000 commit "),
+			   GIT_SHA1_HEXSZ);
+	} else
+		return -1;
+
+	return 0;
+}
+
-- 
2.13.2.695.g117ddefdb4

Re: [PATCH] RFC: A new type of symbolic refs

[Philip Oakley]

From: "Stefan Beller" <sbeller@google.com>
Sent: Tuesday, July 11, 2017 2:06 AM
>A symbolic ref can currently only point at (another symbolic) ref.
> However are use cases for symbolic refs to point at other things
> representing a commit-ish, such as gitlink entries in other
> repositories.  In this use case we can use such a symbolic link
> to strengthen the relationship between a submodule and a superproject.
> Examples:

If I understand this correctly, the new type is the 'starts_with(buf, ))'.

It just wasn't obvious from the text that the new type is "repo:" as you 
never spell it out in the commit message. Should it be included in the 
message?

Have I understood correctly?

--
Philip
[I'll be off-line till Friday, so will pick up then]

>
> 1) It makes it easier to explain when we recurse into submodules and
>   to which sha1 the submodule is updated.
>
>   Currently a submodule (or any repo) is either on a branch (i.e.
>   HEAD is a symbolic ref) or is in 'detached HEAD' state (HEAD is
>   a direct ref).
>   For submodules it is wrong to be on its own branch if it wants to be
>   controlled by the superproject as being on its own branch signals that
>   the submodule is independent and can move the HEAD freely.
>   Being in 'detached HEAD' state is the alternative to that and was
>   chosen when git-submodule was implemented, but it is equally wrong;
>   the lesser of two evils.
>
>   Semantically the correct way to state a submodule is part of the
>   superproject is by pointing its HEAD to the superproject.
>
>   We do have "reset/checkout --recurse-submodules" now, but it is
>   hard to explain what actually happens there (Which submodules are
>   updated? All of them! -- But I want a subset only!)
>
>   With this new mode of symbolic refs, any submodule that tracks the
>   superproject, would 'automatically follow' the superproject as the
>   submodules HEAD moves when the superproject changes.
>
> 2) "git -C submodule commit" would behave the same as it would on branch
>   nowadays: It would make the commit in the submodule and then change
>   the target of the symbolic ref which would be the index of the
>   superproject! That implies that you do not need to 'git add' the
>   submodule to the superproject, but have it done automatically.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
> cache.h              |  2 ++
> refs/files-backend.c | 10 ++++++++++
> submodule.c          | 40 ++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 52 insertions(+)
>
> diff --git a/cache.h b/cache.h
> index 71fe092644..4f79d23202 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -2029,4 +2029,6 @@ void sleep_millisec(int millisec);
>  */
> void safe_create_dir(const char *dir, int share);
>
> +extern int read_external_symref(struct strbuf *from, struct strbuf *out);
> +
> #endif /* CACHE_H */
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 0404f2c233..f56f7b87ce 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -713,6 +713,16 @@ static int files_read_raw_ref(struct ref_store 
> *ref_store,
>  goto out;
>  }
>
> + if (starts_with(buf, "repo:")) {
> + if (read_external_symref(&sb_contents, referent)) {
> + *type |= REF_ISBROKEN;
> + ret = -1;
> + goto out;
> + }
> + *type |= REF_ISSYMREF;
> + ret = 0;
> + }
> +
>  /*
>  * Please note that FETCH_HEAD has additional
>  * data after the sha.
> diff --git a/submodule.c b/submodule.c
> index da2b484879..7297f90485 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -2037,3 +2037,43 @@ int submodule_to_gitdir(struct strbuf *buf, const 
> char *submodule)
> cleanup:
>  return ret;
> }
> +
> +int read_external_symref(struct strbuf *from, struct strbuf *out)
> +{
> + struct child_process cp = CHILD_PROCESS_INIT;
> + const char *repo, *gitlink;
> + int hint, code;
> + struct strbuf **split = strbuf_split(from, 0);
> + struct strbuf cmd_out = STRBUF_INIT;
> +
> + if (!split[0] || !split[1])
> + return -1;
> +
> + repo = split[0]->buf + 5; /* skip 'repo:' */
> + gitlink = split[1]->buf;
> +
> + argv_array_pushl(&cp.args,
> + "ignored-first-arg",
> + "-C", repo,
> + "ls-tree", "-z", "HEAD", "--", gitlink, NULL);
> +
> + /*
> + * 17 accounts for '160000 commit ',
> + * the \t before path and trailing \0.
> + */
> + hint = 17 + GIT_SHA1_HEXSZ + split[1]->len;
> + code = capture_command(&cp, &cmd_out, hint);
> +
> + strbuf_release(split[0]);
> + strbuf_release(split[1]);
> +
> + if (!code) {
> + strbuf_reset(out);
> + strbuf_add(out, cmd_out.buf + strlen("160000 commit "),
> +    GIT_SHA1_HEXSZ);
> + } else
> + return -1;
> +
> + return 0;
> +}
> +
> -- 
> 2.13.2.695.g117ddefdb4
> 

Re: [PATCH] RFC: A new type of symbolic refs

[Philip Oakley]

From: "Philip Oakley" <philipoakley@iee.org>
> From: "Stefan Beller" <sbeller@google.com>
> Sent: Tuesday, July 11, 2017 2:06 AM
>>A symbolic ref can currently only point at (another symbolic) ref.
>> However are use cases for symbolic refs to point at other things
>> representing a commit-ish, such as gitlink entries in other
>> repositories.  In this use case we can use such a symbolic link
>> to strengthen the relationship between a submodule and a superproject.
>> Examples:
>
> If I understand this correctly, the new type is the 'starts_with(buf, ))'.

Oops, my mistaken message prep accidently removed the "repo:" from the 
above.

>
> It just wasn't obvious from the text that the new type is "repo:" as you 
> never spell it out in the commit message. Should it be included in the 
> message?
>
> Have I understood correctly?
>
> --
> Philip
> [I'll be off-line till Friday, so will pick up then]
>
>>
>> 1) It makes it easier to explain when we recurse into submodules and
>>   to which sha1 the submodule is updated.
>>
>>   Currently a submodule (or any repo) is either on a branch (i.e.
>>   HEAD is a symbolic ref) or is in 'detached HEAD' state (HEAD is
>>   a direct ref).
>>   For submodules it is wrong to be on its own branch if it wants to be
>>   controlled by the superproject as being on its own branch signals that
>>   the submodule is independent and can move the HEAD freely.
>>   Being in 'detached HEAD' state is the alternative to that and was
>>   chosen when git-submodule was implemented, but it is equally wrong;
>>   the lesser of two evils.
>>
>>   Semantically the correct way to state a submodule is part of the
>>   superproject is by pointing its HEAD to the superproject.
>>
>>   We do have "reset/checkout --recurse-submodules" now, but it is
>>   hard to explain what actually happens there (Which submodules are
>>   updated? All of them! -- But I want a subset only!)
>>
>>   With this new mode of symbolic refs, any submodule that tracks the
>>   superproject, would 'automatically follow' the superproject as the
>>   submodules HEAD moves when the superproject changes.
>>
>> 2) "git -C submodule commit" would behave the same as it would on branch
>>   nowadays: It would make the commit in the submodule and then change
>>   the target of the symbolic ref which would be the index of the
>>   superproject! That implies that you do not need to 'git add' the
>>   submodule to the superproject, but have it done automatically.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>> cache.h              |  2 ++
>> refs/files-backend.c | 10 ++++++++++
>> submodule.c          | 40 ++++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 52 insertions(+)
>>
>> diff --git a/cache.h b/cache.h
>> index 71fe092644..4f79d23202 100644
>> --- a/cache.h
>> +++ b/cache.h
>> @@ -2029,4 +2029,6 @@ void sleep_millisec(int millisec);
>>  */
>> void safe_create_dir(const char *dir, int share);
>>
>> +extern int read_external_symref(struct strbuf *from, struct strbuf 
>> *out);
>> +
>> #endif /* CACHE_H */
>> diff --git a/refs/files-backend.c b/refs/files-backend.c
>> index 0404f2c233..f56f7b87ce 100644
>> --- a/refs/files-backend.c
>> +++ b/refs/files-backend.c
>> @@ -713,6 +713,16 @@ static int files_read_raw_ref(struct ref_store 
>> *ref_store,
>>  goto out;
>>  }
>>
>> + if (starts_with(buf, "repo:")) {
>> + if (read_external_symref(&sb_contents, referent)) {
>> + *type |= REF_ISBROKEN;
>> + ret = -1;
>> + goto out;
>> + }
>> + *type |= REF_ISSYMREF;
>> + ret = 0;
>> + }
>> +
>>  /*
>>  * Please note that FETCH_HEAD has additional
>>  * data after the sha.
>> diff --git a/submodule.c b/submodule.c
>> index da2b484879..7297f90485 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>> @@ -2037,3 +2037,43 @@ int submodule_to_gitdir(struct strbuf *buf, const 
>> char *submodule)
>> cleanup:
>>  return ret;
>> }
>> +
>> +int read_external_symref(struct strbuf *from, struct strbuf *out)
>> +{
>> + struct child_process cp = CHILD_PROCESS_INIT;
>> + const char *repo, *gitlink;
>> + int hint, code;
>> + struct strbuf **split = strbuf_split(from, 0);
>> + struct strbuf cmd_out = STRBUF_INIT;
>> +
>> + if (!split[0] || !split[1])
>> + return -1;
>> +
>> + repo = split[0]->buf + 5; /* skip 'repo:' */
>> + gitlink = split[1]->buf;
>> +
>> + argv_array_pushl(&cp.args,
>> + "ignored-first-arg",
>> + "-C", repo,
>> + "ls-tree", "-z", "HEAD", "--", gitlink, NULL);
>> +
>> + /*
>> + * 17 accounts for '160000 commit ',
>> + * the \t before path and trailing \0.
>> + */
>> + hint = 17 + GIT_SHA1_HEXSZ + split[1]->len;
>> + code = capture_command(&cp, &cmd_out, hint);
>> +
>> + strbuf_release(split[0]);
>> + strbuf_release(split[1]);
>> +
>> + if (!code) {
>> + strbuf_reset(out);
>> + strbuf_add(out, cmd_out.buf + strlen("160000 commit "),
>> +    GIT_SHA1_HEXSZ);
>> + } else
>> + return -1;
>> +
>> + return 0;
>> +}
>> +
>> -- 
>> 2.13.2.695.g117ddefdb4
>>
> 

Re: [PATCH] RFC: A new type of symbolic refs

[Stefan Beller]

On Sun, Jul 16, 2017 at 6:04 AM, Philip Oakley <philipoakley@iee.org> wrote:

>
> If I understand this correctly, the new type is the 'starts_with(buf, "repo:"))'.
>
> It just wasn't obvious from the text that the new type is "repo:" as you
> never spell it out in the commit message. Should it be included in the
> message?
>
> Have I understood correctly?

Yes, you understood correctly.

The idea is to allow submodules to "borrow" its refs from the superproject.
My original idea was to only borrow HEAD from the superproject, but this
can be extended to more branches.

By having a more generic way of borrowing refs from another repo, this
can also be a different repo than the superproject (I don't know the use
case for this yet, but let's not have a to specific proposal).

Let's see how this RFC fares in further interest from the community.

Re: [PATCH] RFC: A new type of symbolic refs

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> +int read_external_symref(struct strbuf *from, struct strbuf *out)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +	const char *repo, *gitlink;
> +	int hint, code;
> +	struct strbuf **split = strbuf_split(from, 0);
> +	struct strbuf cmd_out = STRBUF_INIT;
> +
> +	if (!split[0] || !split[1])
> +		return -1;
> +
> +	repo = split[0]->buf + 5; /* skip 'repo:' */
> +	gitlink = split[1]->buf;
> +
> +	argv_array_pushl(&cp.args,
> +			"ignored-first-arg",
> +			"-C", repo,
> +			"ls-tree", "-z", "HEAD", "--", gitlink, NULL);
> +
> +	/*
> +	 * 17 accounts for '160000 commit ',
> +	 * the \t before path and trailing \0.
> +	 */
> +	hint = 17 + GIT_SHA1_HEXSZ + split[1]->len;
> +	code = capture_command(&cp, &cmd_out, hint);
> +
> +	strbuf_release(split[0]);
> +	strbuf_release(split[1]);
> +
> +	if (!code) {
> +		strbuf_reset(out);
> +		strbuf_add(out, cmd_out.buf + strlen("160000 commit "),
> +			   GIT_SHA1_HEXSZ);
> +	} else
> +		return -1;
> +
> +	return 0;
> +}

This may help the initial checkout, but to be useful after that, we
need to define what happens when an equivalent of "git update-ref
HEAD" is done in the submodule repository, when HEAD is pointing
elsewhere.  The above only shows read-only operation, which is not
all that interesting.

I _think_ a symbolic HEAD that points upwards to the gitlink entry in
the superproject's index is the easiest to understand and it is
something we can define a clear and useful semantics for.

When a recursive checkout of a branch 'foo' is made in the
superproject, the index in the superproject would name the commit in
the submodule to be checked out.  We traditionally detech the HEAD
at the submodule to that commit, but instead we could say "check the
index of the superproject to see where the HEAD should be pointing
at" in the submodule.  Either way, immediately after such a
recursive checkout, "git status" inside the submodule would find
that the HEAD points at the commit recorded in the 'foo' branch of
the superproject and things are clean.  

After you work in the submodule and make a commit, an equivalent of
"git update-ref HEAD" is done behind the scene to update HEAD in the
submodule.  In the traditional world, that is done to detached HEAD
and nothing else changes, but if the symref HEAD points upwards into
the index of the superproject, what needs to be done is very obvious;
we do "git add submodule" in the superproject.  And this is not just
limited to creating a commit in the submodule.  "reset --hard HEAD~2"
in the submodule to rewind the HEAD by two commits would also be an
update to HEAD and through the symref-ness of the HEAD should result
in an update to the index of the superproject.

However, I do not think a good explanation of what should mean when
this new-style symbolic HEAD points at a commit in the superproject,
whether its limited to its HEAD or a tip of an arbitrary branch that
may not even be checked out.  These are not something we can easily
change without affecting wider context.  Our submodule, when we make
a new commit, may be ready to advance, but our superproject and
other submodules may not be ready to be included in a new commit in
the superproject.

So I think the idea this patch illustrates is on to something
interesting and potentially useful, but I am not sure if it makes
sense to tie it to anything but the index of the superproject.

Even if we limit ourselves to pointing at the index of the
superproject, there probably are a handful of interesting issues
that need to be clarified (not in the sense of "this and that issues
exist, so this won't be a useful feature", but in the sense of "we'd
be able to do these useful things using this feature, and we need to
fill in more details"), such as:

 - Making new commits in the submodule available to the upstream.
   Just like a detached HEAD in the submodule, this is not tied to
   any concrete branch, and it is unclear how a recursive "push"
   from the superproject should propagate the changes to the
   upstream of the submodule;

 - Switching between branches that bind the same commit for the
   submodule in the superproject would work just like switching
   between branches that record the same blob for a path, i.e. it
   will carry forward a local modification.

 - The index entry in the superproject may now have to get involved
   in fsck and reachability study in the submodule as reachability
   root.  A corollary to this is that submodules behave more
   similarly to regular blobs wrt "git reset --hard" in the
   superproject, which is a good thing.  "git -C submodule commit &&
   git reset --hard" will create a new commit in the submodule, add
   it to the index of the superproject, and then lose that change
   from the index of the superproject, making the commit
   unreachable, just like "edit file && git add file && git reset
   --hard" in the superproject will make the blob that records the
   updated content of the file unreachable.

Thanks.

Re: [PATCH] RFC: A new type of symbolic refs

[Stefan Beller]

On Mon, Jul 17, 2017 at 2:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
> I _think_ a symbolic HEAD that points upwards to the gitlink entry in
> the superproject's index is the easiest to understand and it is
> something we can define a clear and useful semantics for.

ok, let's start with that. Once we have that we can see if that also
makes sense for the non-submodule/superproject world.

> When a recursive checkout of a branch 'foo' is made in the
> superproject, the index in the superproject would name the commit in
> the submodule to be checked out.  We traditionally detech the HEAD
> at the submodule to that commit, but instead we could say "check the
> index of the superproject to see where the HEAD should be pointing
> at" in the submodule.  Either way, immediately after such a
> recursive checkout, "git status" inside the submodule would find
> that the HEAD points at the commit recorded in the 'foo' branch of
> the superproject and things are clean.

Right.

> After you work in the submodule and make a commit, an equivalent of
> "git update-ref HEAD" is done behind the scene to update HEAD in the
> submodule.  In the traditional world, that is done to detached HEAD
> and nothing else changes, but if the symref HEAD points upwards into
> the index of the superproject, what needs to be done is very obvious;
> we do "git add submodule" in the superproject.  And this is not just
> limited to creating a commit in the submodule.  "reset --hard HEAD~2"
> in the submodule to rewind the HEAD by two commits would also be an
> update to HEAD and through the symref-ness of the HEAD should result
> in an update to the index of the superproject.

agreed. we'd find out what HEAD~2 is and then write the gitlink in the
superproject index to be that commit.

> However, I do not think a good explanation of what should mean when
> this new-style symbolic HEAD points at a commit in the superproject,
> whether its limited to its HEAD or a tip of an arbitrary branch that
> may not even be checked out.

The proposal only inspects the currently staged value of a gitlink, there
is no way to reference a specific commit or (non-checked-out) branch,
so it is not possible to point at them with the given read syntax.
So in case the superproject changes its HEAD (such as checking
out a different branch, or detaching the HEAD), then the submodules
HEAD changes automatically as the superprojects tree may change.


>  These are not something we can easily
> change without affecting wider context.  Our submodule, when we make
> a new commit, may be ready to advance, but our superproject and
> other submodules may not be ready to be included in a new commit in
> the superproject.

That is an interesting problem, as un-staging the submodule
is not as easy as un-staging a change in a file. Assume we
have a layout as

  git init super
  git -C super init sub
  echo change >>super/sub/file
  git -C super/sub commit -a -m "submodule changes"

then the following behaviors seem reasonable:

    git -C super reset --soft sub
      As the superprojects index is not touched, so is the
      submodules HEAD and working tree. The submodule
      stays as is.

    git -C super reset --mixed sub
      As this resets the index of the superproject, but not the working
      tree, I'd expect the submodules HEAD to be detached keeping
      the new content alive, such that it can be added as
        git -C super add --re-attach-with-new-symref-mode sub

    git -C super reset --hard sub
      This resets the index and the working tree in the superproject,
      so the submodules commit with the new content would be lost,
      the submodules HEAD is still the new symbolic ref.  The subs
      working tree is dirty and contains the change unless
      --recurse-submodules is given in addition, which would wipe
      the subs working directory in addition.

> So I think the idea this patch illustrates is on to something
> interesting and potentially useful, but I am not sure if it makes
> sense to tie it to anything but the index of the superproject.

Hypothetical use case:

  Your project targets multiple platforms and you are interested
  in not producing the perfect history like we do, but you never
  change a commit once created (for policy legal reasons).
  Naturally this produces a lot of duds.
  Create a a gitlink for each platform that points to the latest
  commit known to work for the platform. Point HEAD to the
  repository *itself*, and gitlink of the platform that you want to
  follow. After each fetch, checking out the latest HEAD is known
  to work for your platform.  (This hypothetical HEAD reference
  to itself is entertaining for sure.)

> Even if we limit ourselves to pointing at the index of the
> superproject, there probably are a handful of interesting issues
> that need to be clarified (not in the sense of "this and that issues
> exist, so this won't be a useful feature", but in the sense of "we'd
> be able to do these useful things using this feature, and we need to
> fill in more details"), such as:
>
>  - Making new commits in the submodule available to the upstream.
>    Just like a detached HEAD in the submodule, this is not tied to
>    any concrete branch, and it is unclear how a recursive "push"
>    from the superproject should propagate the changes to the
>    upstream of the submodule;

In Gerrit land people are trained to use
  git push HEAD:refs/for/master
which would work fine when propagated to the submodule as-is.

There are two answers for the life without Gerrit in a dumb git remote.
(a) In the far future in the ideal world, where version control is a
solved problem, submodules can reside inside the superproject
on the remote as well, i.e. in $GIT_DIR/modules/name, the URL
may be twisted to not expose the "modules/name" part, though such that

    example.org/superproject
    example.org/superproject/submodule

is possible. In this world a push of the superproject would also transfer
the relevant submodule commits, such that the remote has all of the
submodules history that contains all gitlink entries.

Pushing from inside the submodule may either push the superproject
(as you can push from a directory of the superproject) or it would
abort to not confuse a user.

(b) In the near future, I have no idea how to best solve this.
Maybe we can push it as a detached HEAD to the remote?

>  - Switching between branches that bind the same commit for the
>    submodule in the superproject would work just like switching
>    between branches that record the same blob for a path, i.e. it
>    will carry forward a local modification.

Right.

>
>  - The index entry in the superproject may now have to get involved
>    in fsck and reachability study in the submodule as reachability
>    root.  A corollary to this is that submodules behave more
>    similarly to regular blobs wrt "git reset --hard" in the
>    superproject, which is a good thing.  "git -C submodule commit &&
>    git reset --hard" will create a new commit in the submodule, add
>    it to the index of the superproject, and then lose that change
>    from the index of the superproject, making the commit
>    unreachable, just like "edit file && git add file && git reset
>    --hard" in the superproject will make the blob that records the
>    updated content of the file unreachable.

This is in sync with my understanding above.

Thanks,
Stefan

Re: [PATCH] RFC: A new type of symbolic refs

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

>> Even if we limit ourselves to pointing at the index of the
>> superproject, there probably are a handful of interesting issues
>> that need to be clarified (not in the sense of "this and that issues
>> exist, so this won't be a useful feature", but in the sense of "we'd
>> be able to do these useful things using this feature, and we need to
>> fill in more details"), such as:
>>
>>  - Making new commits in the submodule available to the upstream.
>>    Just like a detached HEAD in the submodule, this is not tied to
>>    any concrete branch, and it is unclear how a recursive "push"
>>    from the superproject should propagate the changes to the
>>    upstream of the submodule;
>
> In Gerrit land people are trained to use
>   git push HEAD:refs/for/master
> which would work fine when propagated to the submodule as-is.

What I was alluding to is how you find 'master' in 'for/master',
given the submodule HEAD which only says "I point at the index of
the superproject".  Note that 'master' here refers to a branch
in the submodule.  If we further limit ourselves and declare that
a submodule that uses this external-reference as HEAD must always
be on the same branch as the superproject is on, then we can find
what branch the superproject is on, and use that.