Re: RFC: Making submodules "track" branches

[Johan Herland <johan@herland.net>]

On Wednesday 09 June 2010, Steven Michalske wrote:
> On Jun 9, 2010, at 1:22 AM, Johan Herland wrote:
> > On Wednesday 09 June 2010, Jens Lehmann wrote:
> >> Am 08.06.2010 23:52, schrieb Johan Herland:
> >>> [1]: Say your submodule usually tracks a branch, but you're
> >>> creating some tag in the super-repo, and you want that tag to
> >>> uniquely identify
> >>> the submodule. You achieve this by making sure the tagged commit
> >>> removes the relevant "branch = whatever" line from .gitmodules,
> >>> and records the appropriate submodule version in the super-repo
> >>> tree. Then, you can revert the .gitmodules change on the next
> >>> commit to resume tracking the submodule branch.
> >>>
> >>> Now, whenever you checkout the tag, you will always get the exact
> >>> same
> >>> version of the submodule, although the submodule otherwise tracks
> >>> some
> >>> branch.
> >>
> >> Won't work anymore when we would use 0{40} or drop it from the
> >> tree. AFAICS always-tip and referencing a certain commit don't mix
> >> well.
> >
> > AFAICS, it would still work as long as it exists in the tree for
> > that specific commit (but is missing/0{40} in other commits).
> >
> > We're not mixing "always-tip" and "exact-commit" in the same
> > commit. We use "always-tip" in regular commits, and then temporarily
> > switch to "exact-commit" in the commits where a certain submodule
> > version is required.
>
> When making a tag, could the notes system be used for marking what
> commit was exactly on the submodule, perhaps include the closest
> remote commits as well?
>
>
> Something like
>
> Submodule Status:
> 	["foo"]
> 		branch = subtopic:SHA
>
> This assumes that git notes are shared/cloned......

I don't think this is a good use of notes. We already have an 
infrastructure for recording the exact submodule version used (the 
submodule entry in the superproject tree), and I see no reason why we 
should not use that in this case.

> Other thoughts.
>
>
> Things that should still work with tracking submodules.
> 	- bisect   - Must be able to identify that a submodule change
> introduced the bug.

When tracking submodule commits (the default), this is just a matter of 
recursively applying the bisect operation to the good/bad submodule 
commits identified by the superproject bisect.

When tracking submodule branches, you can no longer use bisect in the 
superproject to find bugs in the submodule, since it would always check 
out the same (i.e. latest) version of the submodule (I'm assuming that 
we're not switching submodule branches in .gitmodules here...).

In the tag scenario I present which is quoted above, we're switching 
between tracking submodule commits (in the commits that are tagged), 
and tracking submodule branches (in the other commits), so bisect would 
be next to useless (unless you limit the bisect to only look at the 
tagged commits).

Alternatively, we could reconsider the question I asked Ævar initially 
in this thread:

<quote>
When do you update the commit reference in the submodule? "never" would 
work (since you're checking out a different commit anyway), "always" 
would also work (for the same reason), but would litter the superrepo 
history with submodule updates. There may be a better alternative 
somewhere in between.
</quote>

So if we _always_ (or often) update the submodule commit reference in 
the superproject, then we could disregard the branch tracking while 
bisecting and checkout the commit referenced from the super-repo. That 
would hopefully be as useful as if we'd tracked submodule commits 
explicitly.

But again, the more often we update the submodule commit reference 
(while still primarily tracking a _branch_ in the submodule), the more 
we "litter" the superproject history with these updates.

> 	- archive  - Should it use the version from the commit, or the
> latest?

Good question. In principle, since you've explicitly asked to track a 
branch, the only assumption Git can make is that you really want the 
_latest_ version of that subdmodule/branch.

Again we're back to the question of if/how often we record update 
submodule commits in the superproject, when the submodule primarily 
tracks a branch: If we're religious about recording our submodule 
commit references in the superproject, then we can temporarily 
disregard the branch tracking in order to get a somewhat realistic 
submodule update history.

>       - rebase   - update all of the submodule commits? 

Again, depends. If you really want branch-tracking, rebase will not 
change the submodule (unless you change which branch is tracked 
in .gitmodules).

Here, I don't see a good rationale for updating submodules though. If 
you're tracking submodule branches, then a superproject rebase won't 
affect what's at the tip of a submodule branch.

...unless you're talking about rebasing "foo" onto "bar in the 
superproject causing a corresponding rebase of "subfoo" onto "subbar" 
in the submodule, which is a whole 'nother can of worms...

> 	- checkout - tip vs commit

No question. If you've specified branch-tracking in .gitmodules, you get 
tip, otherwise you get commit.

> 	- reset --hard - Good question... not sure.... probably depend on
> tip vs commit like checkout.

As above, a reset --hard in the superproject does not affect what's at 
the tip of some submodule branch, so if you've chosen to track 
submodule branches, a reset --hard will not touch the submodule (unless 
the reset changes .gitmodules, obviously)

> 	- More????
>
> I would rather the submodule entree in the tree be always updated to
> what is in the submodule on the commit, so that the history is always
> there.

I see your point, as this would enable you to temporarily disable 
branch-tracking after the fact (typically for debugging purposes). But 
we still have to weigh its usefulness (in practice) against the cost of 
adding these extra commit references.

After all, even if Git does not do this automatically, you could still 
fairly easily add a pre-commit hook in the superproject that stages all 
submodule references.

> Then actions updating the repository from remotes 
> automatically pull the latest version.  I feel that the submodule if
> automatically be pulled, merged, etc, than the submodule should get a
> commit, with the message about this being an automatic update of the
> submodule.  Checking out is a different story.... checking out a
> branch tip of the super gets the latest tip from the submodule.  When
> you commit, the submodule gets it's auto commit, then a second commit
> for the code changes.  checking out a previous revision should
> probably put the sub module to the state it was in at that point in
> time.  Creating a branch and adding new content would update
> according to the rules.  but show the change of the subproject as
> from the super's at the branch point, not the tip.
>
> This way older gits have a submodule to work with and newer gits will
> do the right thing.
>
> Example:
>
> s-y-y-z
> A-B-C-D
> \
>   \F-G
> s-z-z
>
> F is branched when the latest sub module is at z  but shows the
> change from s not z because A the parent of F was created with the
> submodule at s
>
> Situational Example:
>
> I am developing away and as I progress in development I get a
> regression bug, so I run git bisect from the last stable release with
> out this bug, and it starts bisecting away.
>
> In the mode where we don't store the state of the project I can't
> bisect the changes against the subproject, where my bug might have
> been introduced from.

No, you would have to run a separate bisect in the subproject. What's 
wrong about that?

> So that issue should be probably handled in the git bisect code, that
> is "Make git bisect submodule aware"  in more verbose terms, when
> bisecting a super project the sub modules should undergo bisection as
> well.  This is a permutation that will expand rapidly, but some
> thoughts on how to dig into the bisection issues.
> This is another email ;-)
>
>
> Rebase:
> 	With the auto commit of submodule scheme, a rebase would change the
> tracking branches to the latest of the tracked version. and auto
> merge and record the previous submodule revision in the commit
> message of the submodule auto commit.

Not sure I understand what you're getting at here. Say you're tracking 
submodule branches (in the superproject's .gitmodules). Now, you do a 
rebase in the superproject. If the rebase does not change .gitmodules 
(i.e. which branch is tracked in the submodule), then there is nothing 
to be done in the submodule (it has already checked out the tip of that 
branch). If the rebase _does_ change .gitmodules - let's say there's a 
conflict in which submodule branch to track - then you first resolve 
that conflict in the superproject, to track the appropriate submodule 
branch. Then the tip of that branch is simply checked out in the 
submodule. You may want to record this update in the superproject, 
either by recording a separate commit, or by amending into the rebased 
commit.

Now, it might be that the correct resolution of the superproject's 
rebase conflict requires a nested rebase in the submodule, but this is 
certainly not a conclusion that Git can reach independently, so you 
will have to do the nested rebase manually, and then record the rebased 
branch in the superproject's .gitmodules (and optionally commit the 
updated submodule reference).

> Checkout with nonexistant submodule sha:
> 	This is the case where the submodules ref was not pushed publicly,
> so, the contents are not available.  You get a nice warning and the
> tip of the submodules branch gets checked out for that submodule.

If the .gitmodules specifies branch "foo" to be checked out in the 
submodule, and branch "foo" does not exist in the submodule, then 
that's the same type of error as if the superproject specifies a commit 
SHA1 for the submodule, and that commit does not exist in the 
submodule.

Granted, in the branch case, we can make it more network/remote-aware, 
by checking for both "foo" and "origin/foo" before giving up.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net