RFC: Making submodules "track" branches

RFC: Making submodules "track" branches

[Ævar Arnfjörð Bjarmason]

On Fri, May 21, 2010 at 16:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> Add a $toplevel variable accessible to `git submodule foreach`, it
> contains the absolute path of the top level directory (where
> .gitmodules is).
>
> This makes it possible to e.g. read data in .gitmodules from within
> foreach commands. I'm using this to configure the branch names I want
> to track for each submodule:
>
>    git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
>
> For a little history: This patch is borne out of my continuing fight
> of trying to have Git track the branches of submodules, not just their
> commits.
>
> Obviously that's not how they work (they only track commits), but I'm
> just interested in being able to do:
>
>    git submodule foreach 'git pull'
>
> Of course that won't work because the submodule is in a disconnected
> head, so I first have to connect it, but connect it *to what*.
>
> For a while I was happy with this because as fate had it, it just so
> happened to do what I meant:
>
>    git submodule foreach 'git checkout $(git describe --all --always) && git pull'
>
> But then that broke down, if there's a tag and a branch the tag will
> win out, and I can't git pull a branch:
>
>    $ git branch -a
>    * master
>      remotes/origin/HEAD -> origin/master
>      remotes/origin/master
>    $ git tag -l
>    release-0.0.6
>    $ git describe --always --all
>    release-0.0.6
>
> So I figured that I might as well start tracking the branches I want
> in .gitmodules itself:
>
>    [submodule "yaml-mode"]
>        path = yaml-mode
>        url = git://github.com/yoshiki/yaml-mode.git
>        branch = master
>
> So now I can just do (as stated above):
>
>    git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
>
> Maybe there's a less painful way to do *that* (I'd love to hear about
> it). But regardless of that I think it's a good idea to be able to
> know what the top-level is from git submodule foreach.

This patch is getting merged to next as per the June 2 What's cooking
in Git post.

But I wonder how evil it would be to expand this this idea to allow
the porcelain to track branches instead of commits at the porcelain
level.

That /could/ work like this. The tree format would be exactly the
same, i.e. bound to a specific commit:

    $ git ls-tree HEAD | grep subthing
    160000 commit 37469ca3fae264e790e4daac0fa8f2ddf8039c93  subthing

*But*, the user could add some new submodule.*.* config key/values
that specify what branch the module should track and whether 'git
pull' on the master project should also pull new changes (from the
'newstuff' branch) into the submodule:

    [submodule "subthing"]
        path = subthing
        url = git://github.com/avar/subthing.git
        branch = newstuff
        update-on-pull = true

Coupled with .gitignore this would allow for SVN-like externals that
always track the latest version of upstream, but it'd all be done on
the porcelain side.

The checked out copy wouldn't match the commit in the tree, but the
user could still git add && git commit it to record the new commit in
the master repository history.

The lack of this ability seems to be a fairly common complaint about
submodules in Git, that you always have to do something in the parent
project to update the submodules, even if you don't care about
specific revisions, or the ability to roll back.

I couldn't find a prior discussion of this on the list, maybe this has
been beaten to death already.

Re: RFC: Making submodules "track" branches

[Johan Herland]

On Tuesday 08 June 2010, Ævar Arnfjörð Bjarmason wrote:
> On Fri, May 21, 2010 at 16:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> 
wrote:
> > Add a $toplevel variable accessible to `git submodule foreach`, it
> > contains the absolute path of the top level directory (where
> > .gitmodules is).
> > 
> > This makes it possible to e.g. read data in .gitmodules from within
> > foreach commands. I'm using this to configure the branch names I want
> > to track for each submodule:
> > 
> >    git submodule foreach 'git checkout $(git config --file
> > $toplevel/.gitmodules submodule.$name.branch) && git pull'
> > 
> > For a little history: This patch is borne out of my continuing fight
> > of trying to have Git track the branches of submodules, not just their
> > commits.
> > 
> > Obviously that's not how they work (they only track commits), but I'm
> > just interested in being able to do:
> > 
> >    git submodule foreach 'git pull'
> > 
> > Of course that won't work because the submodule is in a disconnected
> > head, so I first have to connect it, but connect it *to what*.
> > 
> > For a while I was happy with this because as fate had it, it just so
> > happened to do what I meant:
> > 
> >    git submodule foreach 'git checkout $(git describe --all --always)
> > && git pull'
> > 
> > But then that broke down, if there's a tag and a branch the tag will
> > win out, and I can't git pull a branch:
> > 
> >    $ git branch -a
> >    * master
> >      remotes/origin/HEAD -> origin/master
> >      remotes/origin/master
> >    $ git tag -l
> >    release-0.0.6
> >    $ git describe --always --all
> >    release-0.0.6
> > 
> > So I figured that I might as well start tracking the branches I want
> > in .gitmodules itself:
> > 
> >    [submodule "yaml-mode"]
> >        path = yaml-mode
> >        url = git://github.com/yoshiki/yaml-mode.git
> >        branch = master
> > 
> > So now I can just do (as stated above):
> > 
> >    git submodule foreach 'git checkout $(git config --file
> > $toplevel/.gitmodules submodule.$name.branch) && git pull'
> > 
> > Maybe there's a less painful way to do *that* (I'd love to hear about
> > it). But regardless of that I think it's a good idea to be able to
> > know what the top-level is from git submodule foreach.
> 
> This patch is getting merged to next as per the June 2 What's cooking
> in Git post.
> 
> But I wonder how evil it would be to expand this this idea to allow
> the porcelain to track branches instead of commits at the porcelain
> level.
> 
> That /could/ work like this. The tree format would be exactly the
> same, i.e. bound to a specific commit:
> 
>     $ git ls-tree HEAD | grep subthing
>     160000 commit 37469ca3fae264e790e4daac0fa8f2ddf8039c93  subthing
> 
> *But*, the user could add some new submodule.*.* config key/values
> that specify what branch the module should track and whether 'git
> pull' on the master project should also pull new changes (from the
> 'newstuff' branch) into the submodule:
> 
>     [submodule "subthing"]
>         path = subthing
>         url = git://github.com/avar/subthing.git
>         branch = newstuff
>         update-on-pull = true

I certainly like the idea, and so far this is the best way I've seen for 
associating submodules to branches. I don't like the last "update-on-pull" 
option, though. It should probably be somewhat more general set of 
options/triggers with a richer set of values than true/false. What about 
something like this?

    [submodule "subthing"]
        path = subthing
        url = git://github.com/avar/subthing.git
        branch = newstuff
        on-pull = checkout,pull
        on-checkout = checkout
        on-commit = ignore (or commit?)
        ...

See below for more discussion...

> Coupled with .gitignore this would allow for SVN-like externals that
> always track the latest version of upstream, but it'd all be done on
> the porcelain side.
> 
> The checked out copy wouldn't match the commit in the tree, but the
> user could still git add && git commit it to record the new commit in
> the master repository history.
> 
> The lack of this ability seems to be a fairly common complaint about
> submodules in Git, that you always have to do something in the parent
> project to update the submodules, even if you don't care about
> specific revisions, or the ability to roll back.
> 
> I couldn't find a prior discussion of this on the list, maybe this has
> been beaten to death already.

There are a lot of non-trivial challenges when you want to aggregate several 
submodule operations into a single "toplevel" command. Here are some off the 
top of my head:

- When submodule pulls result in conflicts, these must be presented to the 
user in a way that's simple and straightforward for the user to resolve.

- When switching branches in the superrepo, you sometimes also want to 
switch branches in the submodule. This is signalled by changing the 
submodules.subthing.branch variable in .gitmodules between the two branches. 
However, it means that the submodule's update/pull operation must also be 
done on 'checkout' in the superrepo.

- How to handle local/uncommitted (staged or unstaged) modifications in a 
submodule when pulling or switching branches in the superrepo? The right 
answer here is probably to do the same as in the no-submodule case, i.e. to 
refuse if it would clobber/conflict with the local modifications.

- When you track submodule branches instead of commits, the actual commit 
referenced in the superrepo is no longer as important (provided it's part of 
the ancestry of the submodule branch you're tracking). However, diff/status 
will still list the submodule as changed because you checked out a different 
commit from what Git has recorded. This raises two concerns: (1) What 
_should_ be considered "changed" from the diff/status perspective when 
tracking submodule branches? and (2) When do you update the commit reference 
in the submodule? "never" would work (since you're checking out a different 
commit anyway), "always" would also work (for the same reason), but would 
litter the superrepo history with submodule updates. There may be a better 
alternative somewhere in between.

- If you want to give the illusion of "one big repo" then maybe it should 
also be possible to trigger submodule commits from a superrepo commit? (i.e. 
having a single toplevel "git commit" also trigger commits in submodules). 
Some users will want to specify the commit message for each submodule 
separately (IMHO the better approach), while some will want to give only one 
commit message that is reused in every submodule commit.

- As always with submodules, keep the case of nested submodules in mind.

There are probably more issues that escape me now...

Thanks for resurrecting the discussion.


Have fun! :)

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: RFC: Making submodules "track" branches

[Marc Branchaud]

On 10-06-08 03:12 AM, Johan Herland wrote:
> 
> There are probably more issues that escape me now...

Thanks for bringing this up!  I'm also very interested in this topic.

The main issue I see is that I don't always want my submodules to track (or
not track) a branch.  What I want changes depending on the circumstances.

One aspect I really like about submodules is the ease of tagging.  I can tag
the super-repo, and know that whenever I checkout that tag I'll always get
the corresponding versions of the submodules as they were when the tag was
made.  It would actually be disastrous, at least in my case, if the
submodules were at the latest HEAD of some branch instead.

This goes for almost any commit in the super-repo's history, not just the
tagged ones:  Whenever I checkout a historical committish, I want to get the
submodules as they were when that commit was made.  Even if I'm working at
the HEAD of some branch, often that branch is based on a historical commit
and I want to use the submodules as they were when that historical commit was
made.

All that said, I do think submodule branch tracking is useful.  Quite often a
development topic will change the super-repo and one or more submodules.  It
would be extremely helpful to do that work in a branch that spans the
super-repo and (a subset of) the submodules.  (In my mind this capability is
one of the main benefits Google's "repo" tool has over submodules.)

So, back to the issue at hand: Sometimes I want static (non-tracking)
submodules, and sometimes I want dynamic (tracking) submodules.  IMO, this
makes Ævar's proposed configuration-based approach impractical.  (Of course,
I'm not looking to replicate svn's externals...)

I'm not sure what the right approach is, but I have some thoughts:

 - Maybe "git branch" should be able to create submodule-spanning branches.

 - If so, then checkout, merge, pull and other branch-related commands should
honor submodule-spanning branches.  "checkout" in particular needs to
distinguish between when it's checking out an actual branch vs. some other
committish, and if the branch being checked out is submodule-spanning it
should checkout the latest HEAD of that branch for the submodules as well.

 - It *may* be good enough to assume that matching branch names in the
super-repo and the submodules are in fact submodule-spanning branches.

 - Automating all this is tricky.  In my super-repo I almost never want to
checkout the master HEAD of all my submodules.  In fact, many of my
submodules are big (e.g. they're different Linux kernels), and are only
needed when building particular things, so that checking all of them out at
once is almost always a huge waste of time.

All this is probably not the kind of feedback you were hoping for!  :)

		M.

Re: RFC: Making submodules "track" branches

[Jens Lehmann]

Am 08.06.2010 09:12, schrieb Johan Herland:
> - When switching branches in the superrepo, you sometimes also want to 
> switch branches in the submodule. This is signalled by changing the 
> submodules.subthing.branch variable in .gitmodules between the two branches. 
> However, it means that the submodule's update/pull operation must also be 
> done on 'checkout' in the superrepo.

Hm, I always want the submodules to switch branches along with the super-
project (I posted a RFC patch for that), but i can see other people don't
want that at all or just for some submodules. But am I wrong assuming that
it's either "switch branches in submodules too every time" or "never do
that" for a single submodule?


> - How to handle local/uncommitted (staged or unstaged) modifications in a 
> submodule when pulling or switching branches in the superrepo? The right 
> answer here is probably to do the same as in the no-submodule case, i.e. to 
> refuse if it would clobber/conflict with the local modifications.

Yup. I thing one goal for submodules is that they should blend in with
the superprojects as far as possible (unless configured to not to).


> - When you track submodule branches instead of commits, the actual commit 
> referenced in the superrepo is no longer as important (provided it's part of 
> the ancestry of the submodule branch you're tracking). However, diff/status 
> will still list the submodule as changed because you checked out a different 
> commit from what Git has recorded. This raises two concerns: (1) What 
> _should_ be considered "changed" from the diff/status perspective when 
> tracking submodule branches? and (2) When do you update the commit reference 
> in the submodule? "never" would work (since you're checking out a different 
> commit anyway), "always" would also work (for the same reason), but would 
> litter the superrepo history with submodule updates. There may be a better 
> alternative somewhere in between.

Don't record a commit in the first place, following a branch is not bound
to a special commit, so pretending to do that might do more harm than good.
Just putting the 0-hash there might be the solution.


> - If you want to give the illusion of "one big repo" then maybe it should 
> also be possible to trigger submodule commits from a superrepo commit? (i.e. 
> having a single toplevel "git commit" also trigger commits in submodules). 
> Some users will want to specify the commit message for each submodule 
> separately (IMHO the better approach), while some will want to give only one 
> commit message that is reused in every submodule commit.

Hm, personally I am fine with first committing in the submodules and then
in the superproject.

Re: RFC: Making submodules "track" branches

[Ævar Arnfjörð Bjarmason]

On Tue, Jun 8, 2010 at 15:34, Marc Branchaud <marcnarc@xiplink.com> wrote:
> On 10-06-08 03:12 AM, Johan Herland wrote:
>>
>> There are probably more issues that escape me now...
>
> Thanks for bringing this up!  I'm also very interested in this topic.
>
> The main issue I see is that I don't always want my submodules to track (or
> not track) a branch.  What I want changes depending on the circumstances.
>
> One aspect I really like about submodules is the ease of tagging.  I can tag
> the super-repo, and know that whenever I checkout that tag I'll always get
> the corresponding versions of the submodules as they were when the tag was
> made.  It would actually be disastrous, at least in my case, if the
> submodules were at the latest HEAD of some branch instead.
>
> This goes for almost any commit in the super-repo's history, not just the
> tagged ones:  Whenever I checkout a historical committish, I want to get the
> submodules as they were when that commit was made.  Even if I'm working at
> the HEAD of some branch, often that branch is based on a historical commit
> and I want to use the submodules as they were when that historical commit was
> made.
>
> All that said, I do think submodule branch tracking is useful.  Quite often a
> development topic will change the super-repo and one or more submodules.  It
> would be extremely helpful to do that work in a branch that spans the
> super-repo and (a subset of) the submodules.  (In my mind this capability is
> one of the main benefits Google's "repo" tool has over submodules.)
>
> So, back to the issue at hand: Sometimes I want static (non-tracking)
> submodules, and sometimes I want dynamic (tracking) submodules.  IMO, this
> makes Ævar's proposed configuration-based approach impractical.  (Of course,
> I'm not looking to replicate svn's externals...)

I'm proposing that you be able to configure how you want to handle
submodules on a per-submodule basis.

The exact semantics that I proposed may be impractical for some
reason, but the idea is that it'd be opt in. We'd perhaps have
multiple approaches (via config) to submodules, instead of the current
monolithic scheme.

So if you didn't want a svn:externals like "always track trunk"
repository you'd just not set your superproject up to treat the
submodule like that.

> I'm not sure what the right approach is, but I have some thoughts:
>
>  - Maybe "git branch" should be able to create submodule-spanning branches.
>
>  - If so, then checkout, merge, pull and other branch-related commands should
> honor submodule-spanning branches.  "checkout" in particular needs to
> distinguish between when it's checking out an actual branch vs. some other
> committish, and if the branch being checked out is submodule-spanning it
> should checkout the latest HEAD of that branch for the submodules as well.
>
>  - It *may* be good enough to assume that matching branch names in the
> super-repo and the submodules are in fact submodule-spanning branches.

That won't work for submodules that you don't control. I have a
repository that includes a lot of foreign code, they have a lot of
different names for their "main branch" between them. So it needs to
be configurable in the superproject.

>  - Automating all this is tricky.  In my super-repo I almost never want to
> checkout the master HEAD of all my submodules.  In fact, many of my
> submodules are big (e.g. they're different Linux kernels), and are only
> needed when building particular things, so that checking all of them out at
> once is almost always a huge waste of time.
>
> All this is probably not the kind of feedback you were hoping for!  :)

Re: RFC: Making submodules "track" branches

[Marc Branchaud]

On 10-06-08 12:09 PM, Ævar Arnfjörð Bjarmason wrote:
> On Tue, Jun 8, 2010 at 15:34, Marc Branchaud <marcnarc@xiplink.com> wrote:
>>
>> So, back to the issue at hand: Sometimes I want static (non-tracking)
>> submodules, and sometimes I want dynamic (tracking) submodules.  IMO, this
>> makes Ævar's proposed configuration-based approach impractical.  (Of course,
>> I'm not looking to replicate svn's externals...)
> 
> I'm proposing that you be able to configure how you want to handle
> submodules on a per-submodule basis.

Yes, and that's precisely the problem.  For a given submodule, sometimes it
should track a branch and sometimes it shouldn't.  Having to edit a
configuration to change that is impractical.

> The exact semantics that I proposed may be impractical for some
> reason, but the idea is that it'd be opt in. We'd perhaps have
> multiple approaches (via config) to submodules, instead of the current
> monolithic scheme.

Opting in or out can't just be a monolithic setting for each submodule.  A
submodule's branch tracking has to be on or off depending on the circumstances.

> So if you didn't want a svn:externals like "always track trunk"
> repository you'd just not set your superproject up to treat the
> submodule like that.

Yes, of course.

I guess what I'm saying is that duplicating svn's externals doesn't seem all
that useful to me and I'd rather see git do better.  I've no objection if
folks want to have such a feature, but to me it's not what "submodules
tracking branches" should be about.

>>  - It *may* be good enough to assume that matching branch names in the
>> super-repo and the submodules are in fact submodule-spanning branches.
> 
> That won't work for submodules that you don't control. I have a
> repository that includes a lot of foreign code, they have a lot of
> different names for their "main branch" between them. So it needs to
> be configurable in the superproject.

Good point.  I agree.

		M.

Re: RFC: Making submodules "track" branches

[Ævar Arnfjörð Bjarmason]

On Tue, Jun 8, 2010 at 19:32, Marc Branchaud <marcnarc@xiplink.com> wrote:
> On 10-06-08 12:09 PM, Ævar Arnfjörð Bjarmason wrote:
>> On Tue, Jun 8, 2010 at 15:34, Marc Branchaud <marcnarc@xiplink.com> wrote:
>>>
>>> So, back to the issue at hand: Sometimes I want static (non-tracking)
>>> submodules, and sometimes I want dynamic (tracking) submodules.  IMO, this
>>> makes Ævar's proposed configuration-based approach impractical.  (Of course,
>>> I'm not looking to replicate svn's externals...)
>>
>> I'm proposing that you be able to configure how you want to handle
>> submodules on a per-submodule basis.
>
> Yes, and that's precisely the problem.  For a given submodule, sometimes it
> should track a branch and sometimes it shouldn't.  Having to edit a
> configuration to change that is impractical.

See below.

>> The exact semantics that I proposed may be impractical for some
>> reason, but the idea is that it'd be opt in. We'd perhaps have
>> multiple approaches (via config) to submodules, instead of the current
>> monolithic scheme.
>
> Opting in or out can't just be a monolithic setting for each submodule.  A
> submodule's branch tracking has to be on or off depending on the circumstances.

I don't really get what the objection is exactly. How should "branch
tracking" be achieved do you think?

Anyway, right now what we track is set in a monolithic fashion by a
combination of a commit pointer in a tree and what's being versioned
in .gitmodules. If you want to change anything that's where you have
to do it.

Why should it be any different when the submodule isn't tracking a
specific commit? I.e. when it's "track the latest version of $thingy,
whatever that is", instead of "track version $version of $thingy".

>> So if you didn't want a svn:externals like "always track trunk"
>> repository you'd just not set your superproject up to treat the
>> submodule like that.
>
> Yes, of course.
>
> I guess what I'm saying is that duplicating svn's externals doesn't seem all
> that useful to me and I'd rather see git do better.  I've no objection if
> folks want to have such a feature, but to me it's not what "submodules
> tracking branches" should be about.

Obviously I have no objection to doing better, but how specifically
should that be done? If the semantics you want are "give me the latest
version of $URL, whatever that is" then the SVN semantics are pretty
good.

Re: RFC: Making submodules "track" branches

[Johan Herland]

On Tuesday 08 June 2010, Jens Lehmann wrote:
> Am 08.06.2010 09:12, schrieb Johan Herland:
> > - When switching branches in the superrepo, you sometimes also want to
> > switch branches in the submodule. This is signalled by changing the
> > submodules.subthing.branch variable in .gitmodules between the two
> > branches. However, it means that the submodule's update/pull operation
> > must also be done on 'checkout' in the superrepo.
> 
> Hm, I always want the submodules to switch branches along with the super-
> project (I posted a RFC patch for that), but i can see other people don't
> want that at all or just for some submodules. But am I wrong assuming
> that it's either "switch branches in submodules too every time" or
> "never do that" for a single submodule?

Well, the good thing is that keeping this config/info in .gitmodules (which 
is versioned-controlled along with the rest of the project) enables you to 
choose one or the other, or anything in between. For example, given a 
submodule "foo", say I want to keep it on its "master" branch when the 
super-repo is on _its_ master branch. The .gitmodules file on the super-
repo's "master" branch would contain:

    [submodule "foo"]
        path = foo
        url = git://url/to/foo/upstream
        branch = master

Now, if I create a new branch "topic" in the super-repo, the submodule would 
by default keep on tracking its "master" branch. If I want to track another 
branch, called "subtopic", inside "foo", I change .gitmodules on my super-
repo's "topic" branch to say:

    [submodule "foo"]
        path = foo
        url = git://url/to/foo/upstream
        branch = subtopic

Finally, if I have a branch "release" in the super-repo, in which I want to 
pin submodule "foo" to a specific commit, I change .gitmodules on the super-
repo's "release" branch to say:

    [submodule "foo"]
        path = foo
        url = git://url/to/foo/upstream

(i.e. no branch tracking), and then record the appropriate submodule commit 
the "old-fashioned" way.

The good thing with Ævar's approach is that this is all configurable per 
branch (indeed, per commit[1]) by editing your .gitmodules file.

> > - When you track submodule branches instead of commits, the actual
> > commit referenced in the superrepo is no longer as important (provided
> > it's part of the ancestry of the submodule branch you're tracking).
> > However, diff/status will still list the submodule as changed because
> > you checked out a different commit from what Git has recorded. This
> > raises two concerns: (1) What _should_ be considered "changed" from
> > the diff/status perspective when tracking submodule branches? and (2)
> > When do you update the commit reference in the submodule? "never"
> > would work (since you're checking out a different commit anyway),
> > "always" would also work (for the same reason), but would litter the
> > superrepo history with submodule updates. There may be a better
> > alternative somewhere in between.
> 
> Don't record a commit in the first place, following a branch is not bound
> to a special commit, so pretending to do that might do more harm than
> good. Just putting the 0-hash there might be the solution.

Interesting. Will the object parsing machinery handle that without hiccups? 
What if an older Git version tries to checkout/update a submodule with a 0-
hash?

> > - If you want to give the illusion of "one big repo" then maybe it
> > should also be possible to trigger submodule commits from a superrepo
> > commit? (i.e. having a single toplevel "git commit" also trigger
> > commits in submodules). Some users will want to specify the commit
> > message for each submodule separately (IMHO the better approach),
> > while some will want to give only one commit message that is reused in
> > every submodule commit.
> 
> Hm, personally I am fine with first committing in the submodules and then
> in the superproject.

Me too, but I suspect that if you draw the "one big repo" approach to its 
logical conclusion, there will be some demand for recursive commits.


Have fun! :)

...Johan


[1]: Say your submodule usually tracks a branch, but you're creating some 
tag in the super-repo, and you want that tag to uniquely identify the 
submodule. You achieve this by making sure the tagged commit removes the 
relevant "branch = whatever" line from .gitmodules, and records the 
appropriate submodule version in the super-repo tree. Then, you can revert 
the .gitmodules change on the next commit to resume tracking the submodule 
branch.

Now, whenever you checkout the tag, you will always get the exact same 
version of the submodule, although the submodule otherwise tracks some 
branch.


-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: RFC: Making submodules "track" branches

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Don't record a commit in the first place, following a branch is not bound
> to a special commit, so pretending to do that might do more harm than good.
> Just putting the 0-hash there might be the solution.

Ugh.  Even though I understand that in some scenarios you would want to
say "I don't care what commit is used for this submodule---just use the
tip of the branch 'fred'", I don't think you want to use 0{40} in the
superproject.  I think it would be Ok to add such a note to .gitmodules in
the superproject, but I also think we should still record which _exact_
commit was used to test and validate such a commit in the superproject
when it was made.

If you clone a superproject that contains such a submodule from an
upstream, keeping them up-to-date while working on your own change, it is
perfectly fine to choose to use whatever random commit that happens to be
at the tip of 'fred' branch in a submodule (and needless to say, that
commit might be your own commit that nobody else has, if you have been
actively working in that submodule, that you haven't published), that is
different from what the person who created the commit in the superproject
had.  But at least you would need to be able to tell that the result of a
build from such a state is different from what the superproject had.
Recording 0{40} would make the information contained in the superproject
tree meaningless.

Wouldn't it be enough to say --ignore-submodules for your day-to-day work,
without lying in the gitlink entry in the superproject tree?  An entry
"submodule.foo.branch = fred" in your .gitmodules will still tell your
local git to update the submodule worktree to work on 'fred' branch.  At
least, an arrangement like that would allow the build infrastructure to
use --no-ignore-submodules when running its equivalent of GIT-VERSION-GEN
to notice that what you are building is using something different from
what the superproject specified to use in the submodule, while not bugging
you with differences you do not care about (or you already know about and
are irrelevant to the change you are working on).

Re: RFC: Making submodules "track" branches

[Ævar Arnfjörð Bjarmason]

On Tue, Jun 8, 2010 at 23:09, Junio C Hamano <gitster@pobox.com> wrote:
> Wouldn't it be enough to say --ignore-submodules for your day-to-day work,
> without lying in the gitlink entry in the superproject tree?  An entry
> "submodule.foo.branch = fred" in your .gitmodules will still tell your
> local git to update the submodule worktree to work on 'fred' branch.  At
> least, an arrangement like that would allow the build infrastructure to
> use --no-ignore-submodules when running its equivalent of GIT-VERSION-GEN
> to notice that what you are building is using something different from
> what the superproject specified to use in the submodule, while not bugging
> you with differences you do not care about (or you already know about and
> are irrelevant to the change you are working on).

Yes I think that's even better, to have no entry in the superproject's
tree at all, and just a repo/branch pair in .gitmodules.

Less confusion and the same features.

Re: RFC: Making submodules "track" branches

[Jens Lehmann]

Am 09.06.2010 01:19, schrieb Ævar Arnfjörð Bjarmason:
> On Tue, Jun 8, 2010 at 23:09, Junio C Hamano <gitster@pobox.com> wrote:
>> Wouldn't it be enough to say --ignore-submodules for your day-to-day work,
>> without lying in the gitlink entry in the superproject tree?  An entry
>> "submodule.foo.branch = fred" in your .gitmodules will still tell your
>> local git to update the submodule worktree to work on 'fred' branch.  At
>> least, an arrangement like that would allow the build infrastructure to
>> use --no-ignore-submodules when running its equivalent of GIT-VERSION-GEN
>> to notice that what you are building is using something different from
>> what the superproject specified to use in the submodule, while not bugging
>> you with differences you do not care about (or you already know about and
>> are irrelevant to the change you are working on).
> 
> Yes I think that's even better, to have no entry in the superproject's
> tree at all, and just a repo/branch pair in .gitmodules.

Thats not how I understood Junio proposal, but an alternative to using
0{40} could be to just drop the submodule entry from the tree. You get the
same result, but maybe less problems with older versions of git.


> Less confusion and the same features.

Not knowing the version of a submodule looks to me like a very powerful
source of confusion, but maybe thats just me ;-)

Re: RFC: Making submodules "track" branches

[Jens Lehmann]

Am 09.06.2010 01:09, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> Don't record a commit in the first place, following a branch is not bound
>> to a special commit, so pretending to do that might do more harm than good.
>> Just putting the 0-hash there might be the solution.
> 
> Ugh.  Even though I understand that in some scenarios you would want to
> say "I don't care what commit is used for this submodule---just use the
> tip of the branch 'fred'", I don't think you want to use 0{40} in the
> superproject.  I think it would be Ok to add such a note to .gitmodules in
> the superproject, but I also think we should still record which _exact_
> commit was used to test and validate such a commit in the superproject
> when it was made.

I think we are in violent agreement here. But I as far as understood the
always-tip mode (and I might be wrong here as I never used something like
SVN Externals) it is intended to not be able to tell which exact version
of the submodules branch was used. Otherwise you could just update the
branch in the submodule and commit that in the superproject, which is what
people do not seem to want (please correct me if I am wrong).

Under this assumption it seems to me that it doesn't make sense to record
anything but 0{40}, as this tells people "this submodule was somewhere at
the tip of <branch>, but we can't say where exactly"). Or maybe don't add
the submodule to the tree at all, like Ævar proposed. Same outcome, maybe
even easier to do.

But I always have the feeling there is something I don't get when people
talk about the always-tip mode, so maybe the potential users of such a
feature should speak up now.

Re: RFC: Making submodules "track" branches

[Jens Lehmann]

Am 08.06.2010 23:52, schrieb Johan Herland:
> The good thing with Ævar's approach is that this is all configurable per 
> branch (indeed, per commit[1]) by editing your .gitmodules file.

Yep, I think this is the sane way to do that.


> Interesting. Will the object parsing machinery handle that without hiccups? 
> What if an older Git version tries to checkout/update a submodule with a 0-
> hash?

Maybe Ævar's idea of dropping such a submodule from the tree is better.


> Me too, but I suspect that if you draw the "one big repo" approach to its 
> logical conclusion, there will be some demand for recursive commits.

You may be right here. But as submodules often have a detached HEAD, this
might get interesting ;-)


> [1]: Say your submodule usually tracks a branch, but you're creating some 
> tag in the super-repo, and you want that tag to uniquely identify the 
> submodule. You achieve this by making sure the tagged commit removes the 
> relevant "branch = whatever" line from .gitmodules, and records the 
> appropriate submodule version in the super-repo tree. Then, you can revert 
> the .gitmodules change on the next commit to resume tracking the submodule 
> branch.
> 
> Now, whenever you checkout the tag, you will always get the exact same 
> version of the submodule, although the submodule otherwise tracks some 
> branch.

Won't work anymore when we would use 0{40} or drop it from the tree.
AFAICS always-tip and referencing a certain commit don't mix well.

Re: RFC: Making submodules "track" branches

[Johan Herland]

On Wednesday 09 June 2010, Jens Lehmann wrote:
> Am 08.06.2010 23:52, schrieb Johan Herland:
> > The good thing with Ævar's approach is that this is all configurable
> > per branch (indeed, per commit[1]) by editing your .gitmodules file.
> 
> Yep, I think this is the sane way to do that.
> 
> > Interesting. Will the object parsing machinery handle that without
> > hiccups? What if an older Git version tries to checkout/update a
> > submodule with a 0- hash?
> 
> Maybe Ævar's idea of dropping such a submodule from the tree is better.

Agreed. That will of course cause older Git versions to skip the submodule 
altogether, which is probably the safest failure mode.

> > Me too, but I suspect that if you draw the "one big repo" approach to
> > its logical conclusion, there will be some demand for recursive
> > commits.
> 
> You may be right here. But as submodules often have a detached HEAD, this
> might get interesting ;-)

Yes, trying to recursively commit across a submodule with detached HEAD 
should obviously fail (at least by default). But as long as a local branch 
is checked out in the submodule (which is not necessarily the same as having 
the submodule _track_ that branch), a recursive commit should be relatively 
straightforward.

> > [1]: Say your submodule usually tracks a branch, but you're creating
> > some tag in the super-repo, and you want that tag to uniquely identify
> > the submodule. You achieve this by making sure the tagged commit
> > removes the relevant "branch = whatever" line from .gitmodules, and
> > records the appropriate submodule version in the super-repo tree.
> > Then, you can revert the .gitmodules change on the next commit to
> > resume tracking the submodule branch.
> > 
> > Now, whenever you checkout the tag, you will always get the exact same
> > version of the submodule, although the submodule otherwise tracks some
> > branch.
> 
> Won't work anymore when we would use 0{40} or drop it from the tree.
> AFAICS always-tip and referencing a certain commit don't mix well.

AFAICS, it would still work as long as it exists in the tree for that 
specific commit (but is missing/0{40} in other commits).

We're not mixing "always-tip" and "exact-commit" in the same commit. We use 
"always-tip" in regular commits, and then temporarily switch to "exact-
commit" in the commits where a certain submodule version is required.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: RFC: Making submodules "track" branches

[Steven Michalske]

On Jun 9, 2010, at 1:22 AM, Johan Herland wrote:

> On Wednesday 09 June 2010, Jens Lehmann wrote:
>> Am 08.06.2010 23:52, schrieb Johan Herland:
>>> The good thing with Ævar's approach is that this is all configurable
>>> per branch (indeed, per commit[1]) by editing your .gitmodules file.
>>
>> Yep, I think this is the sane way to do that.
>>
>>> Interesting. Will the object parsing machinery handle that without
>>> hiccups? What if an older Git version tries to checkout/update a
>>> submodule with a 0- hash?
>>
>> Maybe Ævar's idea of dropping such a submodule from the tree is  
>> better.
>
> Agreed. That will of course cause older Git versions to skip the  
> submodule
> altogether, which is probably the safest failure mode.
>
>>> Me too, but I suspect that if you draw the "one big repo" approach  
>>> to
>>> its logical conclusion, there will be some demand for recursive
>>> commits.
>>
>> You may be right here. But as submodules often have a detached  
>> HEAD, this
>> might get interesting ;-)
>
> Yes, trying to recursively commit across a submodule with detached  
> HEAD
> should obviously fail (at least by default). But as long as a local  
> branch
> is checked out in the submodule (which is not necessarily the same  
> as having
> the submodule _track_ that branch), a recursive commit should be  
> relatively
> straightforward.
>
>>> [1]: Say your submodule usually tracks a branch, but you're creating
>>> some tag in the super-repo, and you want that tag to uniquely  
>>> identify
>>> the submodule. You achieve this by making sure the tagged commit
>>> removes the relevant "branch = whatever" line from .gitmodules, and
>>> records the appropriate submodule version in the super-repo tree.
>>> Then, you can revert the .gitmodules change on the next commit to
>>> resume tracking the submodule branch.
>>>
>>> Now, whenever you checkout the tag, you will always get the exact  
>>> same
>>> version of the submodule, although the submodule otherwise tracks  
>>> some
>>> branch.
>>
>> Won't work anymore when we would use 0{40} or drop it from the tree.
>> AFAICS always-tip and referencing a certain commit don't mix well.
>
> AFAICS, it would still work as long as it exists in the tree for that
> specific commit (but is missing/0{40} in other commits).
>
> We're not mixing "always-tip" and "exact-commit" in the same commit.  
> We use
> "always-tip" in regular commits, and then temporarily switch to  
> "exact-
> commit" in the commits where a certain submodule version is required.
>
When making a tag, could the notes system be used for marking what  
commit was exactly on the submodule, perhaps include the closest  
remote commits as well?


Something like

Submodule Status:
	["foo"]
		branch = subtopic:SHA

This assumes that git notes are shared/cloned......


Other thoughts.


Things that should still work with tracking submodules.
	- bisect   - Must be able to identify that a submodule change  
introduced the bug.
	- archive  - Should it use the version from the commit, or the latest?
	- rebase   - update all of the submodule commits?
	- checkout - tip vs commit
	- reset --hard - Good question... not sure.... probably depend on tip  
vs commit like checkout.
	- More????


I would rather the submodule entree in the tree be always updated to  
what is in the submodule on the commit, so that the history is always  
there.  Then actions updating the repository from remotes  
automatically pull the latest version.  I feel that the submodule if  
automatically be pulled, merged, etc, than the submodule should get a  
commit, with the message about this being an automatic update of the  
submodule.  Checking out is a different story.... checking out a  
branch tip of the super gets the latest tip from the submodule.  When  
you commit, the submodule gets it's auto commit, then a second commit  
for the code changes.  checking out a previous revision should  
probably put the sub module to the state it was in at that point in  
time.  Creating a branch and adding new content would update according  
to the rules.  but show the change of the subproject as from the  
super's at the branch point, not the tip.

This way older gits have a submodule to work with and newer gits will  
do the right thing.

Example:

s-y-y-z
A-B-C-D
\
  \F-G
s-z-z

F is branched when the latest sub module is at z  but shows the change  
from s not z because A the parent of F was created with the submodule  
at s

Situational Example:

I am developing away and as I progress in development I get a  
regression bug, so I run git bisect from the last stable release with  
out this bug, and it starts bisecting away.

In the mode where we don't store the state of the project I can't  
bisect the changes against the subproject, where my bug might have  
been introduced from.

So that issue should be probably handled in the git bisect code, that  
is "Make git bisect submodule aware"  in more verbose terms, when  
bisecting a super project the sub modules should undergo bisection as  
well.  This is a permutation that will expand rapidly, but some  
thoughts on how to dig into the bisection issues.
This is another email ;-)


Rebase:
	With the auto commit of submodule scheme, a rebase would change the  
tracking branches to the latest of the tracked version. and auto merge  
and record the previous submodule revision in the commit message of  
the submodule auto commit.

Checkout with nonexistant submodule sha:
	This is the case where the submodules ref was not pushed publicly,  
so, the contents are not available.  You get a nice warning and the  
tip of the submodules branch gets checked out for that submodule.

Steve

Re: RFC: Making submodules "track" branches

[Marc Branchaud]

On 10-06-08 04:23 PM, Ævar Arnfjörð Bjarmason wrote:
> On Tue, Jun 8, 2010 at 19:32, Marc Branchaud <marcnarc@xiplink.com> wrote:
>>
>> Opting in or out can't just be a monolithic setting for each submodule.  A
>> submodule's branch tracking has to be on or off depending on the circumstances.
> 
> I don't really get what the objection is exactly. How should "branch
> tracking" be achieved do you think?

Well, I outlined some ideas in my first message in this thread...

>> I guess what I'm saying is that duplicating svn's externals doesn't seem all
>> that useful to me and I'd rather see git do better.  I've no objection if
>> folks want to have such a feature, but to me it's not what "submodules
>> tracking branches" should be about.
> 
> Obviously I have no objection to doing better, but how specifically
> should that be done? If the semantics you want are "give me the latest
> version of $URL, whatever that is" then the SVN semantics are pretty
> good.

The nuance is that the semantics aren't "*always* give me the latest version
of $URL" but rather "*sometimes* give me the latest version of $URL."

Anyway, others have raised issues that touch on this, and I'm happy to just
see where those discussions go.

		M.

Re: RFC: Making submodules "track" branches

[Johan Herland]

On Wednesday 09 June 2010, Steven Michalske wrote:
> On Jun 9, 2010, at 1:22 AM, Johan Herland wrote:
> > On Wednesday 09 June 2010, Jens Lehmann wrote:
> >> Am 08.06.2010 23:52, schrieb Johan Herland:
> >>> [1]: Say your submodule usually tracks a branch, but you're
> >>> creating some tag in the super-repo, and you want that tag to
> >>> uniquely identify
> >>> the submodule. You achieve this by making sure the tagged commit
> >>> removes the relevant "branch = whatever" line from .gitmodules,
> >>> and records the appropriate submodule version in the super-repo
> >>> tree. Then, you can revert the .gitmodules change on the next
> >>> commit to resume tracking the submodule branch.
> >>>
> >>> Now, whenever you checkout the tag, you will always get the exact
> >>> same
> >>> version of the submodule, although the submodule otherwise tracks
> >>> some
> >>> branch.
> >>
> >> Won't work anymore when we would use 0{40} or drop it from the
> >> tree. AFAICS always-tip and referencing a certain commit don't mix
> >> well.
> >
> > AFAICS, it would still work as long as it exists in the tree for
> > that specific commit (but is missing/0{40} in other commits).
> >
> > We're not mixing "always-tip" and "exact-commit" in the same
> > commit. We use "always-tip" in regular commits, and then temporarily
> > switch to "exact-commit" in the commits where a certain submodule
> > version is required.
>
> When making a tag, could the notes system be used for marking what
> commit was exactly on the submodule, perhaps include the closest
> remote commits as well?
>
>
> Something like
>
> Submodule Status:
> 	["foo"]
> 		branch = subtopic:SHA
>
> This assumes that git notes are shared/cloned......

I don't think this is a good use of notes. We already have an 
infrastructure for recording the exact submodule version used (the 
submodule entry in the superproject tree), and I see no reason why we 
should not use that in this case.

> Other thoughts.
>
>
> Things that should still work with tracking submodules.
> 	- bisect   - Must be able to identify that a submodule change
> introduced the bug.

When tracking submodule commits (the default), this is just a matter of 
recursively applying the bisect operation to the good/bad submodule 
commits identified by the superproject bisect.

When tracking submodule branches, you can no longer use bisect in the 
superproject to find bugs in the submodule, since it would always check 
out the same (i.e. latest) version of the submodule (I'm assuming that 
we're not switching submodule branches in .gitmodules here...).

In the tag scenario I present which is quoted above, we're switching 
between tracking submodule commits (in the commits that are tagged), 
and tracking submodule branches (in the other commits), so bisect would 
be next to useless (unless you limit the bisect to only look at the 
tagged commits).

Alternatively, we could reconsider the question I asked Ævar initially 
in this thread:

<quote>
When do you update the commit reference in the submodule? "never" would 
work (since you're checking out a different commit anyway), "always" 
would also work (for the same reason), but would litter the superrepo 
history with submodule updates. There may be a better alternative 
somewhere in between.
</quote>

So if we _always_ (or often) update the submodule commit reference in 
the superproject, then we could disregard the branch tracking while 
bisecting and checkout the commit referenced from the super-repo. That 
would hopefully be as useful as if we'd tracked submodule commits 
explicitly.

But again, the more often we update the submodule commit reference 
(while still primarily tracking a _branch_ in the submodule), the more 
we "litter" the superproject history with these updates.

> 	- archive  - Should it use the version from the commit, or the
> latest?

Good question. In principle, since you've explicitly asked to track a 
branch, the only assumption Git can make is that you really want the 
_latest_ version of that subdmodule/branch.

Again we're back to the question of if/how often we record update 
submodule commits in the superproject, when the submodule primarily 
tracks a branch: If we're religious about recording our submodule 
commit references in the superproject, then we can temporarily 
disregard the branch tracking in order to get a somewhat realistic 
submodule update history.

>       - rebase   - update all of the submodule commits? 

Again, depends. If you really want branch-tracking, rebase will not 
change the submodule (unless you change which branch is tracked 
in .gitmodules).

Here, I don't see a good rationale for updating submodules though. If 
you're tracking submodule branches, then a superproject rebase won't 
affect what's at the tip of a submodule branch.

...unless you're talking about rebasing "foo" onto "bar in the 
superproject causing a corresponding rebase of "subfoo" onto "subbar" 
in the submodule, which is a whole 'nother can of worms...

> 	- checkout - tip vs commit

No question. If you've specified branch-tracking in .gitmodules, you get 
tip, otherwise you get commit.

> 	- reset --hard - Good question... not sure.... probably depend on
> tip vs commit like checkout.

As above, a reset --hard in the superproject does not affect what's at 
the tip of some submodule branch, so if you've chosen to track 
submodule branches, a reset --hard will not touch the submodule (unless 
the reset changes .gitmodules, obviously)

> 	- More????
>
> I would rather the submodule entree in the tree be always updated to
> what is in the submodule on the commit, so that the history is always
> there.

I see your point, as this would enable you to temporarily disable 
branch-tracking after the fact (typically for debugging purposes). But 
we still have to weigh its usefulness (in practice) against the cost of 
adding these extra commit references.

After all, even if Git does not do this automatically, you could still 
fairly easily add a pre-commit hook in the superproject that stages all 
submodule references.

> Then actions updating the repository from remotes 
> automatically pull the latest version.  I feel that the submodule if
> automatically be pulled, merged, etc, than the submodule should get a
> commit, with the message about this being an automatic update of the
> submodule.  Checking out is a different story.... checking out a
> branch tip of the super gets the latest tip from the submodule.  When
> you commit, the submodule gets it's auto commit, then a second commit
> for the code changes.  checking out a previous revision should
> probably put the sub module to the state it was in at that point in
> time.  Creating a branch and adding new content would update
> according to the rules.  but show the change of the subproject as
> from the super's at the branch point, not the tip.
>
> This way older gits have a submodule to work with and newer gits will
> do the right thing.
>
> Example:
>
> s-y-y-z
> A-B-C-D
> \
>   \F-G
> s-z-z
>
> F is branched when the latest sub module is at z  but shows the
> change from s not z because A the parent of F was created with the
> submodule at s
>
> Situational Example:
>
> I am developing away and as I progress in development I get a
> regression bug, so I run git bisect from the last stable release with
> out this bug, and it starts bisecting away.
>
> In the mode where we don't store the state of the project I can't
> bisect the changes against the subproject, where my bug might have
> been introduced from.

No, you would have to run a separate bisect in the subproject. What's 
wrong about that?

> So that issue should be probably handled in the git bisect code, that
> is "Make git bisect submodule aware"  in more verbose terms, when
> bisecting a super project the sub modules should undergo bisection as
> well.  This is a permutation that will expand rapidly, but some
> thoughts on how to dig into the bisection issues.
> This is another email ;-)
>
>
> Rebase:
> 	With the auto commit of submodule scheme, a rebase would change the
> tracking branches to the latest of the tracked version. and auto
> merge and record the previous submodule revision in the commit
> message of the submodule auto commit.

Not sure I understand what you're getting at here. Say you're tracking 
submodule branches (in the superproject's .gitmodules). Now, you do a 
rebase in the superproject. If the rebase does not change .gitmodules 
(i.e. which branch is tracked in the submodule), then there is nothing 
to be done in the submodule (it has already checked out the tip of that 
branch). If the rebase _does_ change .gitmodules - let's say there's a 
conflict in which submodule branch to track - then you first resolve 
that conflict in the superproject, to track the appropriate submodule 
branch. Then the tip of that branch is simply checked out in the 
submodule. You may want to record this update in the superproject, 
either by recording a separate commit, or by amending into the rebased 
commit.

Now, it might be that the correct resolution of the superproject's 
rebase conflict requires a nested rebase in the submodule, but this is 
certainly not a conclusion that Git can reach independently, so you 
will have to do the nested rebase manually, and then record the rebased 
branch in the superproject's .gitmodules (and optionally commit the 
updated submodule reference).

> Checkout with nonexistant submodule sha:
> 	This is the case where the submodules ref was not pushed publicly,
> so, the contents are not available.  You get a nice warning and the
> tip of the submodules branch gets checked out for that submodule.

If the .gitmodules specifies branch "foo" to be checked out in the 
submodule, and branch "foo" does not exist in the submodule, then 
that's the same type of error as if the superproject specifies a commit 
SHA1 for the submodule, and that commit does not exist in the 
submodule.

Granted, in the branch case, we can make it more network/remote-aware, 
by checking for both "foo" and "origin/foo" before giving up.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: RFC: Making submodules "track" branches

[Marc Branchaud]

On 10-06-09 03:15 AM, Jens Lehmann wrote:
> Am 09.06.2010 01:09, schrieb Junio C Hamano:
>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>
>>> Don't record a commit in the first place, following a branch is not bound
>>> to a special commit, so pretending to do that might do more harm than good.
>>> Just putting the 0-hash there might be the solution.
>>
>> Ugh.  Even though I understand that in some scenarios you would want to
>> say "I don't care what commit is used for this submodule---just use the
>> tip of the branch 'fred'", I don't think you want to use 0{40} in the
>> superproject.  I think it would be Ok to add such a note to .gitmodules in
>> the superproject, but I also think we should still record which _exact_
>> commit was used to test and validate such a commit in the superproject
>> when it was made.
> 
> I think we are in violent agreement here.

I too am in this camp.

If a submodule is tracking the tip of a branch, I think it's vital that
checking out the superproject's HEAD@{3 months ago} gives you the submodule
as it was in the superproject 3 months ago.  Back then, it may have been
tracking a different branch.  It may not have been tracking a branch at all.
 It may have been using a completely different repository altogether.

It's hard for me to see the utility of having the submodule reflect the
tip-of-some-branch-as-of-today when I'm looking at 3-month-old code in the
superproject.

AFAICT, Ævar's original proposal does the right thing here, because a
submodule tracking a branch would look dirty in the superproject if the
branch's HEAD doesn't match the commit ID recorded in the superproject.  So
"submodule update" would restore the submodule's state to what the
superproject says it should be.

I don't think I mind dirty branch-tracking submodules, but folks seem to find
it distasteful.  However, I believe all the proposals made so far to address
it break what I call the superproject's "historical consistency."

I wish I could come up with some way to reconcile clean branch-tracking
submodules with historical consistency, but alas my imagination is so far too
limited.  :(

		M.

Re: RFC: Making submodules "track" branches

[Ævar Arnfjörð Bjarmason]

On Wed, Jun 9, 2010 at 15:36, Marc Branchaud <marcnarc@xiplink.com> wrote:
> I wish I could come up with some way to reconcile clean branch-tracking
> submodules with historical consistency, but alas my imagination is so far too
> limited.  :(

I think the two concepts are fundimentally at odds with each other,
and that that's completely fine.

Sometimes you're promiscuous enough with your history that you don't
care about being able to go back in time, beyond checking out both
trees as they were at some given time that is. As Johan and others
point out above you could get around that with tags if you wanted
snapshots.

I think we might actually have several different modes of operation:

  * Disconnected head + commit sha1 in the superproject's tree: This
   is what we have now.

  * The same, but make it branch aware. I've scripted this locally
    with the $toplevel patch to git-submodule that started this
    thread. But it could be expanded.

    It would be really neat for example to do:

        # Or some shorter way of doing this, perhaps even with
        # git-pull
        git submodule foreach 'git fetch'

        # Tells you that "submodule xyz which you've pinned to SHA1SUM
        # on the FOOBAR branch is 20 commits behind the upstream
        # FOOBAR branch"
        git status --submodules

    You'd still have to take action to update the module and move the
    SHA1SUM in the parent project, but something like this would make
    cases where you've e.g. included a lot of plugins in your project,
    and would like Git to tell you if they get new updates.

  * Branch-only: What I proposed in this thread. It's certainly not
    for everyone, but there's a lot of cases where you just want a
    quick meta-repository but aren't very interested in 100%
    historical consistency.

  * More? Actually if we're doing multiple strategies I see no reason
    not to e.g. include a foreign scm interface. That would be really
    useful to some projects that are in a SVN -> Git transition:

       [submodule "svn-lib"]
           ;; type defaults to git
           type = svn
           path = src/svn-lib
           url = svn://example.net/path/to/include
           ;; driver-specific attributes
           svn:revision = r54238

Re: RFC: Making submodules "track" branches

[nottrobin]

Did any of this ever find its way into the submodule core? I'd like to have a
submodule that tracks a branch.



--
View this message in context: http://git.661346.n2.nabble.com/RFC-Making-submodules-track-branches-tp5151566p7571610.html
Sent from the git mailing list archive at Nabble.com.

Re: RFC: Making submodules "track" branches

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 441 bytes --]

On Tue, Nov 20, 2012 at 03:16:35AM -0800, nottrobin wrote:
> Did any of this ever find its way into the submodule core? I'd like
> to have a submodule that tracks a branch.

In progress.  See:

  http://thread.gmane.org/gmane.comp.version-control.git/208254

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]