Reference a submodule branch instead of a commit

Reference a submodule branch instead of a commit

[Jeremy Morton]

At the moment, supermodules must reference a given commit in each of 
its submodules.  If one is in control of a submodule and it changes on 
a regular basis, this can cause a lot of overhead with "submodule 
updated" commits in the supermodule.  It would be useful of git allows 
the option of referencing a submodule's branch instead of a given 
submodule commit.  How about adding this functionality?

-- 
Best regards,
Jeremy Morton (Jez)

Re: Reference a submodule branch instead of a commit

[Junio C Hamano]

Jeremy Morton <admin@game-point.net> writes:

> At the moment, supermodules must reference a given commit in each of
> its submodules.  If one is in control of a submodule and it changes on
> a regular basis, this can cause a lot of overhead with "submodule
> updated" commits in the supermodule.  It would be useful of git allows
> the option of referencing a submodule's branch instead of a given
> submodule commit.  How about adding this functionality?

When somebody downstream fetches from your superproject and grabs
the set of submodules, how would s/he know what _exact_ state you
meant to record?  When s/he says "I have your superproject commit X,
which binds submodule's branch Y at path sub/, and it simply does
not work.  Your project is broken", how do you go about reproducing
the exact state s/he had trouble with to help her/him?

The only thing s/he knows is that the commit used from the submodule
must be one of the commits that was on branch Y at some point in
time, hopefully close to the timestamp recorded in the commit in the
superproject.  And your record in the history of the superproject
does not tell you more than that, so you wouldn't have any idea
better than what s/he already has to help.

Hence, such a "functionality" will never happen, at least in the
exact form you are describing.

It is conceivable to add some feature that allows you to squelch the
report that the submodule recorded in your superproject is not up to
date from "git status" etc. to help those who thinks it is OK to not
bind the latest submodule commit to the superproject all the time,
though.

Re: Reference a submodule branch instead of a commit

[Heiko Voigt]

On Mon, Oct 03, 2016 at 12:00:45PM -0700, Junio C Hamano wrote:
> Jeremy Morton <admin@game-point.net> writes:
> 
> > At the moment, supermodules must reference a given commit in each of
> > its submodules.  If one is in control of a submodule and it changes on
> > a regular basis, this can cause a lot of overhead with "submodule
> > updated" commits in the supermodule.  It would be useful of git allows
> > the option of referencing a submodule's branch instead of a given
> > submodule commit.  How about adding this functionality?
> 
> When somebody downstream fetches from your superproject and grabs
> the set of submodules, how would s/he know what _exact_ state you
> meant to record?  When s/he says "I have your superproject commit X,
> which binds submodule's branch Y at path sub/, and it simply does
> not work.  Your project is broken", how do you go about reproducing
> the exact state s/he had trouble with to help her/him?
> 
> The only thing s/he knows is that the commit used from the submodule
> must be one of the commits that was on branch Y at some point in
> time, hopefully close to the timestamp recorded in the commit in the
> superproject.  And your record in the history of the superproject
> does not tell you more than that, so you wouldn't have any idea
> better than what s/he already has to help.
> 
> Hence, such a "functionality" will never happen, at least in the
> exact form you are describing.
> 
> It is conceivable to add some feature that allows you to squelch the
> report that the submodule recorded in your superproject is not up to
> date from "git status" etc. to help those who thinks it is OK to not
> bind the latest submodule commit to the superproject all the time,
> though.

We already have options to support these kinds of workflows. Look at the
option '--remote' for 'git submodule update'.

You then only have to commit the submodule if you do not want to see it
as dirty locally, but you will always get the tip of a remote tracking
branch when updating.

Cheers Heiko

Re: Reference a submodule branch instead of a commit

[Stefan Beller]

>
> We already have options to support these kinds of workflows. Look at the
> option '--remote' for 'git submodule update'.
>
> You then only have to commit the submodule if you do not want to see it
> as dirty locally, but you will always get the tip of a remote tracking
> branch when updating.

I wonder if we could make that convenient for users by not tracking
the submodule,
i.e.
* we have the information in the .gitmodules file
* the path itself is in the .gitignore
* no tree entry

Then you can update to the remote latest branch, without Git reporting
a dirty submodule locally, in fact it reports nothing for the submodule.

It sounds like a hack, but maybe it's worth looking into that when
people want to see that workflow.

Re: Reference a submodule branch instead of a commit

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

>>
>> We already have options to support these kinds of workflows. Look at the
>> option '--remote' for 'git submodule update'.
>>
>> You then only have to commit the submodule if you do not want to see it
>> as dirty locally, but you will always get the tip of a remote tracking
>> branch when updating.
>
> I wonder if we could make that convenient for users by not tracking
> the submodule,
> i.e.
> * we have the information in the .gitmodules file
> * the path itself is in the .gitignore
> * no tree entry
>
> Then you can update to the remote latest branch, without Git reporting
> a dirty submodule locally, in fact it reports nothing for the submodule.
>
> It sounds like a hack, but maybe it's worth looking into that when
> people want to see that workflow.

It IS a hack.  

But if you do not touch .git<anything> file and instead say "clone
this other project at that path yourself" in README, that would
probably be sufficient.

Re: Reference a submodule branch instead of a commit

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Stefan Beller <sbeller@google.com> writes:
>
>> I wonder if we could make that convenient for users by not tracking
>> the submodule,
>> i.e.
>> * we have the information in the .gitmodules file
>> * the path itself is in the .gitignore
>> * no tree entry
>>
>> Then you can update to the remote latest branch, without Git reporting
>> a dirty submodule locally, in fact it reports nothing for the submodule.
>>
>> It sounds like a hack, but maybe it's worth looking into that when
>> people want to see that workflow.
>
> It IS a hack.  
>
> But if you do not touch .git<anything> file and instead say "clone
> this other project at that path yourself" in README, that would
> probably be sufficient.

eh,... hit send too early.

It IS a hack, but having this information in .git<something> would
mean that it can be forced to be in machine readable form, unlike a
mention in README.  I do not know if the .gitmodules/.gitignore
combination is a sensible thing to use, but it does smell like a
potentially useful hack.

Re: Reference a submodule branch instead of a commit

[Heiko Voigt]

On Tue, Oct 04, 2016 at 12:01:13PM -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Stefan Beller <sbeller@google.com> writes:
> >
> >> I wonder if we could make that convenient for users by not tracking
> >> the submodule,
> >> i.e.
> >> * we have the information in the .gitmodules file
> >> * the path itself is in the .gitignore
> >> * no tree entry
> >>
> >> Then you can update to the remote latest branch, without Git reporting
> >> a dirty submodule locally, in fact it reports nothing for the submodule.
> >>
> >> It sounds like a hack, but maybe it's worth looking into that when
> >> people want to see that workflow.
> >
> > It IS a hack.  
> >
> > But if you do not touch .git<anything> file and instead say "clone
> > this other project at that path yourself" in README, that would
> > probably be sufficient.
> 
> eh,... hit send too early.
> 
> It IS a hack, but having this information in .git<something> would
> mean that it can be forced to be in machine readable form, unlike a
> mention in README.  I do not know if the .gitmodules/.gitignore
> combination is a sensible thing to use, but it does smell like a
> potentially useful hack.

IIRC the tree entries are the reference for submodules in the code. We
are iterating over the tree entries in many places so that change does
not seem so easy to me.

But you are right maybe we should stop arguing against this workflow and
just let people use it until they find out whats wrong with it ;)

I have another tip for Jeremy:

	git config submodule.<name>.ignore all

and you will not see any changes to the submodule. Put that into your
.gitmodules and you do not see any changes to the submodules anymore.

So now the only thing missing for complete convenience is a config
option for the --remote option in 'git submodule update'.

Jeremy, does the ignore option combined with --remote what you want?

Cheers Heiko

Re: Reference a submodule branch instead of a commit

[Junio C Hamano]

Heiko Voigt <hvoigt@hvoigt.net> writes:

>> It IS a hack, but having this information in .git<something> would
>> mean that it can be forced to be in machine readable form, unlike a
>> mention in README.  I do not know if the .gitmodules/.gitignore
>> combination is a sensible thing to use, but it does smell like a
>> potentially useful hack.
>
> IIRC the tree entries are the reference for submodules in the code. We
> are iterating over the tree entries in many places so that change does
> not seem so easy to me.
>
> But you are right maybe we should stop arguing against this workflow and
> just let people use it until they find out whats wrong with it ;)

I didn't say that, though.  I am fairly firm on _not_ changing what
the superproject records in its tree for the submodule, i.e. it must
record the exact commit, not "a branch name", for reproducibility. 

I am OK if people ignored the unmatch between the recorded commit
from a submodule and what they had in the submodule directory while
they developed and tested the superproject commit.  After all, it is
not an error to make a commit while having a local uncommitted
changes to tracked files, and it is equally valid to have a commit
checked out in a submodule directory that is different from what
goes in the superproject commit.  But we do show "modified but not
committed" in the status output.  In that light, submodule.*.ignore
may have been a mistake.

Re: Reference a submodule branch instead of a commit

[Heiko Voigt]

On Wed, Oct 05, 2016 at 09:13:53AM -0700, Junio C Hamano wrote:
> Heiko Voigt <hvoigt@hvoigt.net> writes:
> 
> >> It IS a hack, but having this information in .git<something> would
> >> mean that it can be forced to be in machine readable form, unlike a
> >> mention in README.  I do not know if the .gitmodules/.gitignore
> >> combination is a sensible thing to use, but it does smell like a
> >> potentially useful hack.
> >
> > IIRC the tree entries are the reference for submodules in the code. We
> > are iterating over the tree entries in many places so that change does
> > not seem so easy to me.
> >
> > But you are right maybe we should stop arguing against this workflow and
> > just let people use it until they find out whats wrong with it ;)
> 
> I didn't say that, though.  I am fairly firm on _not_ changing what
> the superproject records in its tree for the submodule, i.e. it must
> record the exact commit, not "a branch name", for reproducibility. 

I was not talking about changing what the superproject records in its
tree. I was just talking about changing where we look for submodules
(e.g. for updating and such). I.e. in .git* instead of just the tree as
it is at the moment. Thats what I understood from the discussion above.
Sorry that might have been ambiguous.

I agree that there should always be a commit as a reference for a
submodule. But as far as I understand for some projects its to much
overhead to record every change of a submodule but still they want to
use the latest code during development. Those projects might only want
to record the actual commit when they release something. At least thats
what I imagine.

> I am OK if people ignored the unmatch between the recorded commit
> from a submodule and what they had in the submodule directory while
> they developed and tested the superproject commit.  After all, it is
> not an error to make a commit while having a local uncommitted
> changes to tracked files, and it is equally valid to have a commit
> checked out in a submodule directory that is different from what
> goes in the superproject commit.  But we do show "modified but not
> committed" in the status output.  In that light, submodule.*.ignore
> may have been a mistake.

The original intend for submodule.*.ignore was to help people not
showing submodules as dirty when they had untracked files in them. That
was after status learned to look into submodules. 'untracked' to avoid the
performance overhead and 'dirty' for the people that accidentally worked
with dirty submodules. I agree 'all' might have been to much.

For the above workflow what user might actually want is something that
ignores all changes as long as they are part of the remote branch. But I
am just guessing here. My gut feeling is still that most people that
request this feature come from svn. Thats why I asked whether the
options I described provide the behavior that Jeremy wants.

Cheers Heiko