[PATCH] git-submodule add: Record branch name in .gitmodules

[PATCH] git-submodule add: Record branch name in .gitmodules

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This removes a configuration step if you're trying to setup Ævar's

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

workflow from

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

If you're not using that workflow, I see no harm in recording the
branch used to determine the original submodule commit.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 3 ++-
 git-submodule.sh                | 4 ++++
 t/t7400-submodule-basic.sh      | 1 +
 3 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..b9f437f 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -207,7 +207,8 @@ OPTIONS
 
 -b::
 --branch::
-	Branch of repository to add as submodule.
+	Branch of repository to add as submodule.  The branch name is
+	recorded in .gitmodules for future reference.
 
 -f::
 --force::
diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..fd15a54 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -366,6 +366,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
+	if test -n "$branch"
+	then
+		git config -f .gitmodules submodule."$sm_path".branch "$branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5397037..5031e2a 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test "$(git config -f .gitmodules submodule.submod-branch.branch)" = initial &&
 		git submodule init
 	) &&
 
-- 
1.8.0.2.g09b91ca

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2912 bytes --]

On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote:
> Some projects now use the 'branch' config value to record the tracking
> branch for the submodule.  Some ascribe different meaning to the
> configuration if the value is given vs. undefined.  For example, see
> the Gerrit submodule-subscription mechanism.  This change will cause
> those workflows to behave differently than they do now.

For those to lazy to hunt down a reference, Gerrit uses
submodule.$name.branch to

  “indicate the branch of a submodule project that when updated will
  trigger automatic update of its registered gitlink.” [1]

They also have some extra sauce:

  “The branch value could be '.' if the submodule project branch has
  the same name as the destination branch of the commit having
  gitlinks/.gitmodules file.
  …
  Any git submodules which are added and not have the branch field
  available in the .gitmodules file will not be subscribed by Gerrit
  to automatically update the superproject.”

> I do like the idea, but I wish it had a different name for the
> recording.  Maybe --record-branch=${BRANCH} as an extra switch so the
> action is explicitly requested.
> 
>   git submodule add --branch=master --record-branch=master foo bar

Ugh, I don't want to retype the branch name whenever I do this.

<brainstorming>

How about -r/--record, with the recorded name being optional?

  --record-branch[=<recorded_name>]

This would satisfy Gerrit users that wanted to use '.', but also
satisfy me with:

  git submodule add -rb=master foo bar

However, there is a change that people would see that, and then use

  git submodule add -r -b=master foo bar

which would checkout the HEAD from foo and store `-b=master` in
submodule.$name.branch.

A more verbose, but less dangerous, option would be a boolean
-r/--record that enables the recording of whatever was passed to
-b/--branch.  This looses the flexibility of running something like

  git submodule add --branch=master --record-branch=. foo bar

but the Gerrit folks have gotten along OK without any branch recording
so far ;).  They can keep setting '.' the same way they always have.

</brainstorming>

On a tangentially related note, it would be nice to set environment
variables for each of the settings in submodule.$name during a foreach
call.  Then you could use

  git submodule foreach 'git checkout $branch && git pull'

Perhaps you'd want to blacklist/whitelist or downcase settings names
to avoid things like

  [submodule "foo"]
        PATH = /my/rootkit/

but the update line is much cleaner.

Cheers,
Trevor

[1]: https://gerrit.googlesource.com/gerrit/+/master/Documentation/user-submodules.txt

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[Nahor]

On 2012-10-22 09:34, W. Trevor King wrote:
> From: "W. Trevor King" <wking@tremily.us>
>
> This removes a configuration step if you're trying to setup Ævar's
>
>    $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
>
> workflow from
>
>    commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
>    Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>    Date:   Fri May 21 16:10:10 2010 +0000
>
>      git-submodule foreach: Add $toplevel variable
>
> If you're not using that workflow, I see no harm in recording the
> branch used to determine the original submodule commit.

IMHO, the problem is that this works only for a very specific workflow. 
Normal git usage can very easily break this feature.

For instance, the module may later be updated to a commit in branch B 
instead of branch A. Unless you remember to also update .gitmodule, you 
have then inconsistent information.

A similar issue arises if branch A is deleted from the module later (for 
instance because it has been merged in the master branch and is not 
useful anymore). Then .gitmodule points to a non-existant branch.
Same thing if a branch is renamed.

Last issue, the branch that exists in your local repository may not 
exist in someone else's repository, either because the branch is purely 
local, or because it has a different name on the remote repo.


I think a better place to store that kind of information is using 
git-notes. That way, if the branch is renamed or deleted, you can easily 
update the old notes to use the new name instead.


	Nahor

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1635 bytes --]

On Tue, Oct 23, 2012 at 12:16:22PM -0700, Nahor wrote:
> On 2012-10-22 09:34, W. Trevor King wrote:
> For instance, the module may later be updated to a commit in branch B 
> instead of branch A. Unless you remember to also update .gitmodule, you 
> have then inconsistent information.

But you're explicitly *using* the configured setting in

  git config --file $toplevel/.gitmodules submodule.$name.branch

That should be a reminder that the configuration is important, and
you'll remember to change it.  Plus, the text from git-pull should
clearly display the branch you are pulling, which gives you a second
change to notice if something is going wrong.

> A similar issue arises if branch A is deleted from the module later (for 
> instance because it has been merged in the master branch and is not 
> useful anymore). Then .gitmodule points to a non-existant branch.
> Same thing if a branch is renamed.

All of these are possible, and all would reqire manual intervention to
pick out a new branch to follow.  Plus, this patch is not even about
the auto-pull application, it's about storing the initially cloned
branch name.  What you do with that name afterwards is up to you ;).

> I think a better place to store that kind of information is using 
> git-notes. That way, if the branch is renamed or deleted, you can easily 
> update the old notes to use the new name instead.

Interesting.  What would you attach the note too?

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[Jens Lehmann]

Am 23.10.2012 21:16, schrieb Nahor:
> On 2012-10-22 09:34, W. Trevor King wrote:
>> From: "W. Trevor King" <wking@tremily.us>
>>
>> This removes a configuration step if you're trying to setup Ævar's
>>
>>    $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
>>
>> workflow from
>>
>>    commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
>>    Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>    Date:   Fri May 21 16:10:10 2010 +0000
>>
>>      git-submodule foreach: Add $toplevel variable
>>
>> If you're not using that workflow, I see no harm in recording the
>> branch used to determine the original submodule commit.

Except recording the branch name might raise expectations about what git
will do with it. And as far as this patch goes, git won't do anything
with it (yet).

> IMHO, the problem is that this works only for a very specific workflow. Normal git usage can very easily break this feature.
> 
> For instance, the module may later be updated to a commit in branch B instead of branch A. Unless you remember to also update .gitmodule, you have then inconsistent information.
> 
> A similar issue arises if branch A is deleted from the module later (for instance because it has been merged in the master branch and is not useful anymore). Then .gitmodule points to a non-existant branch.
> Same thing if a branch is renamed.

Those are valid points. The next "git submodule update" will leave the
submodule with a detached HEAD, making the branch configuration rather
meaningless (except for your "git submodule foreach ..." use case of
course).

> Last issue, the branch that exists in your local repository may not exist in someone else's repository, either because the branch is purely local, or because it has a different name on the remote repo.

You'll always face this kind of problems with commits too when using
submodules, so I don't see that as a problem here.

> I think a better place to store that kind of information is using git-notes. That way, if the branch is renamed or deleted, you can easily update the old notes to use the new name instead.

I can't comment on that, as I have never been using notes so far.

But I'd rather see a patch series properly implementing the always-tip
mode Ævar mentions in f030c96d86 (and which is requested by some users),
especially the interesting parts: What should git record as commit in
that case and how are "git status" and "git diff" going to handle
submodules which shall follow a specific branch. I assume "git submodule
update" is the right point in time to fetch that branch again and check
out a newer branch tip if necessary, but should that commit be added to
the superproject for that submodule automagically or not? This patch
falls short of this, as it does the easy part but not the interesting
ones ;-)

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2028 bytes --]

On Tue, Oct 23, 2012 at 03:44:36PM -0400, W. Trevor King wrote:
> On Tue, Oct 23, 2012 at 12:16:22PM -0700, Nahor wrote:
> > On 2012-10-22 09:34, W. Trevor King wrote:
> > For instance, the module may later be updated to a commit in branch B 
> > instead of branch A. Unless you remember to also update .gitmodule, you 
> > have then inconsistent information.
> 
> But you're explicitly *using* the configured setting in
> 
>   git config --file $toplevel/.gitmodules submodule.$name.branch
> 
> That should be a reminder that the configuration is important, and
> you'll remember to change it.

To make my case more cleanly, people already handle all the
troublesome cases for branch.$name.remote, so handling similar
upstream volatility for submodule.$name.branch should not be too
difficult or surprising.

On Tue, Oct 23, 2012 at 03:58:48PM -0400, Phil Hord wrote:
> On Mon, Oct 22, 2012 at 6:55 PM, W. Trevor King <wking@tremily.us> wrote:
> > How about -r/--record, with the recorded name being optional?
> >
> >   --record-branch[=<recorded_name>]
> 
> I like that just fine.
> 
> > This would satisfy Gerrit users that wanted to use '.', but also
> > satisfy me with:
> >
> >   git submodule add -rb=master foo bar
> >
> > However, there is a change that people would see that, and then use
> >
> >   git submodule add -r -b=master foo bar
> >
> > which would checkout the HEAD from foo and store `-b=master` in
> > submodule.$name.branch.
> 
> I don't think it would.

Ah, right, forcing the =<name> attached case would mean they'd have to
use

  git submodule add -r=-b=master

which doesn't sound like the sort of thing you'd do accidentally.

> Though I see in rev-parse--parseopts that the use of
> optional-argument options "is discouraged".

I'll work up a v2 patch and we'll see if anyone complains ;).

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2219 bytes --]

On Tue, Oct 23, 2012 at 10:36:44PM +0200, Jens Lehmann wrote:
> Except recording the branch name might raise expectations about what git
> will do with it. And as far as this patch goes, git won't do anything
> with it (yet).

As Phil pointed out, doing anything with this variable is ambiguous:

On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote:
> Some projects now use the 'branch' config value to record the tracking
> branch for the submodule.  Some ascribe different meaning to the
> configuration if the value is given vs. undefined.  For example, see
> the Gerrit submodule-subscription mechanism.  This change will cause
> those workflows to behave differently than they do now.

On Tue, Oct 23, 2012 at 10:36:44PM +0200, Jens Lehmann wrote:
> But I'd rather see a patch series properly implementing the always-tip
> mode Ævar mentions in f030c96d86 (and which is requested by some users),
> especially the interesting parts: What should git record as commit in
> that case and how are "git status" and "git diff" going to handle
> submodules which shall follow a specific branch. I assume "git submodule
> update" is the right point in time to fetch that branch again and check
> out a newer branch tip if necessary, but should that commit be added to
> the superproject for that submodule automagically or not? This patch
> falls short of this, as it does the easy part but not the interesting
> ones ;-)

I agree that I'm not working on always-tip.  I'm just making that
easier.  For people that aren't interested in always-tip submodules
(e.g. Gerrit folks), this patch is still useful.  It would certainly
be possible to build an always-tip implementation on top of
submodule.$name.branch (as Ævar's one-liner does), but that would be
another patch series.

Personally, I think truly updates should be made explicitly, with a
hand written commit message about why the updates are occuring.  I
also think that setting up and running auto-updates should be easy
one-liners, not long, complicated ones ;).

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[Nahor]

On 2012-10-23 12:44, W. Trevor King wrote:
> On Tue, Oct 23, 2012 at 12:16:22PM -0700, Nahor wrote:
>> On 2012-10-22 09:34, W. Trevor King wrote:
>> For instance, the module may later be updated to a commit in branch B
>> instead of branch A. Unless you remember to also update .gitmodule, you
>> have then inconsistent information.
>
> But you're explicitly *using* the configured setting in
>
>    git config --file $toplevel/.gitmodules submodule.$name.branch
>
> That should be a reminder that the configuration is important, and
> you'll remember to change it.

 From my experience with my colleagues at work, that's not going to 
happen. People are very good at forgetting to do something ;)


> Plus, the text from git-pull should
> clearly display the branch you are pulling, which gives you a second
> change to notice if something is going wrong.

That's assuming that the user knows the branch that should be pulled and 
that he's paying attention to the output and not just quick-scanning for 
errors/warnings.
Again, from my experience, that's not going to be the case.

Plus, there isn't always a human being behind a git-pull, e.g. build bots.


>> I think a better place to store that kind of information is using
>> git-notes. That way, if the branch is renamed or deleted, you can easily
>> update the old notes to use the new name instead.
>
> Interesting.  What would you attach the note too?

To the commits in the super-repo where a module branch is selected, i.e.:
- where a module was added
- where the module changed branch
- where a super-branch was merged and there was a conflict on the 
module's branch name

[PATCH v2] git-submodule add: Add -r/--record option.

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This option allows you to record a submodule.<name>.branch option in
.gitmodules.  Git does not currently use this configuration option for
anything, but users have used it for several things, so it makes sense
to add some syntactic sugar for initializing the value.

Current consumers:

Ævar uses this setting to designate the upstream branch for pulling
submodule updates:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

as he describes in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses this setting to

  “indicate the branch of a submodule project that when updated will
  trigger automatic update of its registered gitlink.” [1]

I'm not clear on what that means, but they accept special values like
'.', so their usage is not compatible with Ævar's proposal.

By remaining agnostic on the variable usage, this patch makes
submodule setup more convenient for all parties.

[1] https://gerrit.googlesource.com/gerrit/+/master/Documentation/user-submodules.txt

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 11 ++++++++++-
 git-submodule.sh                | 19 ++++++++++++++++++-
 t/t7400-submodule-basic.sh      | 25 +++++++++++++++++++++++++
 3 files changed, 53 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..f9c74d6 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -9,7 +9,7 @@ git-submodule - Initialize, update or inspect submodules
 SYNOPSIS
 --------
 [verse]
-'git submodule' [--quiet] add [-b branch] [-f|--force]
+'git submodule' [--quiet] add [-b branch] [--record[=<branch>]] [-f|--force]
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
@@ -209,6 +209,15 @@ OPTIONS
 --branch::
 	Branch of repository to add as submodule.
 
+-r::
+--record::
+  Record a branch name used as `submodule.<path>.branch` in
+  `.gitmodules` for future reference.  If you do not list an explicit
+  name here, the name given with `--branch` will be recorded.  If that
+  is not set either, `HEAD` will be recorded.  Because the branch name
+  is optional, you must use the equal-sign form (`-r=<branch>`), not
+  `-r <branch>`.
+
 -f::
 --force::
 	This option is only valid for add and update commands.
diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..bc33112 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -5,7 +5,7 @@
 # Copyright (c) 2007 Lars Hjemli
 
 dashless=$(basename "$0" | sed -e 's/-/ /')
-USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
+USAGE="[--quiet] add [-b branch] [--record[=<branch>]] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
    or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
@@ -20,6 +20,8 @@ require_work_tree
 
 command=
 branch=
+record_branch=
+record_branch_empty=
 force=
 reference=
 cached=
@@ -257,6 +259,12 @@ cmd_add()
 			branch=$2
 			shift
 			;;
+		-r | --record)
+			record_branch_empty=true
+			;;
+		-r=* | --record=*)
+			record_branch="${1#*=}"
+			;;
 		-f | --force)
 			force=$1
 			;;
@@ -328,6 +336,11 @@ cmd_add()
 	git ls-files --error-unmatch "$sm_path" > /dev/null 2>&1 &&
 	die "$(eval_gettext "'\$sm_path' already exists in the index")"
 
+	if test -z "$record_branch" && test "$record_branch_empty" = "true"
+	then
+		record_branch="${branch:=HEAD}"
+	fi
+
 	if test -z "$force" && ! git add --dry-run --ignore-missing "$sm_path" > /dev/null 2>&1
 	then
 		eval_gettextln "The following path is ignored by one of your .gitignore files:
@@ -366,6 +379,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
+	if test -n "$branch"
+	then
+		git config -f .gitmodules submodule."$sm_path".branch "$record_branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5397037..88ae74c 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test -z "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
 		git submodule init
 	) &&
 
@@ -211,6 +212,30 @@ test_expect_success 'submodule add with ./, /.. and // in path' '
 	test_cmp empty untracked
 '
 
+test_expect_success 'submodule add --record' '
+	(
+		cd addtest &&
+		git submodule add -r "$submodurl" submod-record-head &&
+		test "$(git config -f .gitmodules submodule.submod-record-head.branch)" = "HEAD"
+	)
+'
+
+test_expect_success 'submodule add --record --branch' '
+	(
+		cd addtest &&
+		git submodule add -r -b initial "$submodurl" submod-auto-record &&
+		test "$(git config -f .gitmodules submodule.submod-auto-record.branch)" = "initial"
+	)
+'
+
+test_expect_success 'submodule add --record=<name> --branch' '
+	(
+		cd addtest &&
+		git submodule add -r=final -b initial "$submodurl" submod-record &&
+		test "$(git config -f .gitmodules submodule.submod-record.branch)" = "final"
+	)
+'
+
 test_expect_success 'setup - add an example entry to .gitmodules' '
 	GIT_CONFIG=.gitmodules \
 	git config submodule.example.url git://example.com/init.git
-- 
1.8.0.1.g61a31f6.dirty

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[Nahor]

On 2012-10-23 13:36, Jens Lehmann wrote:
> Am 23.10.2012 21:16, schrieb Nahor:
>> Last issue, the branch that exists in your local repository may not
 >> exist in someone else's repository, either because the branch is
 >> purely local, or because it has a different name on the remote repo.
>
> You'll always face this kind of problems with commits too when using
> submodules, so I don't see that as a problem here.

Commits can't change or disappear during "normal" git operation (i.e. 
without using "git push -f" or "git branch -D").
A commit also has the same id in all the clones repository so there is 
no issue of a different name between the local and the remote repositories.

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[Phil Hord]

On Mon, Oct 22, 2012 at 6:55 PM, W. Trevor King <wking@tremily.us> wrote:
> On a tangentially related note, it would be nice to set environment
> variables for each of the settings in submodule.$name during a foreach
> call.  Then you could use
>
>   git submodule foreach 'git checkout $branch && git pull'
>
> Perhaps you'd want to blacklist/whitelist or downcase settings names
> to avoid things like
>
>   [submodule "foo"]
>         PATH = /my/rootkit/
>
> but the update line is much cleaner.

This is ugly as can be, but it works in my meagre testing.  It defines
"submodule_path=/my/rootkit/" for the above bit of .gitmodules.  That
is, it adds definitions for 'submodule_<var-name>' for each <var-name>
in .gitmodules in submodule.$name.<var-name>, but <var-name> is
lowercased and defanged (everything that's not already [a-z0-9] is
replaced with underscore).  For example,

  git submodule foreach 'echo $submodule_url'

--- >8 ---
diff --git c/git-submodule.sh i/git-submodule.sh
index 6dd2338..79b3467 100755
--- c/git-submodule.sh
+++ i/git-submodule.sh
@@ -416,7 +416,15 @@ cmd_foreach()
                                prefix="$prefix$sm_path/"
                                clear_local_git_env
                                # we make $path available to scripts ...
                                path=$sm_path
+
+                               # make all submodule variables
available to scripts
+                               eval $(git config -f .gitmodules
--get-regexp "^submodule\.${name}\..*" |
+                               sed -e "s|^submodule\.${name}\.||" |
+                               while read VAR_NAME VAR_VALUE ; do
+                                       VAR_NAME=$(printf '%s'
"$VAR_NAME" | tr A-Z a-z | sed -e 's/^[^a-z]/_/' -e 's/[^a-z0-9]/_/g')
+                                       printf 'submodule_%s=%s;\n'
"$VAR_NAME" "'$VAR_VALUE'"
+                               done)
                                cd "$sm_path" &&
                                eval "$@" &&
                                if test -n "$recursive"

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[Jens Lehmann]

Am 24.10.2012 00:02, schrieb Nahor:
> On 2012-10-23 13:36, Jens Lehmann wrote:
>> Am 23.10.2012 21:16, schrieb Nahor:
>>> Last issue, the branch that exists in your local repository may not
>>> exist in someone else's repository, either because the branch is
>>> purely local, or because it has a different name on the remote repo.
>>
>> You'll always face this kind of problems with commits too when using
>> submodules, so I don't see that as a problem here.
> 
> Commits can't change or disappear during "normal" git operation (i.e. without using "git push -f" or "git branch -D").
> A commit also has the same id in all the clones repository so there is no issue of a different name between the local and the remote repositories.

But if you forget to push a submodule commit it won't exist in someone
else's repository and so he won't be able to update the submodule after
checking out a commit in the superproject that records that unpushed
submodule commit.

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jens Lehmann]

I still fail to see what adding that functionality to the submodule
command buys us (unless we also add code which really uses the branch
setting). What's wrong with doing a simple:

   git config -f .gitmodules submodule.<path>.branch <record_branch>

on the command line when you want to use the branch setting for your
own purposes? You could easily wrap that into a helper script, no?

Am 23.10.2012 23:57, schrieb W. Trevor King:
> From: "W. Trevor King" <wking@tremily.us>
> 
> This option allows you to record a submodule.<name>.branch option in
> .gitmodules.  Git does not currently use this configuration option for
> anything, but users have used it for several things, so it makes sense
> to add some syntactic sugar for initializing the value.
> 
> Current consumers:
> 
> Ævar uses this setting to designate the upstream branch for pulling
> submodule updates:
> 
>   $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
> 
> as he describes in
> 
>   commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
>   Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>   Date:   Fri May 21 16:10:10 2010 +0000
> 
>     git-submodule foreach: Add $toplevel variable
> 
> Gerrit uses this setting to
> 
>   “indicate the branch of a submodule project that when updated will
>   trigger automatic update of its registered gitlink.” [1]
> 
> I'm not clear on what that means, but they accept special values like
> '.', so their usage is not compatible with Ævar's proposal.
> 
> By remaining agnostic on the variable usage, this patch makes
> submodule setup more convenient for all parties.
> 
> [1] https://gerrit.googlesource.com/gerrit/+/master/Documentation/user-submodules.txt
> 
> Signed-off-by: W. Trevor King <wking@tremily.us>
> ---
>  Documentation/git-submodule.txt | 11 ++++++++++-
>  git-submodule.sh                | 19 ++++++++++++++++++-
>  t/t7400-submodule-basic.sh      | 25 +++++++++++++++++++++++++
>  3 files changed, 53 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index b4683bb..f9c74d6 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -9,7 +9,7 @@ git-submodule - Initialize, update or inspect submodules
>  SYNOPSIS
>  --------
>  [verse]
> -'git submodule' [--quiet] add [-b branch] [-f|--force]
> +'git submodule' [--quiet] add [-b branch] [--record[=<branch>]] [-f|--force]
>  	      [--reference <repository>] [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
> @@ -209,6 +209,15 @@ OPTIONS
>  --branch::
>  	Branch of repository to add as submodule.
>  
> +-r::
> +--record::
> +  Record a branch name used as `submodule.<path>.branch` in
> +  `.gitmodules` for future reference.  If you do not list an explicit
> +  name here, the name given with `--branch` will be recorded.  If that
> +  is not set either, `HEAD` will be recorded.  Because the branch name
> +  is optional, you must use the equal-sign form (`-r=<branch>`), not
> +  `-r <branch>`.
> +
>  -f::
>  --force::
>  	This option is only valid for add and update commands.
> diff --git a/git-submodule.sh b/git-submodule.sh
> index ab6b110..bc33112 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -5,7 +5,7 @@
>  # Copyright (c) 2007 Lars Hjemli
>  
>  dashless=$(basename "$0" | sed -e 's/-/ /')
> -USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
> +USAGE="[--quiet] add [-b branch] [--record[=<branch>]] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
>     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] init [--] [<path>...]
>     or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> @@ -20,6 +20,8 @@ require_work_tree
>  
>  command=
>  branch=
> +record_branch=
> +record_branch_empty=
>  force=
>  reference=
>  cached=
> @@ -257,6 +259,12 @@ cmd_add()
>  			branch=$2
>  			shift
>  			;;
> +		-r | --record)
> +			record_branch_empty=true
> +			;;
> +		-r=* | --record=*)
> +			record_branch="${1#*=}"
> +			;;
>  		-f | --force)
>  			force=$1
>  			;;
> @@ -328,6 +336,11 @@ cmd_add()
>  	git ls-files --error-unmatch "$sm_path" > /dev/null 2>&1 &&
>  	die "$(eval_gettext "'\$sm_path' already exists in the index")"
>  
> +	if test -z "$record_branch" && test "$record_branch_empty" = "true"
> +	then
> +		record_branch="${branch:=HEAD}"
> +	fi
> +
>  	if test -z "$force" && ! git add --dry-run --ignore-missing "$sm_path" > /dev/null 2>&1
>  	then
>  		eval_gettextln "The following path is ignored by one of your .gitignore files:
> @@ -366,6 +379,10 @@ Use -f if you really want to add it." >&2
>  
>  	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
>  	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
> +	if test -n "$branch"
> +	then
> +		git config -f .gitmodules submodule."$sm_path".branch "$record_branch"
> +	fi &&
>  	git add --force .gitmodules ||
>  	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
>  }
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index 5397037..88ae74c 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
>  	(
>  		cd addtest &&
>  		git submodule add -b initial "$submodurl" submod-branch &&
> +		test -z "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
>  		git submodule init
>  	) &&
>  
> @@ -211,6 +212,30 @@ test_expect_success 'submodule add with ./, /.. and // in path' '
>  	test_cmp empty untracked
>  '
>  
> +test_expect_success 'submodule add --record' '
> +	(
> +		cd addtest &&
> +		git submodule add -r "$submodurl" submod-record-head &&
> +		test "$(git config -f .gitmodules submodule.submod-record-head.branch)" = "HEAD"
> +	)
> +'
> +
> +test_expect_success 'submodule add --record --branch' '
> +	(
> +		cd addtest &&
> +		git submodule add -r -b initial "$submodurl" submod-auto-record &&
> +		test "$(git config -f .gitmodules submodule.submod-auto-record.branch)" = "initial"
> +	)
> +'
> +
> +test_expect_success 'submodule add --record=<name> --branch' '
> +	(
> +		cd addtest &&
> +		git submodule add -r=final -b initial "$submodurl" submod-record &&
> +		test "$(git config -f .gitmodules submodule.submod-record.branch)" = "final"
> +	)
> +'
> +
>  test_expect_success 'setup - add an example entry to .gitmodules' '
>  	GIT_CONFIG=.gitmodules \
>  	git config submodule.example.url git://example.com/init.git
> 

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

On Wed, Oct 24, 2012 at 09:15:32PM +0200, Jens Lehmann wrote:
> I still fail to see what adding that functionality to the submodule
> command buys us (unless we also add code which really uses the branch
> setting). What's wrong with doing a simple:
> 
>    git config -f .gitmodules submodule.<path>.branch <record_branch>
> 
> on the command line when you want to use the branch setting for your
> own purposes? You could easily wrap that into a helper script, no?

Sure.  But why maintain my own helper script if I can edit
git-submodules.sh?  It seems like a number of people are using this
config option, and they generally store the same name in it that they
use to create the submodule.  This way I can save them time too.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 623 bytes --]

On Wed, Oct 24, 2012 at 02:12:18PM -0400, Phil Hord wrote:
> +                                       VAR_NAME=$(printf '%s'
> "$VAR_NAME" | tr A-Z a-z | sed -e 's/^[^a-z]/_/' -e 's/[^a-z0-9]/_/g')

Is there a reason why you use printf instead of echo?

Also, this sort of name cleaning should probably live in a new
function:

  clean_environment_variable()

or some such.  Is there a git-utility-functions.sh library hiding
somewhere in the source? ;)

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 431 bytes --]

Should I rebase this so it lands cleanly atop 38ae92e4 in next?

  commit 38ae92e4d027063b9b87e51a9bf12809d10066f6
  Author: W. Trevor King <wking@tremily.us>
  Date:   Tue Oct 23 17:00:21 2012 -0400

    git-submodule: wrap branch option with "<>" in usage strings.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] git-submodule add: Record branch name in .gitmodules

[Jeff King]

On Wed, Oct 24, 2012 at 09:12:27PM -0400, W. Trevor King wrote:

> On Wed, Oct 24, 2012 at 02:12:18PM -0400, Phil Hord wrote:
> > +                                       VAR_NAME=$(printf '%s'
> > "$VAR_NAME" | tr A-Z a-z | sed -e 's/^[^a-z]/_/' -e 's/[^a-z0-9]/_/g')
> 
> Is there a reason why you use printf instead of echo?

Sadly, echo is not entirely portable for arbitrary content. We have run
into problems with backslashes being interpreted differently (bash
requires "echo -e" to interpret them, dash will always do it, and POSIX
leaves it vague).

> Also, this sort of name cleaning should probably live in a new
> function:
> 
>   clean_environment_variable()
> 
> or some such.  Is there a git-utility-functions.sh library hiding
> somewhere in the source? ;)

git-sh-setup?

-Peff

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jeff King]

On Thu, Oct 25, 2012 at 06:14:31PM -0400, W. Trevor King wrote:

> Should I rebase this so it lands cleanly atop 38ae92e4 in next?
> 
>   commit 38ae92e4d027063b9b87e51a9bf12809d10066f6
>   Author: W. Trevor King <wking@tremily.us>
>   Date:   Tue Oct 23 17:00:21 2012 -0400
> 
>     git-submodule: wrap branch option with "<>" in usage strings.

In general, it is not a good idea to base your patches on things in
next, because it means your topic is held hostage to the one in next,
which may or may not graduate to master. We can always do a merge later
(and in this case, it is really just a one-line conflict).

-Peff

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jens Lehmann]

Am 25.10.2012 02:53, schrieb W. Trevor King:
> On Wed, Oct 24, 2012 at 09:15:32PM +0200, Jens Lehmann wrote:
>> I still fail to see what adding that functionality to the submodule
>> command buys us (unless we also add code which really uses the branch
>> setting). What's wrong with doing a simple:
>>
>>    git config -f .gitmodules submodule.<path>.branch <record_branch>
>>
>> on the command line when you want to use the branch setting for your
>> own purposes? You could easily wrap that into a helper script, no?
> 
> Sure.  But why maintain my own helper script if I can edit
> git-submodules.sh?  It seems like a number of people are using this
> config option, and they generally store the same name in it that they
> use to create the submodule.  This way I can save them time too.

But people are already using the "branch" setting in *different* ways:

Am 23.10.2012 22:55, schrieb W. Trevor King:
> As Phil pointed out, doing anything with this variable is ambiguous:
>
> On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote:
>> Some projects now use the 'branch' config value to record the tracking
>> branch for the submodule.  Some ascribe different meaning to the
>> configuration if the value is given vs. undefined.  For example, see
>> the Gerrit submodule-subscription mechanism.  This change will cause
>> those workflows to behave differently than they do now.

I don't have a problem with the amount or complexity of the code being
added, But by adding that option we may be giving the impression that it
is officially sanctioned, or that it will be kept up to date by further
submodule commands. I added Shawn to the CC, maybe he can comment on how
the "branch" setting is used in Gerrit and what he thinks about adding
code to set that with "git submodule add -r <branch> ..." to core git.

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2764 bytes --]

On Sun, Oct 28, 2012 at 09:48:18PM +0100, Jens Lehmann wrote:
> Am 25.10.2012 02:53, schrieb W. Trevor King:
> > On Wed, Oct 24, 2012 at 09:15:32PM +0200, Jens Lehmann wrote:
> >> I still fail to see what adding that functionality to the submodule
> >> command buys us (unless we also add code which really uses the branch
> >> setting). What's wrong with doing a simple:
> >>
> >>    git config -f .gitmodules submodule.<path>.branch <record_branch>
> >>
> >> on the command line when you want to use the branch setting for your
> >> own purposes? You could easily wrap that into a helper script, no?
> > 
> > Sure.  But why maintain my own helper script if I can edit
> > git-submodules.sh?  It seems like a number of people are using this
> > config option, and they generally store the same name in it that they
> > use to create the submodule.  This way I can save them time too.
> 
> But people are already using the "branch" setting in *different* ways:

And they are usually storing the same string.  Now, more easily.  If
they want a different string, it is also easier.  If they don't want
to use --record, they can do things however they were already doing
them.  I don't see the problem.

> Am 23.10.2012 22:55, schrieb W. Trevor King:
> > As Phil pointed out, doing anything with this variable is ambiguous:
> >
> > On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote:
> >> Some projects now use the 'branch' config value to record the tracking
> >> branch for the submodule.  Some ascribe different meaning to the
> >> configuration if the value is given vs. undefined.  For example, see
> >> the Gerrit submodule-subscription mechanism.  This change will cause
> >> those workflows to behave differently than they do now.
> 
> I don't have a problem with the amount or complexity of the code being
> added, But by adding that option we may be giving the impression that it
> is officially sanctioned, or that it will be kept up to date by further
> submodule commands.

Storing something there will be officially sanctioned.  Using it for
anything in particular will not be officially sanctioned.  Phil's
submodule_<var-name> export in foreach will expose the variable so the
user can do whatever they think is appropriate with it, but it's still
up to the user to give the option some kind of semantic meaning.

> I added Shawn to the CC, maybe he can comment on how the "branch"
> setting is used in Gerrit and what he thinks about adding code to
> set that with "git submodule add -r <branch> ..." to core git.

Good idea.

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Shawn Pearce]

On Sun, Oct 28, 2012 at 1:48 PM, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> Am 23.10.2012 22:55, schrieb W. Trevor King:
>> As Phil pointed out, doing anything with this variable is ambiguous:
>>
>> On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote:
>>> Some projects now use the 'branch' config value to record the tracking
>>> branch for the submodule.  Some ascribe different meaning to the
>>> configuration if the value is given vs. undefined.  For example, see
>>> the Gerrit submodule-subscription mechanism.  This change will cause
>>> those workflows to behave differently than they do now.
>
> I don't have a problem with the amount or complexity of the code being
> added, But by adding that option we may be giving the impression that it
> is officially sanctioned, or that it will be kept up to date by further
> submodule commands. I added Shawn to the CC, maybe he can comment on how
> the "branch" setting is used in Gerrit and what he thinks about adding
> code to set that with "git submodule add -r <branch> ..." to core git.

Looks like the Gerrit meaning is basically the same as Ævar's. Gerrit
updates the parent project as if you had done:

  $ git submodule foreach 'git checkout $(git config --file
$toplevel/.gitmodules submodule.$name.branch) && git pull'
  $ git commit -a -m "Updated submodules"
  $ git push

and it does this automatically each time the submodule's branch is
modified by the Gerrit server.

On Tue, Oct 23, 2012 at 2:57 PM, W. Trevor King <wking@tremily.us> wrote:
> I'm not clear on what that means, but they accept special values like
> '.', so their usage is not compatible with Ævar's proposal.

"." is a special value to mean use the parent project's branch name.
So its more like this:

  $ git submodule foreach 'git checkout $(git --git-dir $toplevel/.git
read-ref HEAD | sed s,^refs/heads/,,) && git pull'
  $ git commit -a -m "Updated submodules"
  $ git push

We use "." in Gerrit to make branching an entire forest of projects
easier. Setting up dev-fix-yy in the parent project will automatically
track dev-fix-yy in each submodule.

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

On Sun, Oct 28, 2012 at 02:59:33PM -0700, Shawn Pearce wrote:
> Looks like the Gerrit meaning is basically the same as Ævar's. Gerrit
> updates the parent project as if you had done:
> 
>   $ git submodule foreach 'git checkout $(git config --file
> $toplevel/.gitmodules submodule.$name.branch) && git pull'
>   $ git commit -a -m "Updated submodules"
>   $ git push

Ah, good, then we *are* all using the option for the same thing.

> On Tue, Oct 23, 2012 at 2:57 PM, W. Trevor King <wking@tremily.us> wrote:
> > I'm not clear on what that means, but they accept special values like
> > '.', so their usage is not compatible with Ævar's proposal.
> 
> "." is a special value to mean use the parent project's branch name.
> So its more like this:
> 
>   $ git submodule foreach 'git checkout $(git --git-dir $toplevel/.git
> read-ref HEAD | sed s,^refs/heads/,,) && git pull'
>   $ git commit -a -m "Updated submodules"
>   $ git push
> 
> We use "." in Gerrit to make branching an entire forest of projects
> easier. Setting up dev-fix-yy in the parent project will automatically
> track dev-fix-yy in each submodule.

Ok.  If we wanted "." expansion to be a general submodule thing, it
would add a special case to Phil's submodule_<var-name> export.  I
don't think such a special case would be worth the mental overhead,
but obviously the Gerrit folks think it is.  I don't care either way
on this one.

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jeff King]

On Sun, Oct 28, 2012 at 06:34:31PM -0400, W. Trevor King wrote:

> On Sun, Oct 28, 2012 at 02:59:33PM -0700, Shawn Pearce wrote:
> > Looks like the Gerrit meaning is basically the same as Ævar's. Gerrit
> > updates the parent project as if you had done:
> > 
> >   $ git submodule foreach 'git checkout $(git config --file
> > $toplevel/.gitmodules submodule.$name.branch) && git pull'
> >   $ git commit -a -m "Updated submodules"
> >   $ git push
> 
> Ah, good, then we *are* all using the option for the same thing.

That makes me more comfortable. Your patch adds support for setting the
variable initially. Does it need any special magic for maintenance, or
is it something that would always be updated by hand?

Right now, the variable is not an official git-submodule thing and it is
OK to say "you are on your own by setting and using it from external
tools". But as soon as we support setting it in the first place, it is
reasonable to claim it as a bug if we do not keep it up to date for
certain operations.

I'm not familiar enough with the workflows around branching submodules
to know whether any such operations actually exist.

-Peff

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1647 bytes --]

On Mon, Oct 29, 2012 at 01:34:01AM -0400, Jeff King wrote:
> On Sun, Oct 28, 2012 at 06:34:31PM -0400, W. Trevor King wrote:
> 
> > On Sun, Oct 28, 2012 at 02:59:33PM -0700, Shawn Pearce wrote:
> > > Looks like the Gerrit meaning is basically the same as Ævar's. Gerrit
> > > updates the parent project as if you had done:
> > > 
> > >   $ git submodule foreach 'git checkout $(git config --file
> > > $toplevel/.gitmodules submodule.$name.branch) && git pull'
> > >   $ git commit -a -m "Updated submodules"
> > >   $ git push
> > 
> > Ah, good, then we *are* all using the option for the same thing.
> 
> That makes me more comfortable. Your patch adds support for setting the
> variable initially. Does it need any special magic for maintenance, or
> is it something that would always be updated by hand?

Everyone we've heard from so far interprets the setting as “pull from
$branch in the remote repository $url to update the submodule”.  With
Phil's export, that would become

  $ git submodule foreach 'git checkout "$submodule_branch" && git pull'
  $ git commit -a -m "Updated submodules"
  $ git push

As Nahor mentioned on the 23rd, there are a number of ways that the
upstream branch could disappear, but Git has no way to know what the
new branch setting should be.  This means that even if we make “pull
from $branch” interpretation official, we still couldn't do anything
slick about updating it.  So, yes, it will be updated by hand.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jeff King]

On Mon, Oct 29, 2012 at 06:45:44AM -0400, W. Trevor King wrote:

> > > Ah, good, then we *are* all using the option for the same thing.
> > 
> > That makes me more comfortable. Your patch adds support for setting the
> > variable initially. Does it need any special magic for maintenance, or
> > is it something that would always be updated by hand?
> 
> Everyone we've heard from so far interprets the setting as “pull from
> $branch in the remote repository $url to update the submodule”.  With
> Phil's export, that would become
> 
>   $ git submodule foreach 'git checkout "$submodule_branch" && git pull'
>   $ git commit -a -m "Updated submodules"
>   $ git push
> 
> As Nahor mentioned on the 23rd, there are a number of ways that the
> upstream branch could disappear, but Git has no way to know what the
> new branch setting should be.  This means that even if we make “pull
> from $branch” interpretation official, we still couldn't do anything
> slick about updating it.  So, yes, it will be updated by hand.

OK.

Can you send an updated version of the patch that summarizes the
situation in the commit message?

I also think it is probably worth saying something in the documentation
for the feature like "Note that this value is not actually used by git;
however, some external tools and workflows may make use of it."

-Peff

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

On Mon, Oct 29, 2012 at 06:58:55AM -0400, Jeff King wrote:
> Can you send an updated version of the patch that summarizes the
> situation in the commit message?

Sure.  Should I include Phil's $submodule_<var-name> export, or would
you rather have that be a separate series?

Phil, were you planning on rolling your patch into something more
formal, or was your preliminary patch a suggestion for me to build
from?

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jeff King]

On Mon, Oct 29, 2012 at 07:29:45AM -0400, W. Trevor King wrote:

> On Mon, Oct 29, 2012 at 06:58:55AM -0400, Jeff King wrote:
> > Can you send an updated version of the patch that summarizes the
> > situation in the commit message?
> 
> Sure.  Should I include Phil's $submodule_<var-name> export, or would
> you rather have that be a separate series?

I think it probably makes sense as a separate patch in the same series,
since it is meant to support the same workflows.

I am not sure it is sufficient as-is, though. It does not seem to ever
clear variables, only set them, which means that values could leak
across iterations of the loop, or down to recursive calls. E.g., when
the first submodule has submodule.*.foo set but the second one does not,
you will still end up with $submodule_foo set when you process the
second one.

-Peff

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Phil Hord]

On Mon, Oct 29, 2012 at 7:43 AM, Jeff King <peff@peff.net> wrote:
> On Mon, Oct 29, 2012 at 07:29:45AM -0400, W. Trevor King wrote:
>
>> On Mon, Oct 29, 2012 at 06:58:55AM -0400, Jeff King wrote:
>> > Can you send an updated version of the patch that summarizes the
>> > situation in the commit message?
>>
>> Sure.  Should I include Phil's $submodule_<var-name> export, or would
>> you rather have that be a separate series?
>
> I think it probably makes sense as a separate patch in the same series,
> since it is meant to support the same workflows.

I agree.  I did expect to clean it up some, but also to suffer some
review.  Feel free to clean it up as you see fit and submit it with
your series.

> I am not sure it is sufficient as-is, though. It does not seem to ever
> clear variables, only set them, which means that values could leak
> across iterations of the loop,  [...] E.g., when
> the first submodule has submodule.*.foo set but the second one does not,
> you will still end up with $submodule_foo set when you process the
> second one.

Good point.  That should not happen.

> or down to recursive calls.

Frankly, I consider that to be a feature.  However, I can see how it
would be considered inconsistent in many ways, so it's probably best
to squash it.  :-\

Phil

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jeff King]

On Mon, Oct 29, 2012 at 01:38:28PM -0400, Phil Hord wrote:

> > I am not sure it is sufficient as-is, though. It does not seem to ever
> > clear variables, only set them, which means that values could leak
> > across iterations of the loop,  [...] E.g., when
> > the first submodule has submodule.*.foo set but the second one does not,
> > you will still end up with $submodule_foo set when you process the
> > second one.
> 
> Good point.  That should not happen.
> 
> > or down to recursive calls.
> 
> Frankly, I consider that to be a feature.  However, I can see how it
> would be considered inconsistent in many ways, so it's probably best
> to squash it.  :-\

I think it would depend on the semantics of the option. Some options
would probably make sense to apply recursively, and some not.

Maybe instead of blindly converting config into the environment, it
should forward or clear specific known-meaning config.

-Peff

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Phil Hord]

On Mon, Oct 29, 2012 at 5:36 PM, Jeff King <peff@peff.net> wrote:
> On Mon, Oct 29, 2012 at 01:38:28PM -0400, Phil Hord wrote:
>
>> > I am not sure it is sufficient as-is, though. It does not seem to ever
>> > clear variables, only set them, which means that values could leak
>> > across iterations of the loop,  [...] E.g., when
>> > the first submodule has submodule.*.foo set but the second one does not,
>> > you will still end up with $submodule_foo set when you process the
>> > second one.
>>
>> Good point.  That should not happen.
>>
>> > or down to recursive calls.
>>
>> Frankly, I consider that to be a feature.  However, I can see how it
>> would be considered inconsistent in many ways, so it's probably best
>> to squash it.  :-\
>
> I think it would depend on the semantics of the option. Some options
> would probably make sense to apply recursively, and some not.
>
> Maybe instead of blindly converting config into the environment, it
> should forward or clear specific known-meaning config.

Well, that's where we started.  I was aiming for the more generic
"never needs updating" direction.

P

Re: [PATCH v2] git-submodule add: Add -r/--record option.

[Jeff King]

On Mon, Oct 29, 2012 at 06:21:08PM -0400, Phil Hord wrote:

> > Maybe instead of blindly converting config into the environment, it
> > should forward or clear specific known-meaning config.
> 
> Well, that's where we started.  I was aiming for the more generic
> "never needs updating" direction.

Then I think you are probably stuck taking the conservative approach of
not propagating recursively.

-Peff

[PATCH v3 0/3] git-submodule add: Add -r/--record option

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

Here's my revised patch.  Changes from v2:

* Revised Ævar-vs-Gerrit usage to show agreement, following Shawn's
  comments.
* Added a cleaned up version of Phil's $submodule_* export patch, with
  docs and tests.
* Added a caveat to the -r/--record documentation to make it explicit
  that submodule.<name>.branch is not used internally by Git.  Give an
  example of how the user may use it explicitly for Ævar-style
  updates.

W. Trevor King (3):
  git-submodule add: Add -r/--record option
  git-submodule foreach: export .gitmodules settings as variables
  git-submodule: Motivate --record with an example use case

 Documentation/git-submodule.txt | 22 +++++++++++++++++++++-
 git-sh-setup.sh                 | 20 ++++++++++++++++++++
 git-submodule.sh                | 35 ++++++++++++++++++++++++++++++++++-
 t/t7400-submodule-basic.sh      | 25 +++++++++++++++++++++++++
 t/t7407-submodule-foreach.sh    | 29 +++++++++++++++++++++++++++++
 5 files changed, 129 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 git-sh-setup.sh

-- 
1.8.0.3.gc2eb43a

[PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This option allows you to record a submodule.<name>.branch option in
.gitmodules.  Git does not currently use this configuration option for
anything, but users have used it for several things, so it makes sense
to add some syntactic sugar for initializing the value.

Current consumers:

Ævar uses this setting to designate the upstream branch for pulling
submodule updates:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

as he describes in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.

By remaining agnostic on the variable usage, this patch makes
submodule setup more convenient for all parties.

[1] https://gerrit.googlesource.com/gerrit/+/master/Documentation/user-submodules.txt

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 11 ++++++++++-
 git-submodule.sh                | 19 ++++++++++++++++++-
 t/t7400-submodule-basic.sh      | 25 +++++++++++++++++++++++++
 3 files changed, 53 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..cbec363 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -9,7 +9,7 @@ git-submodule - Initialize, update or inspect submodules
 SYNOPSIS
 --------
 [verse]
-'git submodule' [--quiet] add [-b branch] [-f|--force]
+'git submodule' [--quiet] add [-b branch] [--record[=<branch>]] [-f|--force]
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
@@ -209,6 +209,15 @@ OPTIONS
 --branch::
 	Branch of repository to add as submodule.
 
+-r::
+--record::
+	Record a branch name used as `submodule.<path>.branch` in
+	`.gitmodules` for future reference.  If you do not list an explicit
+	name here, the name given with `--branch` will be recorded.  If that
+	is not set either, `HEAD` will be recorded.  Because the branch name
+	is optional, you must use the equal-sign form (`-r=<branch>`), not
+	`-r <branch>`.
+
 -f::
 --force::
 	This option is only valid for add and update commands.
diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..bc33112 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -5,7 +5,7 @@
 # Copyright (c) 2007 Lars Hjemli
 
 dashless=$(basename "$0" | sed -e 's/-/ /')
-USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
+USAGE="[--quiet] add [-b branch] [--record[=<branch>]] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
    or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
@@ -20,6 +20,8 @@ require_work_tree
 
 command=
 branch=
+record_branch=
+record_branch_empty=
 force=
 reference=
 cached=
@@ -257,6 +259,12 @@ cmd_add()
 			branch=$2
 			shift
 			;;
+		-r | --record)
+			record_branch_empty=true
+			;;
+		-r=* | --record=*)
+			record_branch="${1#*=}"
+			;;
 		-f | --force)
 			force=$1
 			;;
@@ -328,6 +336,11 @@ cmd_add()
 	git ls-files --error-unmatch "$sm_path" > /dev/null 2>&1 &&
 	die "$(eval_gettext "'\$sm_path' already exists in the index")"
 
+	if test -z "$record_branch" && test "$record_branch_empty" = "true"
+	then
+		record_branch="${branch:=HEAD}"
+	fi
+
 	if test -z "$force" && ! git add --dry-run --ignore-missing "$sm_path" > /dev/null 2>&1
 	then
 		eval_gettextln "The following path is ignored by one of your .gitignore files:
@@ -366,6 +379,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
+	if test -n "$branch"
+	then
+		git config -f .gitmodules submodule."$sm_path".branch "$record_branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5397037..88ae74c 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test -z "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
 		git submodule init
 	) &&
 
@@ -211,6 +212,30 @@ test_expect_success 'submodule add with ./, /.. and // in path' '
 	test_cmp empty untracked
 '
 
+test_expect_success 'submodule add --record' '
+	(
+		cd addtest &&
+		git submodule add -r "$submodurl" submod-record-head &&
+		test "$(git config -f .gitmodules submodule.submod-record-head.branch)" = "HEAD"
+	)
+'
+
+test_expect_success 'submodule add --record --branch' '
+	(
+		cd addtest &&
+		git submodule add -r -b initial "$submodurl" submod-auto-record &&
+		test "$(git config -f .gitmodules submodule.submod-auto-record.branch)" = "initial"
+	)
+'
+
+test_expect_success 'submodule add --record=<name> --branch' '
+	(
+		cd addtest &&
+		git submodule add -r=final -b initial "$submodurl" submod-record &&
+		test "$(git config -f .gitmodules submodule.submod-record.branch)" = "final"
+	)
+'
+
 test_expect_success 'setup - add an example entry to .gitmodules' '
 	GIT_CONFIG=.gitmodules \
 	git config submodule.example.url git://example.com/init.git
-- 
1.8.0.3.gc2eb43a

[PATCH v3 2/3] git-submodule foreach: export .gitmodules settings as variables

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This makes it easy to access per-submodule variables.  For example,

  git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

can now be reduced to

  git submodule foreach 'git checkout $submodule_branch && git pull'

Every submodule.<name>.<opt> setting from .gitmodules is available as
a $submodule_<sanitized-opt> variable.  These variables are not
propagated recursively into nested submodules.

Signed-off-by: W. Trevor King <wking@tremily.us>
Based-on-patch-by: Phil Hord <phil.hord@gmail.com>
---
 Documentation/git-submodule.txt |  3 +++
 git-sh-setup.sh                 | 20 ++++++++++++++++++++
 git-submodule.sh                | 16 ++++++++++++++++
 t/t7407-submodule-foreach.sh    | 29 +++++++++++++++++++++++++++++
 4 files changed, 68 insertions(+)
 mode change 100644 => 100755 git-sh-setup.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index cbec363..9a99826 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -175,6 +175,9 @@ foreach::
 	$path is the name of the submodule directory relative to the
 	superproject, $sha1 is the commit as recorded in the superproject,
 	and $toplevel is the absolute path to the top-level of the superproject.
+	In addition, every submodule.<name>.<opt> setting from .gitmodules
+	is available as the variable $submodule_<sanitized_opt>.  These
+	variables are not propagated recursively into nested submodules.
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given `--quiet`, foreach prints the name
 	of each submodule before evaluating the command.
diff --git a/git-sh-setup.sh b/git-sh-setup.sh
old mode 100644
new mode 100755
index ee0e0bc..179a920
--- a/git-sh-setup.sh
+++ b/git-sh-setup.sh
@@ -222,6 +222,26 @@ clear_local_git_env() {
 	unset $(git rev-parse --local-env-vars)
 }
 
+# Remove any suspect characters from a user-generated variable name.
+sanitize_variable_name() {
+	VAR_NAME="$1"
+	printf '%s' "$VAR_NAME" |
+	sed -e 's/^[^a-zA-Z]/_/' -e 's/[^a-zA-Z0-9]/_/g'
+}
+
+# Return a command for setting a new variable.
+# Neither the variable name nor the variable value passed to this
+# function need to be sanitized.  You need to eval the returned
+# string, because new variables set by the function itself don't
+# effect the calling process.
+set_user_variable() {
+	VAR_NAME="$1"
+	VAR_VALUE="$2"
+	VAR_NAME=$(sanitize_variable_name "$VAR_NAME")
+	VAR_VALUE=$(printf '%s' "$VAR_VALUE" |
+		sed -e 's/\\/\\\\/g' -e 's/"/\\"/g')
+	printf '%s=%s;\n' "$VAR_NAME" "\"$VAR_VALUE\""
+}
 
 # Platform specific tweaks to work around some commands
 case $(uname -s) in
diff --git a/git-submodule.sh b/git-submodule.sh
index bc33112..e4d26f9 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -434,8 +434,24 @@ cmd_foreach()
 				clear_local_git_env
 				# we make $path available to scripts ...
 				path=$sm_path
+
+				# make all submodule variables available to scripts
+				eval $(
+					git config -f .gitmodules --get-regexp "^submodule\.${name}\..*" |
+					sed -e "s|^submodule\.${name}\.||" |
+					while read VAR_NAME VAR_VALUE ; do
+						VAR_NAME=$(printf '%s' "$VAR_NAME" | tr A-Z a-z)
+						set_user_variable "submodule_${VAR_NAME}" "$VAR_VALUE"
+					done)
+				UNSET_CMD=$(set |
+					sed -n -e 's|^\(submodule_[a-z_]*\)=.*$|\1|p' |
+					while read VAR_NAME ; do
+						printf 'unset %s;\n' "$VAR_NAME"
+					done)
+
 				cd "$sm_path" &&
 				eval "$@" &&
+				eval "$UNSET_CMD" &&
 				if test -n "$recursive"
 				then
 					cmd_foreach "--recursive" "$@"
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 9b69fe2..46ac746 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -313,4 +313,33 @@ test_expect_success 'command passed to foreach --recursive retains notion of std
 	test_cmp expected actual
 '
 
+cat > expect <<EOF
+Entering 'nested1'
+nested1 nested1 wonky"value
+Entering 'nested1/nested2'
+nested2 nested2 another wonky"value
+Entering 'nested1/nested2/nested3'
+nested3 nested3
+Entering 'nested1/nested2/nested3/submodule'
+submodule submodule
+Entering 'sub1'
+sub1 sub1
+Entering 'sub2'
+sub2 sub2
+Entering 'sub3'
+sub3 sub3
+EOF
+
+test_expect_success 'test foreach environment variables' '
+	(
+		cd clone2 &&
+		git config -f .gitmodules submodule.nested1.wonky-var "wonky\"value" &&
+		git config -f nested1/.gitmodules submodule.nested2.wonky-var "another wonky\"value" &&
+		git submodule foreach --recursive "echo \$path \$submodule_path \$submodule_wonky_var" > ../actual
+	) &&
+	test_i18ncmp expect actual
+'
+#
+#"echo \$toplevel-\$name-\$submodule_path-\$submodule_url"
+
 test_done
-- 
1.8.0.3.gc2eb43a

[PATCH v3 3/3] git-submodule: Motivate --record with an example use case

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 9a99826..d4e993f 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -220,6 +220,14 @@ OPTIONS
 	is not set either, `HEAD` will be recorded.  Because the branch name
 	is optional, you must use the equal-sign form (`-r=<branch>`), not
 	`-r <branch>`.
++
+The recorded setting is not actually used by git; however, some
+external tools and workflows may make use of it.  For example, if the
+upstream branches still exist and you have a recorded branch setting
+for each of your submodules, you can update all of the submodules to
+the current branch tips with:
++
+	git submodule foreach 'git checkout $submodule_branch && git pull'
 
 -f::
 --force::
-- 
1.8.0.3.gc2eb43a

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> By remaining agnostic on the variable usage, this patch makes
> submodule setup more convenient for all parties.

I personally do not think "remaining agnostic on the usage" is a
good thing, at least for any option to commands at the higher level
on the stack, such as "git submodule".  I am afraid that giving an
easier way to set up a variable with undefined semantics may make
setup more confusing for all parties.  One party gives one specific
meaning to the field, while another party uses it for something
slightly different.

I would not object to "git config submodule.$name.branch $value", on
the other hand.  "git config" can be used to set a piece of data
that has specific meaning, but as a low-level tool, it is not
_limited_ to variables that have defined meaning.

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Heiko Voigt]

Hi,

On Thu, Nov 08, 2012 at 11:34:54PM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > By remaining agnostic on the variable usage, this patch makes
> > submodule setup more convenient for all parties.
> 
> I personally do not think "remaining agnostic on the usage" is a
> good thing, at least for any option to commands at the higher level
> on the stack, such as "git submodule".  I am afraid that giving an
> easier way to set up a variable with undefined semantics may make
> setup more confusing for all parties.  One party gives one specific
> meaning to the field, while another party uses it for something
> slightly different.
> 
> I would not object to "git config submodule.$name.branch $value", on
> the other hand.  "git config" can be used to set a piece of data
> that has specific meaning, but as a low-level tool, it is not
> _limited_ to variables that have defined meaning.

I think we should agree on a behavior for this option and implement it
the same time when add learns about it. When we were discussing floating
submodules as an important option for the gerrit people I already started
to implement a proof of concept. Please have a look here:

https://github.com/hvoigt/git/commits/hv/floating_submodules

AFAIK this does not yet implement the same behaviour the gerrit tools
offer for this option. The main reason behind that was because I do not
know the typical workflow behind such an option. So I am open to
changes.

Maybe you can use or base your work on this implementation for submodule
update.

Without submodule update using this option I think it would be better to
implement this option in the tool you are using instead of submodule add.
Everything else feels incomplete to me.

Cheers Heiko

Re: [PATCH v3 2/3] git-submodule foreach: export .gitmodules settings as variables

[Heiko Voigt]

Hi,

On Thu, Nov 08, 2012 at 10:35:13PM -0500, W. Trevor King wrote:
> From: "W. Trevor King" <wking@tremily.us>
> 
> This makes it easy to access per-submodule variables.  For example,
> 
>   git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
> 
> can now be reduced to
> 
>   git submodule foreach 'git checkout $submodule_branch && git pull'

What other use cases are there? Would the need for this maybe go away
once you had floating submodules following branches?

The whole thing looks like its adding some complex code which is not so
easy to read. I would like to make sure its worth it.

> diff --git a/git-submodule.sh b/git-submodule.sh
> index bc33112..e4d26f9 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -434,8 +434,24 @@ cmd_foreach()
>  				clear_local_git_env
>  				# we make $path available to scripts ...
>  				path=$sm_path
> +
> +				# make all submodule variables available to scripts
> +				eval $(
> +					git config -f .gitmodules --get-regexp "^submodule\.${name}\..*" |

For completeness you should make the variables possible to override by
repository from the local repository configuration like all other
submodule options that are read directly from .gitmodules.

Cheers Heiko

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

On Thu, Nov 08, 2012 at 11:34:54PM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > By remaining agnostic on the variable usage, this patch makes
> > submodule setup more convenient for all parties.
> 
> I personally do not think "remaining agnostic on the usage" is a
> good thing, at least for any option to commands at the higher level
> on the stack, such as "git submodule".  I am afraid that giving an
> easier way to set up a variable with undefined semantics may make
> setup more confusing for all parties.  One party gives one specific
> meaning to the field, while another party uses it for something
> slightly different.
> 
> I would not object to "git config submodule.$name.branch $value", on
> the other hand.  "git config" can be used to set a piece of data
> that has specific meaning, but as a low-level tool, it is not
> _limited_ to variables that have defined meaning.

This is what I'm doing now:

  $ git submodule add -b <branch> <repo> <path>
  $ git config --file .gitmodules submodule.<path>.branch <branch>
  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

With my second patch (Phil's config export), that becomes

  $ git submodule add -b <branch> <repo> <path>
  $ git config --file .gitmodules submodule.<path>.branch <branch>
  $ git submodule foreach 'git checkout $submodule_branch && git pull'

With my first patch, that becomes

  $ git submodule add -rb <branch> <repo> <path>
  $ git submodule foreach 'git checkout $submodule_branch && git pull'

This seems pretty useful to me, but I'm still using
submodule.<name>.branch explicitly as a user, and Git is not
interpreting the option directly.  Users are free to store whatever
they like in that option, and use it however they wish:

  $ git submodule foreach 'do-crazy-stuff.sh $submodule_branch'

If we need a semantic interpretation to justify -r/--record, everyone
that's chimed in so far has agreed on the same interpretation.  I
wouldn't be averse to

  $ git submodule add -rb <branch> <repo> <path>
  $ git submodule pull-branch

which makes the foreach pull logic internal.  However, there has been
a reasonable amount of resistance to this workflow in the past, so I
thought that a patch series that avoided a semantic interpretation
would be more acceptable.

If neither an agnostic -r/--record or a semantic pull-branch command
are acceptable, I suppose we'll have to drop my first and third
patches and only keep the second.

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

On Fri, Nov 09, 2012 at 05:29:27PM +0100, Heiko Voigt wrote:
> I think we should agree on a behavior for this option and implement it
> the same time when add learns about it. When we were discussing floating
> submodules as an important option for the gerrit people I already started
> to implement a proof of concept. Please have a look here:
> 
> https://github.com/hvoigt/git/commits/hv/floating_submodules

After skimming through this, something like

  $ git submodule update --pull

would probably be better than introducing a new command:

On Sat, Nov 10, 2012 at 01:44:37PM -0500, W. Trevor King wrote:
>   $ git submodule pull-branch

I think "floating submodules" is a misleading name for this feature
though, since the checkout SHA is explicitly specified.  We're just
making it more convenient to explicitly update the SHA.  How about
"tracking submodules"?

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

On Fri, Nov 09, 2012 at 02:46:07AM -0800, Matt Kraai wrote:
> On Thu, Nov 08, 2012 at 10:35:12PM -0500, W. Trevor King wrote:
> > @@ -366,6 +379,10 @@ Use -f if you really want to add it." >&2
> >  
> >  	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
> >  	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
> > +	if test -n "$branch"
> > +	then
> > +		git config -f .gitmodules submodule."$sm_path".branch "$record_branch"
> > +	fi &&
> >  	git add --force .gitmodules ||
> >  	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
> >  }
> 
> Should the if condition test that $record_branch is not the empty
> string instead of testing that $branch is not the empty string?  It
> seems like this will set submodule."$sm_path".branch to the empty
> string if -b is specified and no -r option is specified.

Oops, thanks for catching that.  Will fix with v4, once we figure out
what to do about the semantic-pull situation.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [PATCH v3 2/3] git-submodule foreach: export .gitmodules settings as variables

[W. Trevor King]

On Fri, Nov 09, 2012 at 05:45:22PM +0100, Heiko Voigt wrote:
> > can now be reduced to
> > 
> >   git submodule foreach 'git checkout $submodule_branch && git pull'
> 
> What other use cases are there? Would the need for this maybe go away
> once you had floating submodules following branches?

None that I can think of, but I don't use submodules very much.  The
idea of easily-accessible per-submodule configuration variables
strikes me as pretty useful, but I agree the code is a bit ugly.
Actually, I think exporting environment variables and calling the
foreach command in a subshell would be better than the current local
variables and eval.  The subshell would also make variable cleanup
irrelevant, which would make for a cleaner patch.

> For completeness you should make the variables possible to override by
> repository from the local repository configuration like all other
> submodule options that are read directly from .gitmodules.

Good idea (I wasn't aware of the override before).  Will do in v4.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> On Thu, Nov 08, 2012 at 11:34:54PM -0800, Junio C Hamano wrote:
>
>> I would not object to "git config submodule.$name.branch $value", on
>> the other hand.  "git config" can be used to set a piece of data
>> that has specific meaning, but as a low-level tool, it is not
>> _limited_ to variables that have defined meaning.
>
> This is what I'm doing now:
>
>   $ git submodule add -b <branch> <repo> <path>
>   $ git config --file .gitmodules submodule.<path>.branch <branch>
>   $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
>
> With my second patch (Phil's config export), that becomes
>
>   $ git submodule add -b <branch> <repo> <path>
>   $ git config --file .gitmodules submodule.<path>.branch <branch>
>   $ git submodule foreach 'git checkout $submodule_branch && git pull'
>
> With my first patch, that becomes
>
>   $ git submodule add -rb <branch> <repo> <path>
>   $ git submodule foreach 'git checkout $submodule_branch && git pull'
>
> This seems pretty useful to me,...

Ah, this reminds me of another thing I noticed when I saw that
patch.  The change seems to think "branch" is the _only_ thing the
user might want to record per submodule upon "git submodule add".
As an interface to muck with an uninterpreted random configuration,
it squats on a good option name for setting one single and arbitrary
variable---quite a selfish change that is not acceptable.

Calling the option "--record-branch-for-submodule" or something more
specific might alleviate the problem, but then it would become even
less useful as a short-hand for "config submodule.$name.branch", I
would suspect.

On the other hand, if this were one small part of a series to define
the "tip following mode" where (at least)

 (1) "git submodule update [$path]" makes sure that the checkout of
     the submodule at $path matches the commit at the tip of the
     branch named by submodule.$name.branch in .gitmodules of the
     superproject, instead of the commit that is recorded in the
     index of the superproject; and

 (2) "git diff [$path]" and friends in the superproject compares the
     HEAD of the checkout of the submodule at $path with the tip of
     the branch named by submodule.$name.branch in .gitmodules of
     the superproject, instead of the commit that is recorded in the
     index of the superproject.

and the option were called something like "--follow-branch=$branch",
it would make much more sense for its initial implementation to set
the name of the branch to submodule.$name.branch variable.  Later
iterations of such a feature may want to do more than just setting
that single variable but that is a part of the implementation detail
of the tip following mode the users do not have to know about, just
like setting the submodule.$name.branch variable is.

So in that sense, too, I would be somewhat unhappy to see this
change in the current form to go in.

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2556 bytes --]

On Sun, Nov 11, 2012 at 02:33:45AM -0800, Junio C Hamano wrote:
> The change seems to think "branch" is the _only_ thing the user
> might want to record per submodule upon "git submodule add".

I felt that earlier floating/tracking submodule patches were biting
off more than they could chew, so I was looking for a lightweight fix
to make the tracking workflow easier.  It seems like I ended up with
something that is too lightweight ;).

> On the other hand, if this were one small part of a series to define
> the "tip following mode" where (at least)
> 
>  (1) "git submodule update [$path]" makes sure that the checkout of
>      the submodule at $path matches the commit at the tip of the
>      branch named by submodule.$name.branch in .gitmodules of the
>      superproject, instead of the commit that is recorded in the
>      index of the superproject; and

As I mentioned earlier, I think

  $ git submodule update [$path]

should keep its current “checkout the already-registered SHA”
functionality, with

  $ git submodule update --pull [$path]

pulling the tracked branch.  I'll add a patch implementing this to v4.

In order to avoid losing (or creating) local-only submodule commits,
I'll probably bail (with an error) on non-fast-forward pulls.  Can
anyone else think of other safety concerns?

This means that I'll probably drop Phil's $submodule_* export in v4,
because the only explicit use we have for it is this branch tracking.
I still think it is a useful idea, but it may not be useful enough to
be worth the complexity.

>  (2) "git diff [$path]" and friends in the superproject compares the
>      HEAD of the checkout of the submodule at $path with the tip of
>      the branch named by submodule.$name.branch in .gitmodules of
>      the superproject, instead of the commit that is recorded in the
>      index of the superproject.
> 

Hmm.  “git diff” compares the working tree with the local HEAD (just a
SHA for submodules), so I don't think it should care about the status
of a remote branch.  This sounds like you want something like:

  $ git submodule foreach 'git diff origin/$submodule_branch'

Perhaps this is enough motivation for keeping $submodule_* exports?

> and the option were called something like "--follow-branch=$branch",
> …

I'll replace -r/--record with --follow-branch in v4.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Heiko Voigt]

Hi,

sorry for the late reply but my git time is limited.

On Sat, Nov 10, 2012 at 02:02:32PM -0500, W. Trevor King wrote:
> On Fri, Nov 09, 2012 at 05:29:27PM +0100, Heiko Voigt wrote:
> > I think we should agree on a behavior for this option and implement it
> > the same time when add learns about it. When we were discussing floating
> > submodules as an important option for the gerrit people I already started
> > to implement a proof of concept. Please have a look here:
> > 
> > https://github.com/hvoigt/git/commits/hv/floating_submodules
> 
> After skimming through this, something like
> 
>   $ git submodule update --pull
> 
> would probably be better than introducing a new command:

Yeah along the lines of that, but one thing to keep in mind:

We already have --rebase and --merge which do slightly different things
(I think). Adding --pull here should behave similar to them. Like fetch
and merge is the same to pull without submodules.

If I am understanding your goal correctly your --pull would be
different. On the other hand: A --pull makes no sense if we apply it to
the existing --merge option since it merges the recorded sha1 into the
current HEAD. Just a fetch would not really make a difference.

Thinking along the existing options I would probably still expect --pull
to merge something into the current HEAD. So maybe we have to iron out
where this command/option should go. But changing that once we have a
patch to discuss should not be that much work. So please proceed with
--pull and once we know exactly what it does we can polish that.

> On Sat, Nov 10, 2012 at 01:44:37PM -0500, W. Trevor King wrote:
> >   $ git submodule pull-branch
> 
> I think "floating submodules" is a misleading name for this feature
> though, since the checkout SHA is explicitly specified.  We're just
> making it more convenient to explicitly update the SHA.  How about
> "tracking submodules"?

Until now we have always called this workflow floating submodules. I
imaging since the submodule floats to the newest revision (whatever the
user chooses that to be) instead of staying at the recorded sha1.

"tracking submodules" sounds strange to me since the term tracked in git
is mainly used in combination with exact recorded history (e.g. tracking
branch). Since it is about *not* checking out the recorded sha1 but
something that can change I think that could cause confusion.

I think floating is a more unambiguous term and already known on the
list.

Cheers Heiko

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Heiko Voigt]

Hi,

On Sun, Nov 11, 2012 at 10:00:48AM -0500, W. Trevor King wrote:
> On Sun, Nov 11, 2012 at 02:33:45AM -0800, Junio C Hamano wrote:
> In order to avoid losing (or creating) local-only submodule commits,
> I'll probably bail (with an error) on non-fast-forward pulls.  Can
> anyone else think of other safety concerns?

That sounds like a good thing to do. We can allow more flexibility later
if people come up with usecases.

> This means that I'll probably drop Phil's $submodule_* export in v4,
> because the only explicit use we have for it is this branch tracking.
> I still think it is a useful idea, but it may not be useful enough to
> be worth the complexity.

Yes lets concentrate on the branch following first.

> >  (2) "git diff [$path]" and friends in the superproject compares the
> >      HEAD of the checkout of the submodule at $path with the tip of
> >      the branch named by submodule.$name.branch in .gitmodules of
> >      the superproject, instead of the commit that is recorded in the
> >      index of the superproject.
> > 
> 
> Hmm.  ???git diff??? compares the working tree with the local HEAD (just a
> SHA for submodules), so I don't think it should care about the status
> of a remote branch.  This sounds like you want something like:
> 
>   $ git submodule foreach 'git diff origin/$submodule_branch'
> 
> Perhaps this is enough motivation for keeping $submodule_* exports?
> 
> > and the option were called something like "--follow-branch=$branch",
> > ???

I am not sure if hiding changes to the recorded SHA1 from the user is
such a useful thing. In the first step I would like it if it was kept
simple and only the submodule update machinery learned to follow a
branch. If that results in local changes that should be shown. The user
is still in charge of recording the updated SHA1 in his commit.

>From what I have heard of projects using this: They usually still have
something that records the SHA1s on a regular basis. Thinking further,
why not record them in git? We could add an option to update which
creates such a commit.

Since git is all about changes I am hesitant to hide them from the user.

> I'll replace -r/--record with --follow-branch in v4.

Sounds good.

Cheers Heiko

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3410 bytes --]

On Sat, Nov 17, 2012 at 04:04:42PM +0100, Heiko Voigt wrote:
> > On Sat, Nov 10, 2012 at 01:44:37PM -0500, W. Trevor King wrote:
> > >   $ git submodule pull-branch
> > 
> > I think "floating submodules" is a misleading name for this feature
> > though, since the checkout SHA is explicitly specified.  We're just
> > making it more convenient to explicitly update the SHA.  How about
> > "tracking submodules"?
> 
> Until now we have always called this workflow floating submodules. I
> imaging since the submodule floats to the newest revision (whatever the
> user chooses that to be) instead of staying at the recorded sha1.
> 
> "tracking submodules" sounds strange to me since the term tracked in git
> is mainly used in combination with exact recorded history (e.g. tracking
> branch). Since it is about *not* checking out the recorded sha1 but
> something that can change I think that could cause confusion.
> 
> I think floating is a more unambiguous term and already known on the
> list.

I had been getting the impression that floating submodules would
automatically update without explicit user intervention.  After
re-reading your initial floating submodules post, it looks like we do
match up after the mapping:

  Git        Heiko               Trevor
  ---------  -----------------   -------------
  update     update --checkout   update
             update              update --pull

So I'll go back to "floating" ;).

On Sat, Nov 17, 2012 at 04:30:07PM +0100, Heiko Voigt wrote:
> > >  (2) "git diff [$path]" and friends in the superproject compares the
> > >      HEAD of the checkout of the submodule at $path with the tip of
> > >      the branch named by submodule.$name.branch in .gitmodules of
> > >      the superproject, instead of the commit that is recorded in the
> > >      index of the superproject.
> > > 
> > 
> > Hmm.  ???git diff??? compares the working tree with the local HEAD (just a
> > SHA for submodules), so I don't think it should care about the status
> > of a remote branch.  This sounds like you want something like:
> > 
> >   $ git submodule foreach 'git diff origin/$submodule_branch'
> > 
> > Perhaps this is enough motivation for keeping $submodule_* exports?
> > 
> > > and the option were called something like "--follow-branch=$branch",
> > > ???
> 
> I am not sure if hiding changes to the recorded SHA1 from the user is
> such a useful thing. In the first step I would like it if it was kept
> simple and only the submodule update machinery learned to follow a
> branch. If that results in local changes that should be shown. The user
> is still in charge of recording the updated SHA1 in his commit.

I understand what you're warning against here, or what it has to do
with "git diff".

> From what I have heard of projects using this: They usually still have
> something that records the SHA1s on a regular basis. Thinking further,
> why not record them in git? We could add an option to update which
> creates such a commit.

I think it's best to have users craft their own commit messages
explaining why the branch was updated.  That said, an auto-generated
hint (a la "git merge") would probably be a useful extra feature.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Heiko Voigt]

On Sat, Nov 17, 2012 at 02:20:27PM -0500, W. Trevor King wrote:
> On Sat, Nov 17, 2012 at 04:30:07PM +0100, Heiko Voigt wrote:
> > > >  (2) "git diff [$path]" and friends in the superproject compares the
> > > >      HEAD of thecheckout of the submodule at $path with the tip of
> > > >      the branch named by submodule.$name.branch in .gitmodules of
> > > >      the superproject, instead of the commit that is recorded in the
> > > >      index of the superproject.
> > > > 
> > > 
> > > Hmm.  ???git diff??? compares the working tree with the local HEAD (just a
> > > SHA for submodules), so I don't think it should care about the status
> > > of a remote branch.  This sounds like you want something like:
> > > 
> > >   $ git submodule foreach 'git diff origin/$submodule_branch'
> > > 
> > > Perhaps this is enough motivation for keeping $submodule_* exports?
> > > 
> > > > and the option were called something like "--follow-branch=$branch",
> > > > ???
> > 
> > I am not sure if hiding changes to the recorded SHA1 from the user is
> > such a useful thing. In the first step I would like it if it was kept
> > simple and only the submodule update machinery learned to follow a
> > branch. If that results in local changes that should be shown. The user
> > is still in charge of recording the updated SHA1 in his commit.
> 
> I understand what you're warning against here, or what it has to do
> with "git diff".

Is there a not missing here? Reads somehow like that. What I am talking
about is the suggestion of Junio.  Instead of showing a diff if the
SHA1 is different we show a diff if the checkout in the worktree is
different from the tip of the configured branch. That would hide the
fact that a submodule has changed during a submodule update operation.

> > From what I have heard of projects using this: They usually still have
> > something that records the SHA1s on a regular basis. Thinking further,
> > why not record them in git? We could add an option to update which
> > creates such a commit.
> 
> I think it's best to have users craft their own commit messages
> explaining why the branch was updated.  That said, an auto-generated
> hint (a la "git merge") would probably be a useful extra feature.

I have the same opinion. Commits should always be created by humans so
you have someone to blame/ask why. But I guess there are people that
expect this to be automatic.

One argument somehow goes along the lines:
"I already created a commit in the submodule why do I need to create
another one in the superproject? Just follow the HEAD revision!" They
think in subversions "submodules" which are merely pointers to other svn
repositories without any revision information. I am unsure if its good
to support this the same way.

Another use case is big projects that have so many submodules that
creating superproject commits would create to much maintenance work.
They want to have their integration server make those commits. That
would already be supported with update checking out the branch tips and
the commit is just one extra thing to do by the integration server.

So I think it should be fine just to teach update to checkout the
configured branch tips (or forward them to their tracking branch tips)
and leave the rest to the user.

Cheers Heiko

Re: Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2212 bytes --]

On Sat, Nov 17, 2012 at 10:31:30PM +0100, Heiko Voigt wrote:
> On Sat, Nov 17, 2012 at 02:20:27PM -0500, W. Trevor King wrote:
> > On Sat, Nov 17, 2012 at 04:30:07PM +0100, Heiko Voigt wrote:
> > > > >  (2) "git diff [$path]" and friends in the superproject compares the
> > > > >      HEAD of thecheckout of the submodule at $path with the tip of
> > > > >      the branch named by submodule.$name.branch in .gitmodules of
> > > > >      the superproject, instead of the commit that is recorded in the
> > > > >      index of the superproject.
> > > > > 
> > > > 
> > > > Hmm.  ???git diff??? compares the working tree with the local HEAD (just a
> > > > SHA for submodules), so I don't think it should care about the status
> > > > of a remote branch.  This sounds like you want something like:
> > > > 
> > > >   $ git submodule foreach 'git diff origin/$submodule_branch'
> > > > 
> > > > Perhaps this is enough motivation for keeping $submodule_* exports?
> > > > 
> > > > > and the option were called something like "--follow-branch=$branch",
> > > > > ???
> > > 
> > > I am not sure if hiding changes to the recorded SHA1 from the user is
> > > such a useful thing. In the first step I would like it if it was kept
> > > simple and only the submodule update machinery learned to follow a
> > > branch. If that results in local changes that should be shown. The user
> > > is still in charge of recording the updated SHA1 in his commit.
> > 
> > I understand what you're warning against here, or what it has to do
> > with "git diff".
> 
> Is there a not missing here?

Thanks.  I'd meant to say "I don't understand…".

> What I am talking about is the suggestion of Junio.  Instead of
> showing a diff if the SHA1 is different we show a diff if the
> checkout in the worktree is different from the tip of the configured
> branch. That would hide the fact that a submodule has changed during
> a submodule update operation.

Ahh, now I understand.  I agree that comparing to the remote tip is a
bad idea.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

>> From what I have heard of projects using this: They usually still have
>> something that records the SHA1s on a regular basis. Thinking further,
>> why not record them in git? We could add an option to update which
>> creates such a commit.
>
> I think it's best to have users craft their own commit messages
> explaining why the branch was updated.  That said, an auto-generated
> hint (a la "git merge") would probably be a useful extra feature.

I am not quite sure I agree.  When the project says "Use the tip of
'bar' branch for the submodule 'foo'" at the top-level, does an
individual user who is not working on the submodule 'foo' but merely
is using it have any clue as to why the submodule's 'foo' branch
'foo' moved, or does he necessarily even care?

For such a user working at the top-level superproject, or working on
one part of the project, possibly on a submodule other than 'foo',
wouldn't the natural thing to do would be to run "git pull" at the
top-level, maybe with "--recursive" to update the top-level and all
the submodules to start the day.

Now, since somebody created the top-level commit you have just
pulled and checked out, other people may have worked on submodule
'foo' [*1*].  What should happen on "git submodule update foo"?  It
would notice that the submodule 'foo' is set to float, and would
check out the tip of the branch 'bar', not the commit recorded in
the top-level superproject, in the working tree for 'foo', no?

What should appear in "git diff"?  The working tree taken as a whole
is different from what the superproject's commit describes (which is
the state the person who created the superproject wanted to record)
even though this user does not have anything to do with the change
at 'foo' from the recorded commit to the current tip of 'bar'.  What
would his description for the reason why the branch was updated?

I think I would agree that "git diff" should not hide such changes
(after all, when this user records his change to the overall project
in the top-level supermodule, he will be recording the state with
the commit at the tip of 'bar' checked out in the working tree of
the submodule 'foo'), but I am not sure if the user can say anything
sensible, other than "tip of 'bar' branch in submodule 'foo' was
changed by others", in the resulting commit.


[Footnote]

*1* This may look like a non-issue if you assume that the person who
updates the 'bar' branch of submodule 'foo' always updates the
gitlink in the superproject's commit to point at that updated
commit, but that assumption is flawed; the submodule project is a
project on its own and can be worked on without what other projects
bind it as their submodules.

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2806 bytes --]

On Mon, Nov 19, 2012 at 04:49:09PM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> >> From what I have heard of projects using this: They usually still have
> >> something that records the SHA1s on a regular basis. Thinking further,
> >> why not record them in git? We could add an option to update which
> >> creates such a commit.
> >
> > I think it's best to have users craft their own commit messages
> > explaining why the branch was updated.  That said, an auto-generated
> > hint (a la "git merge") would probably be a useful extra feature.
> 
> I am not quite sure I agree.  When the project says "Use the tip of
> 'bar' branch for the submodule 'foo'" at the top-level, does an
> individual user who is not working on the submodule 'foo' but merely
> is using it have any clue as to why the submodule's 'foo' branch
> 'foo' moved, or does he necessarily even care?

If he doesn't care, why is he updating the submodule gitlink?

> Now, since somebody created the top-level commit you have just
> pulled and checked out, other people may have worked on submodule
> 'foo' [*1*].  What should happen on "git submodule update foo"?

If the 'foo' checkout is not the one listed in the superproject's
.gitmodules, the update should bail with an appropriate error message,
and let the user sort things out.

  $ git submodule update --pull foo
  error: Your local changes to the following submodule would be
  overwritten by update:…

This is similar to how Git currently bails on dirty-tree branch
switches:

  $ git checkout my-branch
  error: Your local changes to the following files would be
  overwritten by checkout:…

Without "--pull", the update command is intended to checkout the hash
specified in .gitmodules.  If you've committed some local work in foo
and then explicitly ask for an update, I suppose you get clobbered.

> What should appear in "git diff"?  The working tree taken as a whole
> is different from what the superproject's commit describes (which is
> the state the person who created the superproject wanted to record)
> even though this user does not have anything to do with the change
> at 'foo' from the recorded commit to the current tip of 'bar'.  What
> would his description for the reason why the branch was updated?

The submodule content is not part of the superproject.  All the
superproject has is a gitlink.  If the gitlink hasn't changed, "git
diff" in the superproject shouldn't say anything.

I'll probably have time to write up v4 over the weekend.  Maybe having
a more explicit example will clear things up.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> On Mon, Nov 19, 2012 at 04:49:09PM -0800, Junio C Hamano wrote:
>> "W. Trevor King" <wking@tremily.us> writes:
>> ...
>> > I think it's best to have users craft their own commit messages
>> > explaining why the branch was updated.  That said, an auto-generated
>> > hint (a la "git merge") would probably be a useful extra feature.
>> 
>> I am not quite sure I agree.  When the project says "Use the tip of
>> 'bar' branch for the submodule 'foo'" at the top-level, does an
>> individual user who is not working on the submodule 'foo' but merely
>> is using it have any clue as to why the submodule's 'foo' branch
>> 'foo' moved, or does he necessarily even care?
>
> If he doesn't care, why is he updating the submodule gitlink?

He may not be updating the gitlink with "git add foo" at the
top-level superproject level.  He is just using that submodule as
part of the larger whole as he is working on either the top-level or
some other submodule.  And checkout of 'foo' is necessary in the
working tree for him to work in the larger context of the project,
and 'foo' is set to float at the tip of its 'bar' branch.  And that
checkout results in a commit that is different from the commit the
gitlink suggests, perhaps because somebody worked in 'foo' submodule
and advanced the tip of branch 'bar'.

So:

 - at the top-level superproject level, entry 'foo' in the HEAD tree
   points at an older commit;

 - 'foo/.git/HEAD' points at refs/heads/bar, which matches the
   working tree of 'foo' and the index foo/.git/index..

I am not sure what should happen to the entry 'foo' in the index of
the top-level superproject after such a 'submodule floats at the
tip' checkout, but I imagine that it must match the contents of
foo/.git/HEAD's tree.  Otherwise, "git diff" at the top-level would
report local changes.

When committing his work at the top-level, he will see that 'foo'
gitlink is updated in that commit; after all that combination is the
context in which his work was done.

Or are you envisioning that such a check-out will and should show a
local difference at the submodule 'foo' by leaving the index of the
top-level superproject unchanged, and the user should refrain from
using "git commit -a" to avoid having to describe the changes made
on the 'bar' branch in the meantime in his top-level commit?  That
is certainly fine by me (I am no a heavy submodule user to begin
with), but I am not sure if that is useful and helpful to the
submodule users.

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3039 bytes --]

On Mon, Nov 19, 2012 at 09:39:34PM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > On Mon, Nov 19, 2012 at 04:49:09PM -0800, Junio C Hamano wrote:
> >> "W. Trevor King" <wking@tremily.us> writes:
> >> ...
> >> > I think it's best to have users craft their own commit messages
> >> > explaining why the branch was updated.  That said, an auto-generated
> >> > hint (a la "git merge") would probably be a useful extra feature.
> >> 
> >> I am not quite sure I agree.  When the project says "Use the tip of
> >> 'bar' branch for the submodule 'foo'" at the top-level, does an
> >> individual user who is not working on the submodule 'foo' but merely
> >> is using it have any clue as to why the submodule's 'foo' branch
> >> 'foo' moved, or does he necessarily even care?
> >
> > If he doesn't care, why is he updating the submodule gitlink?
> 
> He may not be updating the gitlink with "git add foo" at the
> top-level superproject level.  He is just using that submodule as
> part of the larger whole as he is working on either the top-level or
> some other submodule.  And checkout of 'foo' is necessary in the
> working tree for him to work in the larger context of the project,
> and 'foo' is set to float at the tip of its 'bar' branch.  And that
> checkout results in a commit that is different from the commit the
> gitlink suggests, perhaps because somebody worked in 'foo' submodule
> and advanced the tip of branch 'bar'.

The superproject gitlink should only be updated after

  $ git submodule update --pull

A plain

  $ git submodule update

would still checkout the previously-recorded SHA, not the new upstream
tip.  The uncaring user should skip the "--pull", and there will be no
superproject changes to worry about.

> Or are you envisioning that such a check-out will and should show a
> local difference at the submodule 'foo' by leaving the index of the
> top-level superproject unchanged,

A plain "git submodule update" will, yes.  And this will clobber any
changes that have happened in the submodule directory and its index
(because the user explicitly asked to checkout the
superproject-recorded SHA)

> and the user should refrain from using "git commit -a" to avoid
> having to describe the changes made on the 'bar' branch in the
> meantime in his top-level commit?

What would "git commit -a" be picking up?  Nothing in the superproject
has changed?

> That is certainly fine by me (I am no a heavy submodule user to
> begin with), but I am not sure if that is useful and helpful to the
> submodule users.

The benefit is that Ævar's

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

becomes

  $ git submodule update --pull

Still an explicit pull, but much easier to remember.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> The superproject gitlink should only be updated after
>
>   $ git submodule update --pull
>
> A plain
>
>   $ git submodule update
>
> would still checkout the previously-recorded SHA, not the new upstream
> tip.

Hrm, doesn't it make the "float at the tip of a branch" mode
useless, though?

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Heiko Voigt]

On Tue, Nov 20, 2012 at 11:52:46AM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > The superproject gitlink should only be updated after
> >
> >   $ git submodule update --pull
> >
> > A plain
> >
> >   $ git submodule update
> >
> > would still checkout the previously-recorded SHA, not the new upstream
> > tip.
> 
> Hrm, doesn't it make the "float at the tip of a branch" mode
> useless, though?

How about having a branch config option and reusing our
submodule.$name.update option for specifying whether the user wants to
always float to the tip of the branch?

1. If submodule.$name.update is pull it would checkout the specified tip.

2. If submodule.$name.update is checkout or none it would do the usual
   thing and you need to specify --pull to get the tip.

I am still a little bit undecided about an automatically crafted commit.

At $dayjob we sometimes update submodules to their tip without any
superproject changes just to make sure we use the newest version. Most
of the time the commit messages are along the lines of "updated
submodule x to master".

On one hand Junio is right that the person updating to the newest
submodule stuff has no clue what to write in this message. On the other
hand someone might as well just use this functionality to get all the
tips of all the submodules checked out. He then individually decides
which changes to take by using add but will then still use a commit
message like the one above.

So currently I am more on the "have an automatically generated
commit message" side. Its in a similar corner like merge commits, that
are also generated, for me. We could have it as the default and a
--no-commit option (like merge) for people that want to stage submodules
individually.

Cheers Heiko

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Heiko Voigt]

On Tue, Nov 20, 2012 at 07:19:12AM -0500, W. Trevor King wrote:
> The benefit is that Ævar's
> 
>   $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
> 
> becomes
> 
>   $ git submodule update --pull

There is an important question still unanswered here for me: How does
the submodule get the configuration what the local branch tracks on the
remote side?

Cheers Heiko

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2806 bytes --]

On Fri, Nov 23, 2012 at 04:55:21PM +0100, Heiko Voigt wrote:
> On Tue, Nov 20, 2012 at 11:52:46AM -0800, Junio C Hamano wrote:
> > "W. Trevor King" <wking@tremily.us> writes:
> > 
> > > The superproject gitlink should only be updated after
> > >
> > >   $ git submodule update --pull
> > >
> > > A plain
> > >
> > >   $ git submodule update
> > >
> > > would still checkout the previously-recorded SHA, not the new upstream
> > > tip.
> > 
> > Hrm, doesn't it make the "float at the tip of a branch" mode
> > useless, though?
> 
> How about having a branch config option and reusing our
> submodule.$name.update option for specifying whether the user wants to
> always float to the tip of the branch?

I'm adding "update --pull" as one of the update options in v4, which I
am writing up as we speak ;).

> 1. If submodule.$name.update is pull it would checkout the specified tip.

and pull from the submodule's upstream.  This doesn't need the
recorded $sha1, so I may have to rework the current

  if (clear_local_git_env; cd "$sm_path" && $command "$sha1")

> 2. If submodule.$name.update is checkout or none it would do the usual
>    thing and you need to specify --pull to get the tip.

Exactly.

> So currently I am more on the "have an automatically generated
> commit message" side. Its in a similar corner like merge commits, that
> are also generated, for me. We could have it as the default and a
> --no-commit option (like merge) for people that want to stage submodules
> individually.

This sounds reasonable, but I'd like to postpone message-generation
sugar until we get the basic functionality ironed out.

On Fri, Nov 23, 2012 at 05:03:01PM +0100, Heiko Voigt wrote:
> On Tue, Nov 20, 2012 at 07:19:12AM -0500, W. Trevor King wrote:
> > The benefit is that Ævar's
> > 
> >   $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
> > 
> > becomes
> > 
> >   $ git submodule update --pull
> 
> There is an important question still unanswered here for me: How does
> the submodule get the configuration what the local branch tracks on the
> remote side?

A good point ;).  I'm actaully using submodule.<name>.branch to store
the submodule's local branch name.  The remote branch name for the
pull is implicit, and defaults to something setup according to
branch.autosetupmerge (I think).  If you want to get more complicated
than this, we'll probably have to add submodule.<name>.branch and
submodule.<name>.remote sections to augment the
submodule.<name>.branch setting.  I'm not sure this is worth it.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

On Fri, Nov 23, 2012 at 11:23:29AM -0500, W. Trevor King wrote:
> On Fri, Nov 23, 2012 at 05:03:01PM +0100, Heiko Voigt wrote:
> > There is an important question still unanswered here for me: How does
> > the submodule get the configuration what the local branch tracks on the
> > remote side?
> 
> A good point ;).  I'm actaully using submodule.<name>.branch to store
> the submodule's local branch name.  The remote branch name for the
> pull is implicit, and defaults to something setup according to
> branch.autosetupmerge (I think).  If you want to get more complicated
> than this, we'll probably have to add submodule.<name>.branch and
> submodule.<name>.remote sections to augment the
> submodule.<name>.branch setting.  I'm not sure this is worth it.

These settings are currently stored in

  .git/modules/<name>/config

What we're missing is a place to store them in the .gitmodules file.
I'll poke around in the module-config initialization and wait for
inspiration ;).

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Sascha Cunz]

Am Freitag, 23. November 2012, 16:55:21 schrieb Heiko Voigt:
> I am still a little bit undecided about an automatically crafted commit.
> 
> At $dayjob we sometimes update submodules to their tip without any
> superproject changes just to make sure we use the newest version. Most
> of the time the commit messages are along the lines of "updated
> submodule x to master".
>
> On one hand Junio is right that the person updating to the newest
> submodule stuff has no clue what to write in this message.

I've been thinking about that for a while, when I started using submodules. In 
the end, I concluded, that what I really want to see in the commit message, is 
something similar to $(git shortlog $OLD_SHA1..$NEW_SHA1).

I've scripted that and taught my CI-Server to do it automatically, if 
possible. So most of the time, I really don't want an "automatically crafted 
commit" whenever something causes the tip of a submodule to be at a new SHA1.

Just my $.02, though.

Sascha

Re: Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3742 bytes --]

On Fri, Nov 23, 2012 at 11:23:29AM -0500, W. Trevor King wrote:
> On Fri, Nov 23, 2012 at 04:55:21PM +0100, Heiko Voigt wrote:
> > On Tue, Nov 20, 2012 at 11:52:46AM -0800, Junio C Hamano wrote:
> > > "W. Trevor King" <wking@tremily.us> writes:
> > > 
> > > > The superproject gitlink should only be updated after
> > > >
> > > >   $ git submodule update --pull
> > > >
> > > > A plain
> > > >
> > > >   $ git submodule update
> > > >
> > > > would still checkout the previously-recorded SHA, not the new upstream
> > > > tip.
> > > 
> > > Hrm, doesn't it make the "float at the tip of a branch" mode
> > > useless, though?
> > 
> > How about having a branch config option and reusing our
> > submodule.$name.update option for specifying whether the user wants to
> > always float to the tip of the branch?
> 
> I'm adding "update --pull" as one of the update options in v4, which I
> am writing up as we speak ;).

On second thought, this does not seem to be a good idea.  The current
fancy update styles (--rebase, --merge) are both for cases where you
have local commits in the submodule and are trying to incorporate new
gitlinks from an updated superproject into the submodule's checked out
branch:

  superproject $ cd submod
  superproject $ git checkout next
  submod $ …hack hack hack…
  submod $ git commit …
  submod $ cd ..
  …upstream superproject changes…
  superproject $ git pull
  …updated SHA1 for submod gitlink…
  superproject $ git submodule update --merge
  …merge superproject's gitlink SHA1 into local submod branch…

My submodule.<name>.branch option gives a local branch to
check out:

  …upstream submod changes…
  superproject $ git cd ssubmodule update --pull
  …fetch upstream submod changes and ff-merge into local submodule.<name>.branch…

This seems suitably distinct that bundling it with the other update
options will just add confusion.

So, let's rethink this approach.  I'm trying to pull the upstream
version of my local submod branch.  The difficulties with this are:

1. Checking out a local branch (from the default detached state)
   to do something on it requires an ungainly:

     $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && …'

2. The remote pulling behavior is configured in
   .git/modules/<name>/config, which is not tracked in the repository
   itself.

I'm ok with forcing local users to handle 2 manually (or implicitly),
but 1 is crazy.  Addin submodule.<name>.branch explicitly to
.gitmodules is a step towards fixing 1, but submod pull doesn't match
an existing submodules-implemented workflow.  Perhaps a better choice
would be to borrow the implicit-local-checkout behaviour used by
--rebase and --merge.  We could add

  $ git submodule update --branch

to checkout the gitlinked SHA1 as submodule.<name>.branch in each of
the submodules, leaving the submodules on the .gitmodules-configured
branch.  Effectively (for each submodule):

  $ git branch -f $branch $sha1
  $ git checkout $branch

Then I could use

  $ git submodule foreach 'git pull'

to update my submodule tracking branches (without further "git
submodule" restructuring).

This would help everyone that doesn't like the detached head state (me
and --rebase/--merge users).  I could avoid implementing "update
--pull", and all of the difficulty in configuring upstream merge
choices (2) would be punted to the user making local edits in
.git/modules/<name>/config.

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH v4 0/4] git-submodule add: Add --local-branch option

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

On Fri, Nov 23, 2012 at 12:54:02PM -0500, W. Trevor King wrote:
> We could add
>
>   $ git submodule update --branch
>
> to checkout the gitlinked SHA1 as submodule.<name>.branch in each of
> the submodules, leaving the submodules on the .gitmodules-configured
> branch.  Effectively (for each submodule):
>
>   $ git branch -f $branch $sha1
>   $ git checkout $branch

I haven't gotten any feedback on this as an idea, but perhaps someone
will comment on it as a patch series ;).

Changes since v3:

* --record=… is now --local-branch=…
* Dropped patches 2 ($submodule_ export) and 3 (motivating documentation)
* Added local git-config overrides of .gitmodules' submodule.<name>.branch
* Added `submodule update --branch`

Because you need to recurse through submodules for `update --branch`
even if "$subsha1" == "$sha1", I had to amend the conditional
controlling that block.  This broke one of the existing tests, which I
"fixed" in patch 4.  I think a proper fix would involve rewriting

  (clear_local_git_env; cd "$sm_path" &&
   ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
    test -z "$rev") || git-fetch)) ||
  die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"

but I'm not familiar enough with rev-list to want to dig into that
yet.  If feedback for the earlier three patches is positive, I'll work
up a clean fix and resubmit.

W. Trevor King (4):
  git-submodule add: Add --local-branch option
  git-submodule init: Record submodule.<name>.branch in repository
    config.
  git-submodule update: Add --branch option
  Hack fix for 'submodule update does not fetch already present
    commits'

 Documentation/config.txt        |  9 ++---
 Documentation/git-submodule.txt | 32 ++++++++++++-----
 Documentation/gitmodules.txt    |  5 +++
 git-submodule.sh                | 76 +++++++++++++++++++++++++++++++++--------
 t/t7400-submodule-basic.sh      | 43 +++++++++++++++++++++++
 t/t7406-submodule-update.sh     | 50 ++++++++++++++++++++++++++-
 6 files changed, 187 insertions(+), 28 deletions(-)

-- 
1.8.0.3.g95edff1.dirty

[PATCH v4 1/4] git-submodule add: Add --local-branch option

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This option allows you to record a submodule.<name>.branch option in
.gitmodules.  Git does not currently use this configuration option for
anything, but users have used it for several things, so it makes sense
to add some syntactic sugar for initializing the value.

Current consumers:

Ævar uses this setting to designate the local branch to checkout when
pulling submodule updates:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

as he describes in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.

Earlier version of this patch remained agnostic on the variable usage,
but this was deemed potentially confusing.  Future patches in this
series will extend the submodule command to use the stored value
internally.

[1] https://gerrit.googlesource.com/gerrit/+/master/Documentation/user-submodules.txt

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 12 ++++++++++--
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 19 ++++++++++++++++++-
 t/t7400-submodule-basic.sh      | 25 +++++++++++++++++++++++++
 4 files changed, 58 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..d0b4436 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -9,8 +9,8 @@ git-submodule - Initialize, update or inspect submodules
 SYNOPSIS
 --------
 [verse]
-'git submodule' [--quiet] add [-b branch] [-f|--force]
-	      [--reference <repository>] [--] <repository> [<path>]
+'git submodule' [--quiet] add [-b branch] [--local-branch[=<branch>]]
+	      [-f|--force] [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
@@ -209,6 +209,14 @@ OPTIONS
 --branch::
 	Branch of repository to add as submodule.
 
+--local-branch::
+	Record a branch name used as `submodule.<path>.branch` in
+	`.gitmodules` for future reference.  If you do not list an explicit
+	name here, the name given with `--branch` will be recorded.  If that
+	is not set either, `HEAD` will be recorded.  Because the branch name
+	is optional, you must use the equal-sign form
+	(`--local-branch=<branch>`), not `--local-branch <branch>`.
+
 -f::
 --force::
 	This option is only valid for add and update commands.
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index 4effd78..840ccfe 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -47,6 +47,11 @@ submodule.<name>.update::
 	This config option is overridden if 'git submodule update' is given
 	the '--merge', '--rebase' or '--checkout' options.
 
+submodule.<name>.branch::
+	A local branch name for the submodule (to avoid headless operation).
+	Set with the "--local-branch" option to "git submodule add", or
+	directly using "git config".
+
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
 	submodule. If this option is also present in the submodules entry in
diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..6eed008 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -5,7 +5,7 @@
 # Copyright (c) 2007 Lars Hjemli
 
 dashless=$(basename "$0" | sed -e 's/-/ /')
-USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
+USAGE="[--quiet] add [-b branch] [--local-branch[=<branch>]] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
    or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
@@ -20,6 +20,8 @@ require_work_tree
 
 command=
 branch=
+local_branch=
+local_branch_empty=
 force=
 reference=
 cached=
@@ -257,6 +259,12 @@ cmd_add()
 			branch=$2
 			shift
 			;;
+		--local-branch)
+			local_branch_empty=true
+			;;
+		--local-branch=*)
+			local_branch="${1#*=}"
+			;;
 		-f | --force)
 			force=$1
 			;;
@@ -328,6 +336,11 @@ cmd_add()
 	git ls-files --error-unmatch "$sm_path" > /dev/null 2>&1 &&
 	die "$(eval_gettext "'\$sm_path' already exists in the index")"
 
+	if test -z "$local_branch" && test "$local_branch_empty" = "true"
+	then
+		local_branch="${branch:=HEAD}"
+	fi
+
 	if test -z "$force" && ! git add --dry-run --ignore-missing "$sm_path" > /dev/null 2>&1
 	then
 		eval_gettextln "The following path is ignored by one of your .gitignore files:
@@ -366,6 +379,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
+	if test -n "$local_branch"
+	then
+		git config -f .gitmodules submodule."$sm_path".branch "$local_branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5397037..fc08647 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test -z "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
 		git submodule init
 	) &&
 
@@ -211,6 +212,30 @@ test_expect_success 'submodule add with ./, /.. and // in path' '
 	test_cmp empty untracked
 '
 
+test_expect_success 'submodule add --local-branch' '
+	(
+		cd addtest &&
+		git submodule add --local-branch "$submodurl" submod-follow-head &&
+		test "$(git config -f .gitmodules submodule.submod-follow-head.branch)" = "HEAD"
+	)
+'
+
+test_expect_success 'submodule add --local-branch --branch' '
+	(
+		cd addtest &&
+		git submodule add --local-branch -b initial "$submodurl" submod-auto-follow &&
+		test "$(git config -f .gitmodules submodule.submod-auto-follow.branch)" = "initial"
+	)
+'
+
+test_expect_success 'submodule add --local-branch=<name> --branch' '
+	(
+		cd addtest &&
+		git submodule add --local-branch=final -b initial "$submodurl" submod-follow &&
+		test "$(git config -f .gitmodules submodule.submod-follow.branch)" = "final"
+	)
+'
+
 test_expect_success 'setup - add an example entry to .gitmodules' '
 	GIT_CONFIG=.gitmodules \
 	git config submodule.example.url git://example.com/init.git
-- 
1.8.0.3.g95edff1.dirty

[PATCH v4 2/4] git-submodule init: Record submodule.<name>.branch in repository config.

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This allows users to override the .gitmodules value with a
per-repository value.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/config.txt   |  9 +++++----
 git-submodule.sh           |  7 +++++++
 t/t7400-submodule-basic.sh | 18 ++++++++++++++++++
 3 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 11f320b..1304499 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1994,10 +1994,11 @@ status.submodulesummary::
 submodule.<name>.path::
 submodule.<name>.url::
 submodule.<name>.update::
-	The path within this project, URL, and the updating strategy
-	for a submodule.  These variables are initially populated
-	by 'git submodule init'; edit them to override the
-	URL and other values found in the `.gitmodules` file.  See
+submodule.<name>.branch::
+	The path within this project, URL, the updating strategy, and the
+	local branch name for a submodule.  These variables are initially
+	populated by 'git submodule init'; edit them to override the URL and
+	other values found in the `.gitmodules` file.  See
 	linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
 
 submodule.<name>.fetchRecurseSubmodules::
diff --git a/git-submodule.sh b/git-submodule.sh
index 6eed008..c51b6ae 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -505,6 +505,13 @@ cmd_init()
 		test -n "$(git config submodule."$name".update)" ||
 		git config submodule."$name".update "$upd" ||
 		die "$(eval_gettext "Failed to register update mode for submodule path '\$sm_path'")"
+
+		# Copy "branch" setting when it is not set yet
+		branch="$(git config -f .gitmodules submodule."$name".branch)"
+		test -z "$branch" ||
+		test -n "$(git config submodule."$name".branch)" ||
+		git config submodule."$name".branch "$branch" ||
+		die "$(eval_gettext "Failed to register branch for submodule path '\$sm_path'")"
 	done
 }
 
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index fc08647..3dc8237 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -236,6 +236,24 @@ test_expect_success 'submodule add --local-branch=<name> --branch' '
 	)
 '
 
+test_expect_success 'init should register submodule branch in .git/config' '
+	(
+		cd addtest &&
+		git submodule init &&
+		test "$(git config submodule.submod-follow.branch)" = "final"
+	)
+'
+
+test_expect_success 'local config should override .gitmodules branch' '
+	(
+		cd addtest &&
+		rm -fr submod-follow &&
+		git config submodule.submod-follow.branch initial
+		git submodule init &&
+		test "$(git config submodule.submod-follow.branch)" = "initial"
+	)
+'
+
 test_expect_success 'setup - add an example entry to .gitmodules' '
 	GIT_CONFIG=.gitmodules \
 	git config submodule.example.url git://example.com/init.git
-- 
1.8.0.3.g95edff1.dirty

[PATCH v4 3/4] git-submodule update: Add --branch option

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This allows users to checkout the current
superproject-recorded-submodule-sha as a branch, avoiding the detached
head state that the standard submodule update creates.  This may be
useful for the existing --rebase/--merge workflows which already avoid
detached heads.

It is also useful if you want easy tracking of upstream branches.  The
particular upstream branch to be tracked is configured locally with
.git/modules/<name>/config.  With the new option Ævar's suggested

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitm
odules submodule.$name.branch) && git pull'

reduces to a

  $ git submodule update --branch

after each supermodule .gitmodules edit, and a

  $ git submodule foreach 'git pull'

whenever you feel like updating the submodules.  Your still on you're
own to commit (or not) the updated submodule hashes in the
superproject's .gitmodules.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 20 +++++++++++------
 git-submodule.sh                | 48 +++++++++++++++++++++++++++++----------
 t/t7406-submodule-update.sh     | 50 ++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 98 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index d0b4436..34392a1 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,7 +13,7 @@ SYNOPSIS
 	      [-f|--force] [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
-'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
+'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--branch] [--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
@@ -136,11 +136,11 @@ init::
 
 update::
 	Update the registered submodules, i.e. clone missing submodules and
-	checkout the commit specified in the index of the containing repository.
-	This will make the submodules HEAD be detached unless `--rebase` or
-	`--merge` is specified or the key `submodule.$name.update` is set to
-	`rebase`, `merge` or `none`. `none` can be overridden by specifying
-	`--checkout`.
+	checkout the commit specified in the index of the containing
+	repository.  This will make the submodules HEAD be detached unless
+	`--branch`, `--rebase`, `--merge` is specified or the key
+	`submodule.$name.update` is set to `branch`, `rebase`, `merge` or
+	`none`. `none` can be overridden by specifying `--checkout`.
 +
 If the submodule is not yet initialized, and you just want to use the
 setting as stored in .gitmodules, you can automatically initialize the
@@ -207,7 +207,13 @@ OPTIONS
 
 -b::
 --branch::
-	Branch of repository to add as submodule.
+	When used with the add command, gives the branch of repository to
+	add as submodule.
++
+When used with the update command, checks out a branch named
+`submodule.<name>.branch` (as set by `--local-branch`) pointing at the
+current HEAD SHA-1.  This is useful for commands like `update
+--rebase` that do not work on detached heads.
 
 --local-branch::
 	Record a branch name used as `submodule.<path>.branch` in
diff --git a/git-submodule.sh b/git-submodule.sh
index c51b6ae..28eb4b1 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,7 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b branch] [--local-branch[=<branch>]] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
-   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--branch] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--] [<path>...]"
@@ -539,6 +539,9 @@ cmd_update()
 		-f|--force)
 			force=$1
 			;;
+		-b|--branch)
+			update="branch"
+			;;
 		-r|--rebase)
 			update="rebase"
 			;;
@@ -593,6 +596,7 @@ cmd_update()
 		fi
 		name=$(module_name "$sm_path") || exit
 		url=$(git config submodule."$name".url)
+		branch=$(git config submodule."$name".branch)
 		if ! test -z "$update"
 		then
 			update_module=$update
@@ -627,7 +631,7 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
 		fi
 
-		if test "$subsha1" != "$sha1" -o -n "$force"
+		if test "$subsha1" != "$sha1" -o -n "$force" -o "$update_module" = "branch"
 		then
 			subforce=$force
 			# If we don't already have a -f flag and the submodule has never been checked out
@@ -650,16 +654,21 @@ Maybe you want to use 'update --init'?")"
 			case ";$cloned_modules;" in
 			*";$name;"*)
 				# then there is no local change to integrate
-				update_module= ;;
+				case "$update_module" in
+					rebase|merge)
+						update_module=
+						;;
+				esac
+				;;
 			esac
 
 			must_die_on_failure=
 			case "$update_module" in
 			rebase)
 				command="git rebase"
-				die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$sm_path'")"
+				die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$sm_path'")"	
 				say_msg="$(eval_gettext "Submodule path '\$sm_path': rebased into '\$sha1'")"
-				must_die_on_failure=yes
+			must_die_on_failure=yes
 				;;
 			merge)
 				command="git merge"
@@ -674,15 +683,30 @@ Maybe you want to use 'update --init'?")"
 				;;
 			esac
 
-			if (clear_local_git_env; cd "$sm_path" && $command "$sha1")
+			if test "$subsha1" != "$sha1" -o -n "$force"
 			then
-				say "$say_msg"
-			elif test -n "$must_die_on_failure"
+				if (clear_local_git_env; cd "$sm_path" && $command "$sha1")
+				then
+					say "$say_msg"
+				elif test -n "$must_die_on_failure"
+				then
+					die_with_status 2 "$die_msg"
+				else
+					err="${err};$die_msg"
+					continue
+				fi
+			fi
+
+			if test "$update_module" = "branch" -a -n "$branch"
 			then
-				die_with_status 2 "$die_msg"
-			else
-				err="${err};$die_msg"
-				continue
+				if (clear_local_git_env; cd "$sm_path" &&
+					git branch -f "$branch" "$sha1" &&
+					git checkout "$branch")
+				then
+					say "$(eval_gettext "Submodule path '\$sm_path': checked out branch '\$branch'")"
+				else
+					err="${err};$(eval_gettext "Unable to checkout branch '\$branch' in submodule path '\$sm_path'")"
+				fi
 			fi
 		fi
 
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 1542653..c876a8b 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -6,7 +6,8 @@
 test_description='Test updating submodules
 
 This test verifies that "git submodule update" detaches the HEAD of the
-submodule and "git submodule update --rebase/--merge" does not detach the HEAD.
+submodule and "git submodule update --branch/--rebase/--merge" does not
+detach the HEAD.
 '
 
 . ./test-lib.sh
@@ -135,6 +136,53 @@ test_expect_success 'submodule update --force forcibly checks out submodules' '
 	)
 '
 
+test_expect_success 'submodule update --branch detaches without submodule.<name>.branch' '
+	(cd super/submodule &&
+	  git checkout master
+	) &&
+	(cd super &&
+	 (cd submodule &&
+	  compare_head
+	 ) &&
+	 git submodule update --branch submodule &&
+	 (cd submodule &&
+	  test "$(git status -s file)" = ""
+	 )
+	)
+'
+
+test_expect_success 'submodule update --branch staying on master' '
+	(cd super/submodule &&
+	  git checkout master
+	) &&
+	(cd super &&
+	 (cd submodule &&
+	  compare_head
+	 ) &&
+	 git config submodule.submodule.branch master
+	 git submodule update --branch submodule &&
+	 cd submodule &&
+	 test "refs/heads/master" = "$(git symbolic-ref -q HEAD)" &&
+	 compare_head
+	)
+'
+
+test_expect_success 'submodule update --branch creating a new branch' '
+	(cd super/submodule &&
+	  git checkout master
+	) &&
+	(cd super &&
+	 (cd submodule &&
+	  compare_head
+	 ) &&
+	 git config submodule.submodule.branch new-branch
+	 git submodule update --branch submodule &&
+	 cd submodule &&
+	 test "refs/heads/new-branch" = "$(git symbolic-ref -q HEAD)" &&
+	 compare_head
+	)
+'
+
 test_expect_success 'submodule update --rebase staying on master' '
 	(cd super/submodule &&
 	  git checkout master
-- 
1.8.0.3.g95edff1.dirty

[PATCH v4 4/4] Hack fix for 'submodule update does not fetch already present commits'

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

---
 git-submodule.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 28eb4b1..f4a681c 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -640,7 +640,7 @@ Maybe you want to use 'update --init'?")"
 				subforce="-f"
 			fi
 
-			if test -z "$nofetch"
+			if test -z "$nofetch" -a "$subsha1" != "$sha1"
 			then
 				# Run fetch only if $sha1 isn't present or it
 				# is not reachable from a ref.
-- 
1.8.0.3.g95edff1.dirty

Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option

[Heiko Voigt]

Hi,

On Mon, Nov 26, 2012 at 04:00:15PM -0500, W. Trevor King wrote:
> From: "W. Trevor King" <wking@tremily.us>
> 
> On Fri, Nov 23, 2012 at 12:54:02PM -0500, W. Trevor King wrote:
> > We could add
> >
> >   $ git submodule update --branch
> >
> > to checkout the gitlinked SHA1 as submodule.<name>.branch in each of
> > the submodules, leaving the submodules on the .gitmodules-configured
> > branch.  Effectively (for each submodule):
> >
> >   $ git branch -f $branch $sha1
> >   $ git checkout $branch
> 
> I haven't gotten any feedback on this as an idea, but perhaps someone
> will comment on it as a patch series ;).

I am not sure I understand you correctly. You are suggesting that the
branch option as an alias for the registered SHA1 in the superproject?

I though the goal of your series was that you want to track submodules
branch which come from the remote side?

Doing the above does not assist you much in that does it?

I would think more of some convention like:

	$ git checkout -t origin/$branch

when first initialising the submodule with e.g.

	$ git submodule update --init --branch

Then later calls of

	$ git submodule update --branch

would have a branch configured to pull from. I imagine that results in
a similar behavior gerrit is doing on the server side?

> Changes since v3:
> 
> * --record=??? is now --local-branch=???
> * Dropped patches 2 ($submodule_ export) and 3 (motivating documentation)
> * Added local git-config overrides of .gitmodules' submodule.<name>.branch
> * Added `submodule update --branch`

I would prefer if we could squash all these commits together into one
since it seems to me one logical step, using the new variable for update
belongs together with its configuration on initialization.

How about reusing the -b|--branch option for add? Since we only change
the behavior when submodule.$name.update is set to branch it seems
reasonable to me. Opinions?

> Because you need to recurse through submodules for `update --branch`
> even if "$subsha1" == "$sha1", I had to amend the conditional
> controlling that block.  This broke one of the existing tests, which I
> "fixed" in patch 4.  I think a proper fix would involve rewriting
> 
>   (clear_local_git_env; cd "$sm_path" &&
>    ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
>     test -z "$rev") || git-fetch)) ||
>   die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
> 
> but I'm not familiar enough with rev-list to want to dig into that
> yet.  If feedback for the earlier three patches is positive, I'll work
> up a clean fix and resubmit.

You probably need to separate your handling here. The comparison of the
currently checked out sha1 and the recorded sha1 is an optimization
which skips unnecessary fetching in case the submodules commits are
already correct. This code snippet checks whether the to be checked out
sha1 is already local and also skips the fetch if it is. We should not
break that.

Maybe we need an else block here and possibly extract the current code
inside the if statement into a function. E.g. that the final code looks
something like this:

	if test "$subsha1" != "$sha1"
	then
		handle_on_demand_fetch_update ...
	else
		handle_tracked_branch_update ...
	fi

Not sure about the function names though. If we decide to go that route:
The extraction into a function should go in an extra preparation patch
which does not change any functionality.

I will reply to the patches for further comments.

Cheers Heiko

Re: [PATCH v4 3/4] git-submodule update: Add --branch option

[Heiko Voigt]

On Mon, Nov 26, 2012 at 04:00:18PM -0500, W. Trevor King wrote:
> From: "W. Trevor King" <wking@tremily.us>
> 
> This allows users to checkout the current
> superproject-recorded-submodule-sha as a branch, avoiding the detached
> head state that the standard submodule update creates.  This may be
> useful for the existing --rebase/--merge workflows which already avoid
> detached heads.
> 
> It is also useful if you want easy tracking of upstream branches.  The
> particular upstream branch to be tracked is configured locally with
> .git/modules/<name>/config.  With the new option Ævar's suggested
> 
>   $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitm
> odules submodule.$name.branch) && git pull'
> 
> reduces to a
> 
>   $ git submodule update --branch
> 
> after each supermodule .gitmodules edit, and a
> 
>   $ git submodule foreach 'git pull'
> 
> whenever you feel like updating the submodules.  Your still on you're
> own to commit (or not) the updated submodule hashes in the
> superproject's .gitmodules.
> 
> Signed-off-by: W. Trevor King <wking@tremily.us>
> ---
>  Documentation/git-submodule.txt | 20 +++++++++++------
>  git-submodule.sh                | 48 +++++++++++++++++++++++++++++----------
>  t/t7406-submodule-update.sh     | 50 ++++++++++++++++++++++++++++++++++++++++-
>  3 files changed, 98 insertions(+), 20 deletions(-)
> 
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index d0b4436..34392a1 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -13,7 +13,7 @@ SYNOPSIS
>  	      [-f|--force] [--reference <repository>] [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
> -'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
> +'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--branch] [--rebase]
>  	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
>  	      [commit] [--] [<path>...]
> @@ -136,11 +136,11 @@ init::
>  
>  update::
>  	Update the registered submodules, i.e. clone missing submodules and
> -	checkout the commit specified in the index of the containing repository.
> -	This will make the submodules HEAD be detached unless `--rebase` or
> -	`--merge` is specified or the key `submodule.$name.update` is set to
> -	`rebase`, `merge` or `none`. `none` can be overridden by specifying
> -	`--checkout`.
> +	checkout the commit specified in the index of the containing
> +	repository.  This will make the submodules HEAD be detached unless
> +	`--branch`, `--rebase`, `--merge` is specified or the key
> +	`submodule.$name.update` is set to `branch`, `rebase`, `merge` or
> +	`none`. `none` can be overridden by specifying `--checkout`.
>  +
>  If the submodule is not yet initialized, and you just want to use the
>  setting as stored in .gitmodules, you can automatically initialize the
> @@ -207,7 +207,13 @@ OPTIONS
>  
>  -b::
>  --branch::
> -	Branch of repository to add as submodule.
> +	When used with the add command, gives the branch of repository to
> +	add as submodule.
> ++
> +When used with the update command, checks out a branch named
> +`submodule.<name>.branch` (as set by `--local-branch`) pointing at the
> +current HEAD SHA-1.  This is useful for commands like `update
> +--rebase` that do not work on detached heads.

Since you are reusing this option for update it further convinces me
that reusing it for add makes sense and simplifies the logic for users.

I think an optional argument for --branch would be nice in the update
case:

	$ git submodule update --branch=master

would then allow a user that has not configured anything (except the
branch tracking info in the submodule of course) to pull all submodules
master branches.

> diff --git a/git-submodule.sh b/git-submodule.sh
> index c51b6ae..28eb4b1 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -627,7 +631,7 @@ Maybe you want to use 'update --init'?")"
>  			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
>  		fi
>  
> -		if test "$subsha1" != "$sha1" -o -n "$force"
> +		if test "$subsha1" != "$sha1" -o -n "$force" -o "$update_module" = "branch"

As said before I think separating your code from the current update
logic will simplify the handling below.

>  		then
>  			subforce=$force
>  			# If we don't already have a -f flag and the submodule has never been checked out
> @@ -650,16 +654,21 @@ Maybe you want to use 'update --init'?")"
>  			case ";$cloned_modules;" in
>  			*";$name;"*)
>  				# then there is no local change to integrate
> -				update_module= ;;
> +				case "$update_module" in
> +					rebase|merge)
> +						update_module=
> +						;;
> +				esac
> +				;;
>  			esac
>  
>  			must_die_on_failure=
>  			case "$update_module" in
>  			rebase)
>  				command="git rebase"
> -				die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$sm_path'")"
> +				die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$sm_path'")"	
>  				say_msg="$(eval_gettext "Submodule path '\$sm_path': rebased into '\$sha1'")"
> -				must_die_on_failure=yes
> +			must_die_on_failure=yes

Please always cleanup whitespace changes.

>  				;;
>  			merge)
>  				command="git merge"
> @@ -674,15 +683,30 @@ Maybe you want to use 'update --init'?")"
>  				;;
>  			esac
>  
>  			then
> -				die_with_status 2 "$die_msg"
> -			else
> -				err="${err};$die_msg"
> -				continue
> +				if (clear_local_git_env; cd "$sm_path" &&
> +					git branch -f "$branch" "$sha1" &&
> +					git checkout "$branch")

You wrote in earlier emails that you wanted to protect the user from
non-fastforward changes. So I would expect a

	$ git pull --ff-only

here and the setup of that in the initialization of the submodule.

BTW, I am more and more convinced that an automatically manufactured
commit on update with --branch should be the default. What do other
think? Sascha raised a concern that he would not want this, but as far as
I understood he let the CI-server do that so I see no downside to
natively adding that to git. People who want to manually craft those
commits can still amend the generated commit. Since this is all about
helping people keeping their submodules updated why not go the full way?

Cheers Heiko

Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 4525 bytes --]

On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> On Mon, Nov 26, 2012 at 04:00:15PM -0500, W. Trevor King wrote:
> > From: "W. Trevor King" <wking@tremily.us>
> > 
> > On Fri, Nov 23, 2012 at 12:54:02PM -0500, W. Trevor King wrote:
> > > We could add
> > >
> > >   $ git submodule update --branch
> > >
> > > to checkout the gitlinked SHA1 as submodule.<name>.branch in each of
> > > the submodules, leaving the submodules on the .gitmodules-configured
> > > branch.  Effectively (for each submodule):
> > >
> > >   $ git branch -f $branch $sha1
> > >   $ git checkout $branch
> > 
> > I haven't gotten any feedback on this as an idea, but perhaps someone
> > will comment on it as a patch series ;).
> 
> I am not sure I understand you correctly. You are suggesting that the
> branch option as an alias for the registered SHA1 in the superproject?
> 
> I though the goal of your series was that you want to track submodules
> branch which come from the remote side?

That's what I'd initially thought, but when I went to implement
`update --pull`, I realized that

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && …'

is using submodule.<name>.branch as the local branch name.  The remote
branch name was actually setup in .git/modules/<name>/config during
the initial "clone -b <branch> …".

The v4 series leaves the remote branch amigious, but it helps you
point the local branch at the right hash so that future calls to

  $ git submodule foreach 'git pull'

can use the branch's .git/modules/<name>/config settings.

> I would think more of some convention like:
> 
> 	$ git checkout -t origin/$branch
> 
> when first initialising the submodule with e.g.
> 
> 	$ git submodule update --init --branch
> 
> Then later calls of
> 
> 	$ git submodule update --branch
> 
> would have a branch configured to pull from. I imagine that results in
> a similar behavior gerrit is doing on the server side?

That sounds like it's doing pretty much the same thing.  Can you think
of a test that would distinguish it from my current v4 implementation?

> > Changes since v3:
> > 
> > * --record=??? is now --local-branch=???
> > * Dropped patches 2 ($submodule_ export) and 3 (motivating documentation)
> > * Added local git-config overrides of .gitmodules' submodule.<name>.branch
> > * Added `submodule update --branch`
> 
> I would prefer if we could squash all these commits together into one
> since it seems to me one logical step, using the new variable for update
> belongs together with its configuration on initialization.
> 
> How about reusing the -b|--branch option for add? Since we only change
> the behavior when submodule.$name.update is set to branch it seems
> reasonable to me. Opinions?

That was the approach I used in v1, but people were concerned that we
would be stomping on previously unclaimed config space.  Since noone
has pointed out other uses besides Gerrit's very similar case, I'm not
sure if that is still an issue.

> > Because you need to recurse through submodules for `update --branch`
> > even if "$subsha1" == "$sha1", I had to amend the conditional
> > controlling that block.  This broke one of the existing tests, which I
> > "fixed" in patch 4.  I think a proper fix would involve rewriting
> > 
> >   (clear_local_git_env; cd "$sm_path" &&
> >    ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
> >     test -z "$rev") || git-fetch)) ||
> >   die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
> > 
> > but I'm not familiar enough with rev-list to want to dig into that
> > yet.  If feedback for the earlier three patches is positive, I'll work
> > up a clean fix and resubmit.
> 
> You probably need to separate your handling here. The comparison of the
> currently checked out sha1 and the recorded sha1 is an optimization
> which skips unnecessary fetching in case the submodules commits are
> already correct. This code snippet checks whether the to be checked out
> sha1 is already local and also skips the fetch if it is. We should not
> break that.

Agreed.  However, determining if the target $sha1 is local should have
nothing to do with the current checked out $subsha1.

Thanks for the feedback!
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 704 bytes --]

On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> I would prefer if we could squash all these commits together into
> one since it seems to me one logical step, using the new variable
> for update belongs together with its configuration on
> initialization.

Works for me.  I could also try to rework the patch boundaries if a
monolithic patch is not acceptable.  I agree that the current
documentation assignments are fairly arbitrary.  If I don't hear from
anyone in favor of keeping them separate, v5 will be monolithic.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option

[Heiko Voigt]

Hi,

I just realized that I gave you an confusing suggestion.

On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> 	if test "$subsha1" != "$sha1"
> 	then
> 		handle_on_demand_fetch_update ...
> 	else
> 		handle_tracked_branch_update ...
> 	fi

That obviously does not work. Here I meant of course something like:

 	if test "$update_module" = "branch"
 	then
 		handle_tracked_branch_update ...
 	else
 		handle_on_demand_fetch_update ...
 	fi

Cheers Heiko

Re: [PATCH v4 3/4] git-submodule update: Add --branch option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3615 bytes --]

On Tue, Nov 27, 2012 at 07:51:42PM +0100, Heiko Voigt wrote:
> On Mon, Nov 26, 2012 at 04:00:18PM -0500, W. Trevor King wrote:
> >  -b::
> >  --branch::
> > -	Branch of repository to add as submodule.
> > +	When used with the add command, gives the branch of repository to
> > +	add as submodule.
> > ++
> > +When used with the update command, checks out a branch named
> > +`submodule.<name>.branch` (as set by `--local-branch`) pointing at the
> > +current HEAD SHA-1.  This is useful for commands like `update
> > +--rebase` that do not work on detached heads.
> 
> Since you are reusing this option for update it further convinces me
> that reusing it for add makes sense and simplifies the logic for users.
> 
> I think an optional argument for --branch would be nice in the update
> case:
> 
> 	$ git submodule update --branch=master
> 
> would then allow a user that has not configured anything (except the
> branch tracking info in the submodule of course) to pull all submodules
> master branches.

Sounds good to me.  Remember that this is checking the branch and
pointing it at $sha1 (preparing for the pull), not pulling remote
branches.  The pull happens in a later

  $ git submodules foreach 'git pull'

> > diff --git a/git-submodule.sh b/git-submodule.sh
> > index c51b6ae..28eb4b1 100755
> > --- a/git-submodule.sh
> > +++ b/git-submodule.sh
> > @@ -627,7 +631,7 @@ Maybe you want to use 'update --init'?")"
> >  			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
> >  		fi
> >  
> > -		if test "$subsha1" != "$sha1" -o -n "$force"
> > +		if test "$subsha1" != "$sha1" -o -n "$force" -o "$update_module" = "branch"
> 
> As said before I think separating your code from the current update
> logic will simplify the handling below.

This felt less invasive (it avoids duplicating the recursion logic),
but I don't mind breaking it into a separate function/block.

> >  			must_die_on_failure=
> >  			case "$update_module" in
> >  			rebase)
> >  				command="git rebase"
> > -				die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$sm_path'")"
> > +				die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$sm_path'")"	
> >  				say_msg="$(eval_gettext "Submodule path '\$sm_path': rebased into '\$sha1'")"
> > -				must_die_on_failure=yes
> > +			must_die_on_failure=yes
> 
> Please always cleanup whitespace changes.

Oops, sloppy me.  Will fix.

> >  			then
> > -				die_with_status 2 "$die_msg"
> > -			else
> > -				err="${err};$die_msg"
> > -				continue
> > +				if (clear_local_git_env; cd "$sm_path" &&
> > +					git branch -f "$branch" "$sha1" &&
> > +					git checkout "$branch")
> 
> You wrote in earlier emails that you wanted to protect the user from
> non-fastforward changes. So I would expect a
> 
> 	$ git pull --ff-only

I'm not pulling here, I'm doing a regular `submodule update`, and
after that's done I checkout the branch pointing at the $sha1 to which
the branch was just updated.  All the submodule-state-clobbering
caveats of a usual `submodule update` still apply to this new
`submodule update --branch`, and I'm fine with that.

> BTW, I am more and more convinced that an automatically manufactured
> commit on update with --branch should be the default.

Again, there's nothing to update.  The pull happens in a separate
step.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v4 4/4] Hack fix for 'submodule update does not fetch already present commits'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1886 bytes --]

On Tue, Nov 27, 2012 at 02:01:05PM -0500, W. Trevor King wrote:
> On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> > On Mon, Nov 26, 2012 at 04:00:15PM -0500, W. Trevor King wrote:
> > > Because you need to recurse through submodules for `update --branch`
> > > even if "$subsha1" == "$sha1", I had to amend the conditional
> > > controlling that block.  This broke one of the existing tests, which I
> > > "fixed" in patch 4.  I think a proper fix would involve rewriting
> > > 
> > >   (clear_local_git_env; cd "$sm_path" &&
> > >    ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
> > >     test -z "$rev") || git-fetch)) ||
> > >   die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
> > > 
> > > but I'm not familiar enough with rev-list to want to dig into that
> > > yet.  If feedback for the earlier three patches is positive, I'll work
> > > up a clean fix and resubmit.
> > 
> > You probably need to separate your handling here. The comparison of the
> > currently checked out sha1 and the recorded sha1 is an optimization
> > which skips unnecessary fetching in case the submodules commits are
> > already correct. This code snippet checks whether the to be checked out
> > sha1 is already local and also skips the fetch if it is. We should not
> > break that.
> 
> Agreed.  However, determining if the target $sha1 is local should have
> nothing to do with the current checked out $subsha1.

Erm, I clearly wasn't getting enough sleep heading into yesterday,
because when I drop the hack patch #4, reinstall, and retest, I no
longer get the bad-fetch error.  I'm not quite sure what was going on,
but please pretend I never mentioned it ;).

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v4 2/4] git-submodule init: Record submodule.<name>.branch in repository config.

[Jens Lehmann]

Am 26.11.2012 22:00, schrieb W. Trevor King:
> From: "W. Trevor King" <wking@tremily.us>
> 
> This allows users to override the .gitmodules value with a
> per-repository value.

Your intentions makes lots of sense, but your patch does more than
that. Copying the branch setting into .git/config sets the initial
branch setting into stone. That makes it impossible to have a branch
"foo" in the superproject using a branch "bar" in a submodule and
another superproject branch "frotz" using branch "nitfol" for the
same submodule. You should use the branch setting from .git/config
if present and fall back to the branch setting from .gitmodules if
not, which would enable the user to have her own setting if she
doesn't like what upstream provides but would still enable others
to follow different submodule branches in different superproject
branches.

> Signed-off-by: W. Trevor King <wking@tremily.us>
> ---
>  Documentation/config.txt   |  9 +++++----
>  git-submodule.sh           |  7 +++++++
>  t/t7400-submodule-basic.sh | 18 ++++++++++++++++++
>  3 files changed, 30 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 11f320b..1304499 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1994,10 +1994,11 @@ status.submodulesummary::
>  submodule.<name>.path::
>  submodule.<name>.url::
>  submodule.<name>.update::
> -	The path within this project, URL, and the updating strategy
> -	for a submodule.  These variables are initially populated
> -	by 'git submodule init'; edit them to override the
> -	URL and other values found in the `.gitmodules` file.  See
> +submodule.<name>.branch::
> +	The path within this project, URL, the updating strategy, and the
> +	local branch name for a submodule.  These variables are initially
> +	populated by 'git submodule init'; edit them to override the URL and
> +	other values found in the `.gitmodules` file.  See
>  	linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
>  
>  submodule.<name>.fetchRecurseSubmodules::
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 6eed008..c51b6ae 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -505,6 +505,13 @@ cmd_init()
>  		test -n "$(git config submodule."$name".update)" ||
>  		git config submodule."$name".update "$upd" ||
>  		die "$(eval_gettext "Failed to register update mode for submodule path '\$sm_path'")"
> +
> +		# Copy "branch" setting when it is not set yet
> +		branch="$(git config -f .gitmodules submodule."$name".branch)"
> +		test -z "$branch" ||
> +		test -n "$(git config submodule."$name".branch)" ||
> +		git config submodule."$name".branch "$branch" ||
> +		die "$(eval_gettext "Failed to register branch for submodule path '\$sm_path'")"
>  	done
>  }
>  
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index fc08647..3dc8237 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -236,6 +236,24 @@ test_expect_success 'submodule add --local-branch=<name> --branch' '
>  	)
>  '
>  
> +test_expect_success 'init should register submodule branch in .git/config' '
> +	(
> +		cd addtest &&
> +		git submodule init &&
> +		test "$(git config submodule.submod-follow.branch)" = "final"
> +	)
> +'
> +
> +test_expect_success 'local config should override .gitmodules branch' '
> +	(
> +		cd addtest &&
> +		rm -fr submod-follow &&
> +		git config submodule.submod-follow.branch initial
> +		git submodule init &&
> +		test "$(git config submodule.submod-follow.branch)" = "initial"
> +	)
> +'
> +
>  test_expect_success 'setup - add an example entry to .gitmodules' '
>  	GIT_CONFIG=.gitmodules \
>  	git config submodule.example.url git://example.com/init.git
> 

Re: Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option

[Heiko Voigt]

Hi,

On Tue, Nov 27, 2012 at 02:01:05PM -0500, W. Trevor King wrote:
> On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> The v4 series leaves the remote branch amigious, but it helps you
> point the local branch at the right hash so that future calls to
> 
>   $ git submodule foreach 'git pull'
> 
> can use the branch's .git/modules/<name>/config settings.

But IMO thats the functionality which should be implemented in submodule
update and not left to the user.

> > I would think more of some convention like:
> > 
> > 	$ git checkout -t origin/$branch
> > 
> > when first initialising the submodule with e.g.
> > 
> > 	$ git submodule update --init --branch
> > 
> > Then later calls of
> > 
> > 	$ git submodule update --branch
> > 
> > would have a branch configured to pull from. I imagine that results in
> > a similar behavior gerrit is doing on the server side?
> 
> That sounds like it's doing pretty much the same thing.  Can you think
> of a test that would distinguish it from my current v4 implementation?

Well the main difference is that gerrit is automatically updating the
superproject AFAIK. I would like it if we could implement the same
workflow support in the submodule script. It seems to me that this is
already proven to be useful workflow.

I do not have a test but a small draft diff (completely untested, quick and
dirty) to illustrate the approach I am talking about.

You can find the whole change at

https://github.com/hvoigt/git/commits/hv/floating_submodules_draft

and the interesting patch for easy commenting below[1].

> > How about reusing the -b|--branch option for add? Since we only change
> > the behavior when submodule.$name.update is set to branch it seems
> > reasonable to me. Opinions?
> 
> That was the approach I used in v1, but people were concerned that we
> would be stomping on previously unclaimed config space.  Since noone
> has pointed out other uses besides Gerrit's very similar case, I'm not
> sure if that is still an issue.

Could you point me to that mail? I cannot seem to find it in my archive.

> > > Because you need to recurse through submodules for `update --branch`
> > > even if "$subsha1" == "$sha1", I had to amend the conditional
> > > controlling that block.  This broke one of the existing tests, which I
> > > "fixed" in patch 4.  I think a proper fix would involve rewriting
> > > 
> > >   (clear_local_git_env; cd "$sm_path" &&
> > >    ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
> > >     test -z "$rev") || git-fetch)) ||
> > >   die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
> > > 
> > > but I'm not familiar enough with rev-list to want to dig into that
> > > yet.  If feedback for the earlier three patches is positive, I'll work
> > > up a clean fix and resubmit.
> > 
> > You probably need to separate your handling here. The comparison of the
> > currently checked out sha1 and the recorded sha1 is an optimization
> > which skips unnecessary fetching in case the submodules commits are
> > already correct. This code snippet checks whether the to be checked out
> > sha1 is already local and also skips the fetch if it is. We should not
> > break that.
> 
> Agreed.  However, determining if the target $sha1 is local should have
> nothing to do with the current checked out $subsha1.

See my draft or the diff below for an illustration of the splitup.

Cheers Heiko

[1]
diff --git a/git-submodule.sh b/git-submodule.sh
index 9ad4370..3fa1465 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -183,6 +183,7 @@ module_clone()
 	sm_path=$1
 	url=$2
 	reference="$3"
+	branch="$4"
 	quiet=
 	if test -n "$GIT_QUIET"
 	then
@@ -209,6 +210,8 @@ module_clone()
 			clear_local_git_env
 			git clone $quiet -n ${reference:+"$reference"} \
 				--separate-git-dir "$gitdir" "$url" "$sm_path"
+			test -n "$branch" && (cd $sm_path &&
+				git checkout -t origin/$branch)
 		) ||
 		die "$(eval_gettext "Clone of '\$url' into submodule path '\$sm_path' failed")"
 	fi
@@ -361,7 +364,7 @@ Use -f if you really want to add it." >&2
 
 	else
 
-		module_clone "$sm_path" "$realrepo" "$reference" || exit
+		module_clone "$sm_path" "$realrepo" "$reference" "$local_branch" || exit
 		(
 			clear_local_git_env
 			cd "$sm_path" &&
@@ -577,6 +580,12 @@ handle_on_demand_update () {
 	fi
 }
 
+handle_tracking_branch_update () {
+	(clear_local_git_env; cd "$sm_path" &&
+		git-checkout $branch && git-pull --ff-only) ||
+	die "$(eval_gettext "Unable to pull branch '\$branch' in submodule path '\$sm_path'")"
+}
+
 #
 # Update each submodule path to correct revision, using clone and checkout as needed
 #
@@ -648,6 +657,7 @@ cmd_update()
 	cloned_modules=
 	module_list "$@" | {
 	err=
+	floating_submodules=
 	while read mode sha1 stage sm_path
 	do
 		die_if_unmatched "$mode"
@@ -684,7 +694,7 @@ Maybe you want to use 'update --init'?")"
 
 		if ! test -d "$sm_path"/.git -o -f "$sm_path"/.git
 		then
-			module_clone "$sm_path" "$url" "$reference"|| exit
+			module_clone "$sm_path" "$url" "$reference" "$branch" || exit
 			cloned_modules="$cloned_modules;$name"
 			subsha1=
 		else
@@ -693,7 +703,13 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
 		fi
 
-		handle_on_demand_update
+		if test "$update_module" = "branch"
+		then
+			handle_tracking_branch_update
+			floating_submodules="$floating_submodules $sm_path"
+		else
+			handle_on_demand_update
+		fi
 
 		if test -n "$recursive"
 		then
@@ -727,6 +743,11 @@ Maybe you want to use 'update --init'?")"
 		IFS=$OIFS
 		exit 1
 	fi
+	if test -n "$floating_submodules"
+	then
+		git add $floating_submodules &&
+		git commit -m "Updated submodules"
+	fi
 	}
 }

Re: [PATCH v4 2/4] git-submodule init: Record submodule.<name>.branch in repository config.

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

On Wed, Nov 28, 2012 at 12:19:04AM +0100, Jens Lehmann wrote:
> Am 26.11.2012 22:00, schrieb W. Trevor King:
> > From: "W. Trevor King" <wking@tremily.us>
> > 
> > This allows users to override the .gitmodules value with a
> > per-repository value.
> 
> [snip problems].  You should use the branch setting from .git/config
> if present and fall back to the branch setting from .gitmodules if
> not, which would enable the user to have her own setting if she
> doesn't like what upstream provides but would still enable others to
> follow different submodule branches in different superproject
> branches.

Sounds good.  Will fix in v5.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 5913 bytes --]

On Wed, Nov 28, 2012 at 12:28:58AM +0100, Heiko Voigt wrote:
> On Tue, Nov 27, 2012 at 02:01:05PM -0500, W. Trevor King wrote:
> > On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> > The v4 series leaves the remote branch amigious, but it helps you
> > point the local branch at the right hash so that future calls to
> > 
> >   $ git submodule foreach 'git pull'
> > 
> > can use the branch's .git/modules/<name>/config settings.
> 
> But IMO thats the functionality which should be implemented in submodule
> update and not left to the user.

Then you might need submodule.<name>.local-branch,
submodule.<name>.remote-repository, and submodule.<name>.remote-branch
to configure

  $ git checkout submodule.<name>.local-branch
  $ git pull submodule.<name>.remote-repository submodule.<name>.remote-branch

and this would ignore the $sha1 stored in the gitlink (which all of
the other update commands use).  This ignoring-the-$sha1 bit made me
think that a built-in pull wasn't a good fit for 'submodule update'.
Maybe if it went into a new 'submodule pull'?  Then users have a clear
distinction:

* 'update' to push superproject $sha1 changes into the submodules
* 'pull' to push upstream-branch changes into the submodules

> > > I would think more of some convention like:
> > > 
> > > 	$ git checkout -t origin/$branch
> > > 
> > > when first initialising the submodule with e.g.
> > > 
> > > 	$ git submodule update --init --branch
> > > 
> > > Then later calls of
> > > 
> > > 	$ git submodule update --branch
> > > 
> > > would have a branch configured to pull from. I imagine that results in
> > > a similar behavior gerrit is doing on the server side?
> > 
> > That sounds like it's doing pretty much the same thing.  Can you think
> > of a test that would distinguish it from my current v4 implementation?
> 
> Well the main difference is that gerrit is automatically updating the
> superproject AFAIK. I would like it if we could implement the same
> workflow support in the submodule script. It seems to me that this is
> already proven to be useful workflow.

Ah, sorry, I meant the configuring which remote branch you were
pulling from happens at submodule initialization (via .git/modules/…)
for both your workflow and my v4.

You're right that having a builtin pull is different from my v4.

> https://github.com/hvoigt/git/commits/hv/floating_submodules_draft

I looked over this before, but maybe not thoroughly enough ;).

> > > How about reusing the -b|--branch option for add? Since we only change
> > > the behavior when submodule.$name.update is set to branch it seems
> > > reasonable to me. Opinions?
> > 
> > That was the approach I used in v1, but people were concerned that we
> > would be stomping on previously unclaimed config space.  Since noone
> > has pointed out other uses besides Gerrit's very similar case, I'm not
> > sure if that is still an issue.
> 
> Could you point me to that mail? I cannot seem to find it in my archive.

Hmm.  It seems like Phil's initial response was (accidentally?) off
list.  The relevant portion was:

On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote:
> Some projects now use the 'branch' config value to record the tracking
> branch for the submodule.  Some ascribe different meaning to the
> configuration if the value is given vs. undefined.  For example, see
> the Gerrit submodule-subscription mechanism.  This change will cause
> those workflows to behave differently than they do now.
>
> I do like the idea, but I wish it had a different name for the
> recording.  Maybe --record-branch=${BRANCH} as an extra switch so the
> action is explicitly requested.

As I said, I'm happy to go back to --branch if opinions have changed.

On Wed, Nov 28, 2012 at 12:28:58AM +0100, Heiko Voigt wrote:
> On Tue, Nov 27, 2012 at 02:01:05PM -0500, W. Trevor King wrote:
> > On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> > > > Because you need to recurse through submodules for `update --branch`
> > > > even if "$subsha1" == "$sha1", I had to amend the conditional
> > > > controlling that block.  This broke one of the existing tests, which I
> > > > "fixed" in patch 4.  I think a proper fix would involve rewriting
> > > > 
> > > >   (clear_local_git_env; cd "$sm_path" &&
> > > >    ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
> > > >     test -z "$rev") || git-fetch)) ||
> > > >   die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
> > > > 
> > > > but I'm not familiar enough with rev-list to want to dig into that
> > > > yet.  If feedback for the earlier three patches is positive, I'll work
> > > > up a clean fix and resubmit.
> > > 
> > > You probably need to separate your handling here. The comparison of the
> > > currently checked out sha1 and the recorded sha1 is an optimization
> > > which skips unnecessary fetching in case the submodules commits are
> > > already correct. This code snippet checks whether the to be checked out
> > > sha1 is already local and also skips the fetch if it is. We should not
> > > break that.
> > 
> > Agreed.  However, determining if the target $sha1 is local should have
> > nothing to do with the current checked out $subsha1.
> 
> See my draft or the diff below for an illustration of the splitup.
> 
> [snip diff]

This looks fine, but my current --branch implementation (which doesn't
pull) is only a thin branch-checkout layer on top of the standard
`update` functionality.  I'm still unsure if built-in pulls are worth
the configuration trouble.  I'll sleep on it.  Maybe I'll feel better
about them tomorrow ;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option (summary)

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 5531 bytes --]

On Tue, Nov 27, 2012 at 09:42:05PM -0500, W. Trevor King wrote:
> On Wed, Nov 28, 2012 at 12:28:58AM +0100, Heiko Voigt wrote:
> > https://github.com/hvoigt/git/commits/hv/floating_submodules_draft
> 
> I looked over this before, but maybe not thoroughly enough ;).

Heiko pointed out that I likely hadn't looked at this branch at all,
which is true.  I'd confused it with his earlier:

On Fri, Nov 09, 2012 at 05:29:27PM +0100, Heiko Voigt wrote:
> https://github.com/hvoigt/git/commits/hv/floating_submodules

but the new floating_submodules_draft branch adds Heiko's function
reorganization and floating submodule pull (with 'update --branch') on
top of my v4 commits (instead of my branch-checkout with 'update
--branch').

> This looks fine, but my current --branch implementation (which doesn't
> pull) is only a thin branch-checkout layer on top of the standard
> `update` functionality.  I'm still unsure if built-in pulls are worth
> the configuration trouble.  I'll sleep on it.  Maybe I'll feel better
> about them tomorrow ;).

I do feel better about them today.  To get a better feel for how
people see this going forward, here is a summary of proposals to date:

v1. Record submodule.<name>.branch with 'submodule add --branch …',
    leave interpretation up to the user.

    Feedback (paraphrased):
      Phil Hord: that clobbers a configure option used by Gerritt and
        possibly others, rename to --record-branch.  Maybe we should
        export submodule.<name>.* as $submodule_* in 'foreach'?
      Nahor: what about corner cases (e.g. upstream renames branches)?
      Jens Lehmann: if you record it, people will expect Git to use
        it.  What about automatic pulls & commits?

v2. Add --record instead of using --branch to address Phil's v1
    comment.

    Feedback (paraphrased):
      Jens Lehmann: this is still a tiny optimization over using 'git
        config'.  People still use this option in different ways.
      Shawn Pearce: the Gerrit use is basically the same as Ævar's,
        but Gerrit has special handling for '.'.
      Jeff King: if we set up submodule.<name>.branch, we should
        either use it in Git, or make it very clear that we do not use
        it.  If we use Phil's $submodule_* export, we should clear the
        variables for recursive submodules.

v3. Renamed Added $submodule_* export to v2.  Give an example of
    Ævar's pull workflow when explaining that Git does not use the
    value internally.

    Feedback (paraphrased):
      Heiko Voigt: what about automatic pulls & commits?  You should
        allow .git/config overrides of the .gitmodules settings.  if
        we set up submodule.<name>.branch, we should use it in Git.
        How does the submodule know which remote branch to track?
        Junio's proposed 'diff' changes may hide $sha1 information.
      Junio C Hamano: if we set up submodule.<name>.branch, we should
        use it in Git.  Use something more specific than --record.
        Update 'git diff [$path]' and friends in the superproject
        compares the HEAD of the checkout of the submodule at $path
        with the tip of the branch named by submodule.$name.branch in
        .gitmodules of the superproject, instead of the commit that is
        recorded in the index of the superproject.
      Sascha Cunz: you can use 'git shortlog $OLD_SHA1..$NEW_SHA1' for
        the automatic commit message.
      Trevor King: actually, Ævar's update command only specifies the
        local branch name.  The remote is recorded for that branch in
        .git/modules/<name>/config.

v4. Rename --record to --local-branch.  Remove $submodule_* export.
    Add .git/config overrides for submodule.<name>.branch.  Add
    'submodule update --branch' to check out $sha1 as
    submodule.<name>.branch.

    Feedback (paraphrased):
      Heiko Voigt: who cares about the local branch name?  You should
        be pulling origin/$branch, but still using
        .git/modules/<name>/config to record the tracked branch.  You
        should also use 'submodule add --branch[=branch]' instead of
        '--local-branch'.  You should pull the 'update --branch'
        functionality out into a sub-function like
        'handle_tracked_branch_update'.
      Jens Lehmann: .git/config overrides are nice, but they should be
        optional.
      Trevor King: floating branches are following the submodule's
        remote, while 'update --rebase/--merge' are following the
        superproject's recorded $sha1.  Bundling remote-following and
        superproject-following in the same command may be confusing.

Here's what I think we need in v5:

* Jens' optional .git/config overrides.
* Return --local-branch handling to a side effect of --branch (as in
  v1).
* A new 'submodule pull' for tracking the submodule's remote, which is
  pulling --ff-only origin/$branch into a whatever state the submodule
  is currently in.  If any changes were made to submodule $shas,
  optionally commit them with a shortlog-generated commit message.
* Remove my current 'submodule update --branch' functionality.

Thoughs?  Do we need any more information for an automatic pull, or is

  $ git pull --ff-only origin/$branch

on a headless checkout sufficient?

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option (summary)

[W. Trevor King]

On Wed, Nov 28, 2012 at 08:09:03AM -0500, W. Trevor King wrote:
> * A new 'submodule pull' for tracking the submodule's remote, which is
>   pulling --ff-only origin/$branch into a whatever state the submodule
>   is currently in.  If any changes were made to submodule $shas,
>   optionally commit them with a shortlog-generated commit message.

I thought of a better idea on the train.  How about adding `--remote`
to `submodule update` that overrides the gitlinked SHA-1 with the
SHA-1 for origin/$branch?  All of the other checkout/merge/rebase
functionality is untouched.  The only thing that changes is where we
look for the update.  I'm working up the patch now, and will submit v5
shortly.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[PATCH] submodule update: document exisiting -r form for --rebase

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 3 ++-
 git-submodule.sh                | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..ec78fa7 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,7 +13,7 @@ SYNOPSIS
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
-'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
+'git submodule' [--quiet] update [--init] [-N|--no-fetch] [-r|--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
@@ -251,6 +251,7 @@ OPTIONS
 	If the key `submodule.$name.update` is set to `merge`, this option is
 	implicit.
 
+-r::
 --rebase::
 	This option is only valid for the update command.
 	Rebase the current branch onto the commit recorded in the
diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..f2e8026 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,7 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
-   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [-r|--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--] [<path>...]"
-- 
1.8.0.2.gcc766b6

Re: [PATCH] submodule update: document exisiting -r form for --rebase

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> From: "W. Trevor King" <wking@tremily.us>
>
> Signed-off-by: W. Trevor King <wking@tremily.us>
> ---
>  Documentation/git-submodule.txt | 3 ++-
>  git-submodule.sh                | 2 +-
>  2 files changed, 3 insertions(+), 2 deletions(-)

Hmm, I wonder why I have this funny feeling that this was proposed
and rejected already...

As the command takes other options whose names begin with 'r', I
thought the longer term plan was to stop letting "--rebase" squat on
short and sweet "-r" and leaving it undocumented (even though the
short one was added by mistake) was meant to be the first step in
that process.

But maybe I am confusing an undocumented single-letter option from
some other subcommand.  Anybody remembers?

> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index b4683bb..ec78fa7 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -13,7 +13,7 @@ SYNOPSIS
>  	      [--reference <repository>] [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
> -'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
> +'git submodule' [--quiet] update [--init] [-N|--no-fetch] [-r|--rebase]
>  	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
>  	      [commit] [--] [<path>...]
> @@ -251,6 +251,7 @@ OPTIONS
>  	If the key `submodule.$name.update` is set to `merge`, this option is
>  	implicit.
>  
> +-r::
>  --rebase::
>  	This option is only valid for the update command.
>  	Rebase the current branch onto the commit recorded in the
> diff --git a/git-submodule.sh b/git-submodule.sh
> index ab6b110..f2e8026 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -8,7 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
>  USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
>     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] init [--] [<path>...]
> -   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> +   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [-r|--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
>     or: $dashless [--quiet] foreach [--recursive] <command>
>     or: $dashless [--quiet] sync [--] [<path>...]"

[PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

On Wed, Nov 28, 2012 at 11:53:34AM -0500, W. Trevor King wrote:
> I thought of a better idea on the train.  How about adding `--remote`
> to `submodule update` that overrides the gitlinked SHA-1 with the
> SHA-1 for origin/$branch?  All of the other checkout/merge/rebase
> functionality is untouched.  The only thing that changes is where we
> look for the update.  I'm working up the patch now, and will submit v5
> shortly.

Here it is.

Changes since v4:

* Remove `update --branch` in favor of the new `update --remote` logic
  as mentioned above.
* Optional config overrides for submodule.<name>.branch (as suggested
  by Jens)
* Save --branch as submodule.<name>.branch instead of requiring
  --local-branch.
* Restructure doc edits.

I'm a lot happier with this two-patch proposal.  The first patch
implements --remote and documents submodule.<name>.branch.  The second
patch just gives you a convenient way to set it.  I haven't heard from
anyone other than Heiko recently about the --branch/--remote-branch
choice, so I returned to the simpler --branch side effect storage from
v1.  I'd be happy to submit v6 with explicit --remote-branch recording
if desired.

W. Trevor King (2):
  submodule update: add --remote for submodule's upstream changes
  submodule add: If --branch is given, record it in .gitmodules

 Documentation/config.txt        |  9 +++++----
 Documentation/git-submodule.txt | 26 +++++++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 30 +++++++++++++++++++++++++++++-
 t/t7400-submodule-basic.sh      |  1 +
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
 6 files changed, 96 insertions(+), 6 deletions(-)

-- 
1.8.0.2.gad10246.dirty

[PATCH v5 1/2] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1).  Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1.  This helps you keep up with
changes in the upstream superproject.

However, it's also useful to stay up to date with changes in the
upstream subproject.  Previous workflows for incorporating such
changes include the ungainly:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates.  When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch.  If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`.  You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).

Previous use of submodule.<name>.branch
=======================================

Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option.  The foreach-pull
example above was described by Ævar in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.

Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same.  The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls.  The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`.  Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.

Implementation details
======================

In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1.  However, I didn't change the logic guarding the existing fetch:

  if test -z "$nofetch"
  then
    # Run fetch only if $sha1 isn't present or it
    # is not reachable from a ref.
    (clear_local_git_env; cd "$path" &&
      ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
       test -z "$rev") || git-fetch)) ||
    die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
  fi

There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository.  If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/config.txt        |  9 +++++----
 Documentation/git-submodule.txt | 24 +++++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 26 +++++++++++++++++++++++++-
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
 5 files changed, 89 insertions(+), 6 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 11f320b..de39b1c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1994,10 +1994,11 @@ status.submodulesummary::
 submodule.<name>.path::
 submodule.<name>.url::
 submodule.<name>.update::
-	The path within this project, URL, and the updating strategy
-	for a submodule.  These variables are initially populated
-	by 'git submodule init'; edit them to override the
-	URL and other values found in the `.gitmodules` file.  See
+submodule.<name>.branch::
+	The path within this project, URL, the updating strategy, and the
+	remote branch name for a submodule.  These variables are initially
+	populated by 'git submodule init'; edit them to override the URL and
+	other values found in the `.gitmodules` file.  See
 	linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
 
 submodule.<name>.fetchRecurseSubmodules::
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..39aa02d 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,7 +13,7 @@ SYNOPSIS
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
-'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
+'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
@@ -236,6 +236,28 @@ OPTIONS
 	(the default). This limit only applies to modified submodules. The
 	size is always limited to 1 for added/deleted/typechanged submodules.
 
+--remote::
+	This option is only valid for the update command.
+	Instead of using the superproject's recorded SHA-1 to update the
+	submodule, use the status of the submodule's remote tracking branch.
+	The remote tracking branch defaults to origin/master, but the branch
+	name may be overriden by setting the `submodule.<name>.branch`
+	option in either `.gitmodules` or `.git/config` (with `.git/config`
+	taking precedence).
++
+This works for any of the supported update procedures (`--checkout`,
+`--rebase`, etc.).  The only change is the source of the target SHA-1.
+For example, `submodule update --remote --merge` will merge upstream
+submodule changes into the submodules, while `submodule update
+--merge` will merge superproject gitlink changes into the submodules.
++
+In order to ensure a current tracking branch state, `update --remote`
+fetches the submodule's remote repository before calculating the
+SHA-1.  This makes `submodule update --remote --merge` similar to
+running `git pull` in the submodule.  If you don't want to fetch (for
+something closer to `git merge`), you should use `submodule update
+--remote --no-fetch --merge`.
+
 -N::
 --no-fetch::
 	This option is only valid for the update command.
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index 4effd78..4004fa6 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -47,6 +47,11 @@ submodule.<name>.update::
 	This config option is overridden if 'git submodule update' is given
 	the '--merge', '--rebase' or '--checkout' options.
 
+submodule.<name>.branch::
+	A remote branch name for tracking updates in the upstream submodule.
+	If the option is not specified, it defaults to 'master'.  See the
+	`--remote` documentation in linkgit:git-submodule[1] for details.
+
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
 	submodule. If this option is also present in the submodules entry in
diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..b63d869 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
-   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+ges
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--] [<path>...]"
@@ -26,6 +27,7 @@ cached=
 recursive=
 init=
 files=
+remote=
 nofetch=
 update=
 prefix=
@@ -509,6 +511,9 @@ cmd_update()
 		-i|--init)
 			init=1
 			;;
+		--remote)
+			remote=1
+			;;
 		-N|--no-fetch)
 			nofetch=1
 			;;
@@ -569,6 +574,12 @@ cmd_update()
 		fi
 		name=$(module_name "$sm_path") || exit
 		url=$(git config submodule."$name".url)
+		branch=$(git config submodule."$name".branch)
+		if test -z "$branch"
+		then  # fall back on .gitmodules
+			branch=$(git config -f .gitmodules submodule."$name".branch)
+		fi
+		branch="${branch:-master}"
 		if ! test -z "$update"
 		then
 			update_module=$update
@@ -603,6 +614,19 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
 		fi
 
+		if test -n "$remote"
+		then
+			if test -z "$nofetch"
+			then
+				# Fetch remote before determining tracking $sha1
+				(clear_local_git_env; cd "$sm_path" && git-fetch) ||
+				die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
+			fi
+			sha1=$(clear_local_git_env; cd "$sm_path" &&
+				git rev-parse --verify origin/"$branch") ||
+			die "$(eval_gettext "Unable to find current origin/$branch revision in submodule path '\$sm_path'")"
+		fi
+
 		if test "$subsha1" != "$sha1" -o -n "$force"
 		then
 			subforce=$force
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 1542653..a567834 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' '
 	)
 '
 
+test_expect_success 'submodule update --remote should fetch upstream changes' '
+	(cd submodule &&
+	 echo line4 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line4"
+	) &&
+	(cd super &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)"
+	)
+'
+
+test_expect_success 'local config should override .gitmodules branch' '
+	(cd submodule &&
+	 git checkout -b test-branch &&
+	 echo line5 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line5" &&
+	 git checkout master
+	) &&
+	(cd super &&
+	 git config submodule.submodule.branch test-branch &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)"
+	)
+'
+
 test_expect_success 'submodule update --rebase staying on master' '
 	(cd super/submodule &&
 	  git checkout master
-- 
1.8.0.2.gad10246.dirty

[PATCH v5 2/2] submodule add: If --branch is given, record it in .gitmodules

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This allows you to easily record a submodule.<name>.branch option in
.gitmodules when you add a new submodule.  With this patch,

  $ git submodule add -b <branch> <repository> [<path>]
  $ git config -f .gitmodules submodule.<path>.branch <branch>

reduces to

  $ git submodule add -b <branch> <repository> [<path>]

This means that future calls to

  $ git submodule update --remote ...

will get updates from the same branch that you used to initialize the
submodule, which is usually what you want.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 2 ++
 git-submodule.sh                | 4 ++++
 t/t7400-submodule-basic.sh      | 1 +
 3 files changed, 7 insertions(+)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 39aa02d..43e5b4b 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -208,6 +208,8 @@ OPTIONS
 -b::
 --branch::
 	Branch of repository to add as submodule.
+	The name of the branch is recorded as `submodule.<path>.branch` in
+	`.gitmodules` for `update --remote`.
 
 -f::
 --force::
diff --git a/git-submodule.sh b/git-submodule.sh
index b63d869..1f76893 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -368,6 +368,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
+	if test -n "$branch"
+	then
+		git config -f .gitmodules submodule."$sm_path".branch "$branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5397037..90e2915 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test "initial" = "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
 		git submodule init
 	) &&
 
-- 
1.8.0.2.gad10246.dirty

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[W. Trevor King]

On Wed, Nov 28, 2012 at 11:02:45AM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > From: "W. Trevor King" <wking@tremily.us>
> >
> > Signed-off-by: W. Trevor King <wking@tremily.us>
> > ---
> >  Documentation/git-submodule.txt | 3 ++-
> >  git-submodule.sh                | 2 +-
> >  2 files changed, 3 insertions(+), 2 deletions(-)
> 
> Hmm, I wonder why I have this funny feeling that this was proposed
> and rejected already...
> 
> As the command takes other options whose names begin with 'r', I
> thought the longer term plan was to stop letting "--rebase" squat on
> short and sweet "-r" and leaving it undocumented (even though the
> short one was added by mistake) was meant to be the first step in
> that process.
> 
> But maybe I am confusing an undocumented single-letter option from
> some other subcommand.  Anybody remembers?

Perhaps you are remembering:

On Sun, Nov 11, 2012 at 02:33:45AM -0800, Junio C Hamano wrote:
> Ah, this reminds me of another thing I noticed when I saw that
> patch.  The change seems to think "branch" is the _only_ thing the
> user might want to record per submodule upon "git submodule add".
> As an interface to muck with an uninterpreted random configuration,
> it squats on a good option name for setting one single and arbitrary
> variable---quite a selfish change that is not acceptable.
> 
> Calling the option "--record-branch-for-submodule" or something more
> specific might alleviate the problem, but then it would become even
> less useful as a short-hand for "config submodule.$name.branch", I
> would suspect.

With this recent patch, I'm just documenting someone else's squatting
;).  But yes, the reason I noticed was because I was tempted to make
the same mistake again :p.  In my defense, I think `update --remote`
is a good deal more general than my earlier `add --record`.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [PATCH v3 1/3] git-submodule add: Add -r/--record option

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

>> As the command takes other options whose names begin with 'r', I
>> thought the longer term plan was to stop letting "--rebase" squat on
>> short and sweet "-r" and leaving it undocumented (even though the
>> short one was added by mistake) was meant to be the first step in
>> that process.
>> 
>> But maybe I am confusing an undocumented single-letter option from
>> some other subcommand.  Anybody remembers?
>
> Perhaps you are remembering:
>
> On Sun, Nov 11, 2012 at 02:33:45AM -0800, Junio C Hamano wrote:
>> Ah, this reminds me of another thing I noticed when I saw that
>> ...

No.  The discussion might or might not be the "-r" option to
"submodule update", but even if it were so, I wasn't refering to
that exchange but something more in the further past.

[RFC] git-submodule update: Add --commit option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 6638 bytes --]

This option triggers automatic commits when `submodule update` changes
any gitlinked submodule SHA-1s.  The commit message contains a
`shortlog` summary of the changes for each changed submodule.
---

On Tue, Nov 27, 2012 at 07:51:42PM +0100, Heiko Voigt wrote:
> BTW, I am more and more convinced that an automatically manufactured
> commit on update with --branch should be the default. What do other
> think? Sascha raised a concern that he would not want this, but as far as
> I understood he let the CI-server do that so I see no downside to
> natively adding that to git. People who want to manually craft those
> commits can still amend the generated commit. Since this is all about
> helping people keeping their submodules updated why not go the full way?

Here's a first pass (without documentation) for automatic commits on
submodule updates.  There have been a number of requests for
automatically-committed submodule updates due to submodule upstreams.
This patch shows how you can do that (if applied with my `submodule
update --remote` series), and reuse the same logic to automatically
commit changes due to local submodule changes (as shown here in the
new test).

I think the logic is pretty good, but the implementation is pretty
ugly due to POSIX shell variable limitations.  I'm basically trying to
pass an array of [(name, sm_path, sha1, subsha1), ...] into
commit_changes().  I though about perling-out in commit_changes(), but
I lack sufficient perl-fu to know how to tie clear_local_git_env, cd,
and shortlog up in a single open2 call.  If anyone can give me some
implementation pointers, that would be very helpful.

This is against v1.8.0 (without my --remote series).  To apply on top
of the --remote series, you'd have to save the original gitlinked
$sha1 and use that original value when constructing changed_modules.
I can attach this to the end of the --remote series if desired, but I
think this patch could also stand on its own.

Obviously this still needs documentation, etc., but I wanted feedback
on the implementation before I started digging into that.

Cheers,
Trevor

---
 git-submodule.sh            | 67 ++++++++++++++++++++++++++++++++++++++++++++-
 t/t7406-submodule-update.sh | 19 +++++++++++++
 2 files changed, 85 insertions(+), 1 deletion(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..d9a59af 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,7 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
-   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--commit] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--] [<path>...]"
@@ -21,6 +21,7 @@ require_work_tree
 command=
 branch=
 force=
+commit=
 reference=
 cached=
 recursive=
@@ -240,6 +241,52 @@ module_clone()
 }
 
 #
+# Commit changed submodule gitlinks
+#
+# $1 = name-a;sha1-a;subsha1-a\n[name-b;sha1-b;subsha1-b\n...]
+#
+commit_changes()
+{
+	echo "commiting $1"
+	OIFS="$IFS"
+	IFS=";"
+	paths=$(echo "$1" |
+		while read name sm_path sha1 subsha1
+		do
+			echo "$sm_path"
+		done
+		)
+	names=$(echo "$1" |
+		while read name sm_path sha1 subsha1
+		do
+			printf ' %s' "$name"
+		done
+		)
+	summary="$(eval_gettext "Updated submodules:")$names"
+	body=$(echo "$1" |
+		while read name sm_path sha1 subsha1
+		do
+			if test "$name" = "$sm_path"
+			then
+				printf 'Changes to %s:\n\n' "$name"
+			else
+				printf 'Changes to %s (%s):\n\n' "$name" "$sm_path"
+			fi
+			(
+				clear_local_git_env
+				cd "$sm_path" &&
+				git shortlog "${sha1}..${subsha1}" ||
+				die "$(eval_gettext "Unable to generate shortlog in submodule path '\$sm_path'")"
+			)
+		done
+		)
+	IFS="$OIFS"
+	message="$(printf '%s\n\n%s\n' "$summary" "$body")"
+	echo "message: [$message]"
+	git commit -m "$message" $paths
+}
+
+#
 # Add a new submodule to the working tree, .gitmodules and the index
 #
 # $@ = repo path
@@ -515,6 +562,9 @@ cmd_update()
 		-f|--force)
 			force=$1
 			;;
+		--commit)
+			commit=1
+			;;
 		-r|--rebase)
 			update="rebase"
 			;;
@@ -557,6 +607,7 @@ cmd_update()
 	fi
 
 	cloned_modules=
+	changed_modules=
 	module_list "$@" | {
 	err=
 	while read mode sha1 stage sm_path
@@ -660,6 +711,15 @@ Maybe you want to use 'update --init'?")"
 				err="${err};$die_msg"
 				continue
 			fi
+
+			subsha1=$(clear_local_git_env; cd "$sm_path" &&
+				git rev-parse --verify HEAD) ||
+			die "$(eval_gettext "Unable to find new revision in submodule path '\$sm_path'")"
+
+			if test "$subsha1" != "$sha1"
+			then
+				changed_modules=$(printf '%s%s\n' "$changed_modules" "$name;$sm_path;$sha1;$subsha1")
+			fi
 		fi
 
 		if test -n "$recursive"
@@ -680,6 +740,11 @@ Maybe you want to use 'update --init'?")"
 		fi
 	done
 
+	if test -z "$err" -a -n "$commit" -a -n "$changed_modules"
+	then
+		commit_changes "$changed_modules"
+	fi
+
 	if test -n "$err"
 	then
 		OIFS=$IFS
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 1542653..4c8bb5d 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -163,6 +163,25 @@ test_expect_success 'submodule update --merge staying on master' '
 	)
 '
 
+test_expect_success 'submodule update --commit --rebase should commit gitlink changes' '
+	(cd super/submodule &&
+	 git reset --hard HEAD~1 &&
+	 echo "local change" > local-file &&
+	 git add local-file &&
+	 test_tick &&
+	 git commit -m "local change"
+	) &&
+	(cd super &&
+	 git submodule update --commit --rebase submodule &&
+	 test "$(git log -1 --oneline)" = "bbdbe2d Updated submodules: submodule"
+	) &&
+	(cd submodule &&
+	 git remote add super-submodule ../super/submodule &&
+	 git pull super-submodule master
+	) &&
+  test "a" = "b"
+'
+
 test_expect_success 'submodule update - rebase in .git/config' '
 	(cd super &&
 	 git config submodule.submodule.update rebase
-- 
1.8.0.1.gaaf2ac7.dirty

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] git-submodule update: Add --commit option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 443 bytes --]

On Thu, Nov 29, 2012 at 11:12:16AM -0500, W. Trevor King wrote:
> +  test "a" = "b"

This kills the test (with --immediate) so you can look at the
generated commit.  If you actually want the test to pass (e.g. if this
becomes a PATCH and not an RFC), this line should be removed.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] git-submodule update: Add --commit option

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 571 bytes --]

On Thu, Nov 29, 2012 at 11:12:16AM -0500, W. Trevor King wrote:
> +	 test "$(git log -1 --oneline)" = "bbdbe2d Updated submodules: submodule"

s/bbdbe2d/cd69713/

I forgot to update the SHA-1 here after tweaking the commit message
format.  I'd like to rewrite this test so it won't use the SHA-1, but
this was the quickest way to check that the commit message and gitlink
were both changed appropriately.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Re: [PATCH v4 0/4] git-submodule add: Add --local-branch option

[Phil Hord]

On Tue, Nov 27, 2012 at 6:28 PM, Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> Hi,
>
> On Tue, Nov 27, 2012 at 02:01:05PM -0500, W. Trevor King wrote:
> > On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
> > The v4 series leaves the remote branch amigious, but it helps you
> > point the local branch at the right hash so that future calls to
> >
> >   $ git submodule foreach 'git pull'
> >
> > can use the branch's .git/modules/<name>/config settings.
>
> But IMO thats the functionality which should be implemented in submodule
> update and not left to the user.
>
> > > I would think more of some convention like:
> > >
> > >     $ git checkout -t origin/$branch
> > >
> > > when first initialising the submodule with e.g.
> > >
> > >     $ git submodule update --init --branch
> > >
> > > Then later calls of
> > >
> > >     $ git submodule update --branch
> > >
> > > would have a branch configured to pull from. I imagine that results in
> > > a similar behavior gerrit is doing on the server side?
> >
> > That sounds like it's doing pretty much the same thing.  Can you think
> > of a test that would distinguish it from my current v4 implementation?
>
> Well the main difference is that gerrit is automatically updating the
> superproject AFAIK. I would like it if we could implement the same
> workflow support in the submodule script. It seems to me that this is
> already proven to be useful workflow.


It is proven in Gerrit, but Gerrit implements a central-server
workflow.  That is, only Gerrit ever floats the submodules, and he
pushes the result for everyone else to share.  I fear the consequences
of everyone pulling submodules and then later trying to merge
superprojects with someone else's breadcrumbs.

Do you have some idea how this would be handled?

Phil

ps. Apologies for my lateness on this topic. I'm trying to catch up now.

pps. Re-sent since Gmail has hidden the "plain text" option in a
different place, now.

On Tue, Nov 27, 2012 at 9:42 PM, W. Trevor King <wking@tremily.us> wrote:
> On Wed, Nov 28, 2012 at 12:28:58AM +0100, Heiko Voigt wrote:
>> On Tue, Nov 27, 2012 at 02:01:05PM -0500, W. Trevor King wrote:
>> > On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
>> > The v4 series leaves the remote branch amigious, but it helps you
>> > point the local branch at the right hash so that future calls to
>> >
>> >   $ git submodule foreach 'git pull'
>> >
>> > can use the branch's .git/modules/<name>/config settings.
>>
>> But IMO thats the functionality which should be implemented in submodule
>> update and not left to the user.
>
> Then you might need submodule.<name>.local-branch,
> submodule.<name>.remote-repository, and submodule.<name>.remote-branch
> to configure
>
>   $ git checkout submodule.<name>.local-branch
>   $ git pull submodule.<name>.remote-repository submodule.<name>.remote-branch
>
> and this would ignore the $sha1 stored in the gitlink (which all of
> the other update commands use).  This ignoring-the-$sha1 bit made me
> think that a built-in pull wasn't a good fit for 'submodule update'.
> Maybe if it went into a new 'submodule pull'?  Then users have a clear
> distinction:
>
> * 'update' to push superproject $sha1 changes into the submodules
> * 'pull' to push upstream-branch changes into the submodules
>
>> > > I would think more of some convention like:
>> > >
>> > >   $ git checkout -t origin/$branch
>> > >
>> > > when first initialising the submodule with e.g.
>> > >
>> > >   $ git submodule update --init --branch
>> > >
>> > > Then later calls of
>> > >
>> > >   $ git submodule update --branch
>> > >
>> > > would have a branch configured to pull from. I imagine that results in
>> > > a similar behavior gerrit is doing on the server side?
>> >
>> > That sounds like it's doing pretty much the same thing.  Can you think
>> > of a test that would distinguish it from my current v4 implementation?
>>
>> Well the main difference is that gerrit is automatically updating the
>> superproject AFAIK. I would like it if we could implement the same
>> workflow support in the submodule script. It seems to me that this is
>> already proven to be useful workflow.
>
> Ah, sorry, I meant the configuring which remote branch you were
> pulling from happens at submodule initialization (via .git/modules/…)
> for both your workflow and my v4.
>
> You're right that having a builtin pull is different from my v4.
>
>> https://github.com/hvoigt/git/commits/hv/floating_submodules_draft
>
> I looked over this before, but maybe not thoroughly enough ;).
>
>> > > How about reusing the -b|--branch option for add? Since we only change
>> > > the behavior when submodule.$name.update is set to branch it seems
>> > > reasonable to me. Opinions?
>> >
>> > That was the approach I used in v1, but people were concerned that we
>> > would be stomping on previously unclaimed config space.  Since noone
>> > has pointed out other uses besides Gerrit's very similar case, I'm not
>> > sure if that is still an issue.
>>
>> Could you point me to that mail? I cannot seem to find it in my archive.
>
> Hmm.  It seems like Phil's initial response was (accidentally?) off
> list.  The relevant portion was:
>
> On Mon, Oct 22, 2012 at 06:03:53PM -0400, Phil Hord wrote:
>> Some projects now use the 'branch' config value to record the tracking
>> branch for the submodule.  Some ascribe different meaning to the
>> configuration if the value is given vs. undefined.  For example, see
>> the Gerrit submodule-subscription mechanism.  This change will cause
>> those workflows to behave differently than they do now.
>>
>> I do like the idea, but I wish it had a different name for the
>> recording.  Maybe --record-branch=${BRANCH} as an extra switch so the
>> action is explicitly requested.
>
> As I said, I'm happy to go back to --branch if opinions have changed.
>
> On Wed, Nov 28, 2012 at 12:28:58AM +0100, Heiko Voigt wrote:
>> On Tue, Nov 27, 2012 at 02:01:05PM -0500, W. Trevor King wrote:
>> > On Tue, Nov 27, 2012 at 07:31:25PM +0100, Heiko Voigt wrote:
>> > > > Because you need to recurse through submodules for `update --branch`
>> > > > even if "$subsha1" == "$sha1", I had to amend the conditional
>> > > > controlling that block.  This broke one of the existing tests, which I
>> > > > "fixed" in patch 4.  I think a proper fix would involve rewriting
>> > > >
>> > > >   (clear_local_git_env; cd "$sm_path" &&
>> > > >    ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
>> > > >     test -z "$rev") || git-fetch)) ||
>> > > >   die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
>> > > >
>> > > > but I'm not familiar enough with rev-list to want to dig into that
>> > > > yet.  If feedback for the earlier three patches is positive, I'll work
>> > > > up a clean fix and resubmit.
>> > >
>> > > You probably need to separate your handling here. The comparison of the
>> > > currently checked out sha1 and the recorded sha1 is an optimization
>> > > which skips unnecessary fetching in case the submodules commits are
>> > > already correct. This code snippet checks whether the to be checked out
>> > > sha1 is already local and also skips the fetch if it is. We should not
>> > > break that.
>> >
>> > Agreed.  However, determining if the target $sha1 is local should have
>> > nothing to do with the current checked out $subsha1.
>>
>> See my draft or the diff below for an illustration of the splitup.
>>
>> [snip diff]
>
> This looks fine, but my current --branch implementation (which doesn't
> pull) is only a thin branch-checkout layer on top of the standard
> `update` functionality.  I'm still unsure if built-in pulls are worth
> the configuration trouble.  I'll sleep on it.  Maybe I'll feel better
> about them tomorrow ;).
>
> Cheers,
> Trevor
>
> --
> This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
> For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1990 bytes --]


On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote:
> On Fri, Nov 23, 2012 at 12:54 PM, W. Trevor King <wking@tremily.us> wrote:
> > [snip initial thoughts leading to the update --remote v5]
>
> I was thinking the same thing, but reading this whole thread a couple of
> weeks late.  Thanks for noticing.
> 
> Moreover, I think that 'git submodule update --pull' is also the wrong way
> to spell this action.   Maybe you are misled from the outset by your
> current workflow:

Did you see my v5 (add --remote) series?

> For that reason, I don't like the --pull switch since it implies a
> fetch, but I will not always want to do a fetch.

  $ git submodule update --remote --no-fetch

will not fetch the submodule remotes.

> I don't know which remote I should be tracking, though.  I suppose
> it is 'origin' for now, but maybe it is just whatever
> $superproject's HEAD's remote-tracking branch indicates.

With the --remote series, I always use "origin" because that's what
`submodule add` should be setting up.  If people want to change that
up by hand, we may need a submodule.<name>.remote configuration
option.

> I am not sure I want the gitlinks in superproject to update automatically
> in the index, but I definitely do not want to automatically create a commit
> for them.

Commits are dissabled by default (see my recent --commit RFC for how
they would be enabled).

> But I really don't want to figure out how to handle submodule
> collisions during a merge (or rebase!) of my superproject with changes that
> someone else auto-committed in his local $superproject as he and I
> arbitrarily floated up the upstream independently.  There is nothing but
> loathing down that path.

This is true.  I'm not sure how gitlink collisions are currently
handled…

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes

[Phil Hord]

On Thu, Nov 29, 2012 at 2:13 PM, W. Trevor King <wking@tremily.us> wrote:
>
> On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote:
>> On Fri, Nov 23, 2012 at 12:54 PM, W. Trevor King <wking@tremily.us> wrote:
>> > [snip initial thoughts leading to the update --remote v5]
>>
>> I was thinking the same thing, but reading this whole thread a couple of
>> weeks late.  Thanks for noticing.
>>
>> Moreover, I think that 'git submodule update --pull' is also the wrong way
>> to spell this action.   Maybe you are misled from the outset by your
>> current workflow:
>
> Did you see my v5 (add --remote) series?

Eventually, I did.  Sorry for the out-of-order replies.


>> For that reason, I don't like the --pull switch since it implies a
>> fetch, but I will not always want to do a fetch.
>
>   $ git submodule update --remote --no-fetch
>
> will not fetch the submodule remotes.

This seems precisely backwards to me. Why not use

  $ git submodule update --remote --fetch

to do your "default" behavior instead?   I suppose I am arguing
against the tide of the dominant workflow, but the fetch-by-default
idea needlessly conflates two primitive operations:  "float" and
"fetch".

>> I don't know which remote I should be tracking, though.  I suppose
>> it is 'origin' for now, but maybe it is just whatever
>> $superproject's HEAD's remote-tracking branch indicates.
>
> With the --remote series, I always use "origin" because that's what
> `submodule add` should be setting up.  If people want to change that
> up by hand, we may need a submodule.<name>.remote configuration
> option.

I've always felt that the "origin" defaults are broken and are simply
being ignored because most users do not trip over them.  But ISTR that
submodule commands use the remote indicated by the superproject's
current remote-tracking configuration, with a fallback to 'origin' if
there is none.  Sort of a "best effort" algorithm, I think.  Am I
remembering that wrong?


>> I am not sure I want the gitlinks in superproject to update automatically
>> in the index, but I definitely do not want to automatically create a commit
>> for them.
>
> Commits are dissabled by default (see my recent --commit RFC for how
> they would be enabled).
>
>> But I really don't want to figure out how to handle submodule
>> collisions during a merge (or rebase!) of my superproject with changes that
>> someone else auto-committed in his local $superproject as he and I
>> arbitrarily floated up the upstream independently.  There is nothing but
>> loathing down that path.
>
> This is true.  I'm not sure how gitlink collisions are currently
> handled…


They've always been trouble for me.  But it may be that I am ignorant.

Phil

Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3121 bytes --]

On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote:
> On Thu, Nov 29, 2012 at 2:13 PM, W. Trevor King <wking@tremily.us> wrote:
> > On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote:
> >> For that reason, I don't like the --pull switch since it implies a
> >> fetch, but I will not always want to do a fetch.
> >
> >   $ git submodule update --remote --no-fetch
> >
> > will not fetch the submodule remotes.
> 
> This seems precisely backwards to me. Why not use
> 
>   $ git submodule update --remote --fetch
> 
> to do your "default" behavior instead?   I suppose I am arguing
> against the tide of the dominant workflow, but the fetch-by-default
> idea needlessly conflates two primitive operations:  "float" and
> "fetch".

Because --no-fetch is the existing option, and if it ain't broke… ;).

> >> I don't know which remote I should be tracking, though.  I suppose
> >> it is 'origin' for now, but maybe it is just whatever
> >> $superproject's HEAD's remote-tracking branch indicates.
> >
> > With the --remote series, I always use "origin" because that's what
> > `submodule add` should be setting up.  If people want to change that
> > up by hand, we may need a submodule.<name>.remote configuration
> > option.
> 
> I've always felt that the "origin" defaults are broken and are simply
> being ignored because most users do not trip over them.  But ISTR that
> submodule commands use the remote indicated by the superproject's
> current remote-tracking configuration, with a fallback to 'origin' if
> there is none.  Sort of a "best effort" algorithm, I think.  Am I
> remembering that wrong?

The current code uses a bare "git-fetch".  I'm not sure what that
defaults to if you're on a detached head.  If it bothers you, I'm fine
adding the submodule.<name>.remote option in v6.

> >> I am not sure I want the gitlinks in superproject to update automatically
> >> in the index, but I definitely do not want to automatically create a commit
> >> for them.
> >
> > Commits are dissabled by default (see my recent --commit RFC for how
> > they would be enabled).
> >
> >> But I really don't want to figure out how to handle submodule
> >> collisions during a merge (or rebase!) of my superproject with changes that
> >> someone else auto-committed in his local $superproject as he and I
> >> arbitrarily floated up the upstream independently.  There is nothing but
> >> loathing down that path.
> >
> > This is true.  I'm not sure how gitlink collisions are currently
> > handled…
> 
> They've always been trouble for me.  But it may be that I am ignorant.

I haven't dealt with any gitlink merges, but I think that supporting
easy gitlink merges is orthogonal to this --remote option.  For simple
cases like "autocommitted submodule floats", one of the conflicting
gitlinks will be an ancestor of the other, so it should be easy to
automate that merge.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

On Wed, Nov 28, 2012 at 12:19:04AM +0100, Jens Lehmann wrote:
> Am 26.11.2012 22:00, schrieb W. Trevor King:
> > From: "W. Trevor King" <wking@tremily.us>
> > 
> > This allows users to override the .gitmodules value with a
> > per-repository value.
> 
> Your intentions makes lots of sense, but your patch does more than
> that. Copying the branch setting into .git/config sets the initial
> branch setting into stone. That makes it impossible to have a branch
> "foo" in the superproject using a branch "bar" in a submodule and
> another superproject branch "frotz" using branch "nitfol" for the
> same submodule. You should use the branch setting from .git/config
> if present and fall back to the branch setting from .gitmodules if
> not, which would enable the user to have her own setting if she
> doesn't like what upstream provides but would still enable others
> to follow different submodule branches in different superproject
> branches.

I've mulling this over, and when I started coding support for
submodule.<name>.remote, I had an idea.

On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote:
> On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote:
> > I've always felt that the "origin" defaults are broken and are simply
> > being ignored because most users do not trip over them.  But ISTR that
> > submodule commands use the remote indicated by the superproject's
> > current remote-tracking configuration, with a fallback to 'origin' if
> > there is none.  Sort of a "best effort" algorithm, I think.  Am I
> > remembering that wrong?
> 
> The current code uses a bare "git-fetch".  I'm not sure what that
> defaults to if you're on a detached head.  If it bothers you, I'm fine
> adding the submodule.<name>.remote option in v6.

In my v5 patch, I check for submodule.<name>.remote first in the usual
`git config` files.  If I don't find what I'm looking for I fall back
on .gitmodules (basically Jens' suggestion).  However, my initial
copying-to-.git/config approach was mostly done to mimic existing
configuration handling in git-submodule.sh.  Since I agree with Jens
on configuration precendence, and I now had two options to read
(.branch and .remote), I thought I'd pull the logic out into its own
function (code included at the end).  While I was shifting the
existing submodule config handling over to my new function, I noticed
that with this logic, `submodule init` doesn't really do anything
important anymore.  Likewise for `submodule sync`, which seems to be
quite similar to `init`.

What to do about this?  `init` has been around for a while, so we
can't just remove it (maybe in 2.0?).  Leaving it in place is not
really a problem though, it just means that the user is locking in the
current .gitmodules configuration (as Jens pointed out with respect to
.branch).

I may be way off base here, as I'm fairly new to submodules in general
and these two commands in particular, but I thought I'd float the
idea.

Cheers,
Trevor

---
#
# Print a submodule configuration setting
#
# $1 = submodule name
# $2 = option name
# $3 = default value
#
# Checks in the usual git-config places first (for overrides),
# otherwise it falls back on .gitmodules.  This allows you to
# distribute project-wide defaults in .gitmodules, while still
# customizing individual repositories if necessary.  If the option is
# not in .gitmodules either, print a default value.
#
get_submodule_config()
{
	name="$1"
	option="$2"
	default="$3"
	value=$(git config submodule."$name"."$option")
	if test -z "$value"
	then
		value=$(git config -f .gitmodules submodule."$name"."$option")
	fi
	printf '%s' "${value:-$default}"
}

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

On Fri, Nov 30, 2012 at 12:53:09PM -0500, W. Trevor King wrote:
> Likewise for `submodule sync`, which seems to be
> quite similar to `init`.

Ah, I'd remove the part of `sync` that touches the superproject's
.git/config, but keep the part that stores the superproject-reorded
URL in the submodule's config:

    url=$(get_submodule_config "$name" url)
    up_path=$(get_up_path "$sm_path")
    url=$(resolve_relative_url "$url" "$up_path") &&
    if test -n "$url"
    then
      if test -e "$sm_path"/.git
      then
      (
        clear_local_git_env
        cd "$sm_path"
        remote=$(get_default_remote)
        git config remote."$remote".url "$url"
      )
      fi
    fi

I should probably also tweak sync to do similar things with
submodule.<name>.branch and .remote as part of my `--update remote`
series.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[Phil Hord]

On Fri, Nov 30, 2012 at 12:53 PM, W. Trevor King <wking@tremily.us> wrote:
> On Wed, Nov 28, 2012 at 12:19:04AM +0100, Jens Lehmann wrote:
>> Am 26.11.2012 22:00, schrieb W. Trevor King:
>> > From: "W. Trevor King" <wking@tremily.us>
>> >
>> > This allows users to override the .gitmodules value with a
>> > per-repository value.
>>
>> Your intentions makes lots of sense, but your patch does more than
>> that. Copying the branch setting into .git/config sets the initial
>> branch setting into stone. That makes it impossible to have a branch
>> "foo" in the superproject using a branch "bar" in a submodule and
>> another superproject branch "frotz" using branch "nitfol" for the
>> same submodule. You should use the branch setting from .git/config
>> if present and fall back to the branch setting from .gitmodules if
>> not, which would enable the user to have her own setting if she
>> doesn't like what upstream provides but would still enable others
>> to follow different submodule branches in different superproject
>> branches.
>
> I've mulling this over, and when I started coding support for
> submodule.<name>.remote, I had an idea.
>
> On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote:
>> On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote:
>> > I've always felt that the "origin" defaults are broken and are simply
>> > being ignored because most users do not trip over them.  But ISTR that
>> > submodule commands use the remote indicated by the superproject's
>> > current remote-tracking configuration, with a fallback to 'origin' if
>> > there is none.  Sort of a "best effort" algorithm, I think.  Am I
>> > remembering that wrong?
>>
>> The current code uses a bare "git-fetch".  I'm not sure what that
>> defaults to if you're on a detached head.  If it bothers you, I'm fine
>> adding the submodule.<name>.remote option in v6.
>
> In my v5 patch, I check for submodule.<name>.remote first in the usual
> `git config` files.  If I don't find what I'm looking for I fall back
> on .gitmodules (basically Jens' suggestion).  However, my initial
> copying-to-.git/config approach was mostly done to mimic existing
> configuration handling in git-submodule.sh.  Since I agree with Jens
> on configuration precendence, and I now had two options to read
> (.branch and .remote), I thought I'd pull the logic out into its own
> function (code included at the end).  While I was shifting the
> existing submodule config handling over to my new function, I noticed
> that with this logic, `submodule init` doesn't really do anything
> important anymore.

If I never 'submodule init' a submodule, it does not get visited by
'git submodule foreach', among others.  I think some people use this
behavior explicitly.

On the other hand, I've also notice that a submodule which I have
removed does not get de-inited later one.  It causes my 'git submodule
foreach' to emit errors.  :-(

Phil


> Likewise for `submodule sync`, which seems to be
> quite similar to `init`.
>
> What to do about this?  `init` has been around for a while, so we
> can't just remove it (maybe in 2.0?).  Leaving it in place is not
> really a problem though, it just means that the user is locking in the
> current .gitmodules configuration (as Jens pointed out with respect to
> .branch).
>
> I may be way off base here, as I'm fairly new to submodules in general
> and these two commands in particular, but I thought I'd float the
> idea.
>
> Cheers,
> Trevor
>
> ---
> #
> # Print a submodule configuration setting
> #
> # $1 = submodule name
> # $2 = option name
> # $3 = default value
> #
> # Checks in the usual git-config places first (for overrides),
> # otherwise it falls back on .gitmodules.  This allows you to
> # distribute project-wide defaults in .gitmodules, while still
> # customizing individual repositories if necessary.  If the option is
> # not in .gitmodules either, print a default value.
> #
> get_submodule_config()
> {
>         name="$1"
>         option="$2"
>         default="$3"
>         value=$(git config submodule."$name"."$option")
>         if test -z "$value"
>         then
>                 value=$(git config -f .gitmodules submodule."$name"."$option")
>         fi
>         printf '%s' "${value:-$default}"
> }
>
> --
> This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
> For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Fri, Nov 30, 2012 at 06:52:22PM -0500, Phil Hord wrote:
> If I never 'submodule init' a submodule, it does not get visited by
> 'git submodule foreach', among others.  I think some people use this
> behavior explicitly.

This is something I'll fix while working up a trial patch.  Currently
cmd_update calls module_clone if the <submodule>/.git does not exist.
This should probably happen in each command (in a wrapper around
module_list?).  It's possible that module_list itself would need some
work, but I haven't absorbed its implementation yet [1].

Trevor

[1]: I read Perl by sounding out each letter ;).

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[Jens Lehmann]

Am 30.11.2012 18:53, schrieb W. Trevor King:
> In my v5 patch, I check for submodule.<name>.remote first in the usual
> `git config` files.  If I don't find what I'm looking for I fall back
> on .gitmodules (basically Jens' suggestion).  However, my initial
> copying-to-.git/config approach was mostly done to mimic existing
> configuration handling in git-submodule.sh.  Since I agree with Jens
> on configuration precendence, and I now had two options to read
> (.branch and .remote), I thought I'd pull the logic out into its own
> function (code included at the end).  While I was shifting the
> existing submodule config handling over to my new function, I noticed
> that with this logic, `submodule init` doesn't really do anything
> important anymore.  Likewise for `submodule sync`, which seems to be
> quite similar to `init`.

You need to handle the 'url' setting differently. While I think the
'update' setting should not be copied into .git/config at all
(because it makes it impossible for upstream to change that later
without the user copying that himself as 'sync' doesn't do that) the
'url' setting in .git/config has two important implications:

1) It tells the submodule commands that the user wants to have that
   submodule populated  (which is done in a subsequent "update" after
   "init" copied the url there).

2) It can be used to follow moving upstreams (think of checking out
   an earlier commit before the upstream was moved, you won't be able
   to clone it from there without having the new setting persist).
   And which repository you follow is a matter of trust, so the extra
   "git submodule sync" in that case is a good thing to have.

So I believe 'url' is the only setting that should be copied into
.git/config while all the others shouldn't.

> What to do about this?  `init` has been around for a while, so we
> can't just remove it (maybe in 2.0?).  Leaving it in place is not
> really a problem though, it just means that the user is locking in the
> current .gitmodules configuration (as Jens pointed out with respect to
> .branch).

We still need those commands to set and update the "url" setting.

> ---
> #
> # Print a submodule configuration setting
> #
> # $1 = submodule name
> # $2 = option name
> # $3 = default value
> #
> # Checks in the usual git-config places first (for overrides),
> # otherwise it falls back on .gitmodules.  This allows you to
> # distribute project-wide defaults in .gitmodules, while still
> # customizing individual repositories if necessary.  If the option is
> # not in .gitmodules either, print a default value.
> #
> get_submodule_config()
> {
> 	name="$1"
> 	option="$2"
> 	default="$3"
> 	value=$(git config submodule."$name"."$option")
> 	if test -z "$value"
> 	then
> 		value=$(git config -f .gitmodules submodule."$name"."$option")
> 	fi
> 	printf '%s' "${value:-$default}"
> }

Something like that makes sense. You can use it for the settings you add
first and we can then reuse that for 'update' in a separate patch later.

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[Jens Lehmann]

Am 01.12.2012 13:48, schrieb W. Trevor King:
> On Fri, Nov 30, 2012 at 06:52:22PM -0500, Phil Hord wrote:
>> If I never 'submodule init' a submodule, it does not get visited by 'git submodule foreach', among others.  I think some people use this behavior explicitly.
> 
> This is something I'll fix while working up a trial patch.  Currently cmd_update calls module_clone if the <submodule>/.git does not exist. This should probably happen in each command (in a wrapper around module_list?).  It's possible that module_list itself would need some work, but I haven't absorbed its implementation yet [1].

Please do not fix it, this is a feature. "update" is the only command
where that should happen (and only if "url" is set in .git/config, as
I explained in my other mail). So everything should be fine here.

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[Jens Lehmann]

Am 01.12.2012 00:52, schrieb Phil Hord:
> If I never 'submodule init' a submodule, it does not get visited by
> 'git submodule foreach', among others.  I think some people use this
> behavior explicitly.
> 
> On the other hand, I've also notice that a submodule which I have
> removed does not get de-inited later one.  It causes my 'git submodule
> foreach' to emit errors.  :-(

I'm currently hacking on "git submodule deinit" which removes the 'url'
setting from git/config. This should do the trick for you, right?

Just removing that submodule automagically would not work that well, as
it would deinitialize a submodule when you switch to a branch where it
isn't present and you'd have to reinitialize it when you come back.

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3387 bytes --]

On Sat, Dec 01, 2012 at 04:38:02PM +0100, Jens Lehmann wrote:
> Am 30.11.2012 18:53, schrieb W. Trevor King:
> > In my v5 patch, I check for submodule.<name>.remote first in the usual
> > `git config` files.  If I don't find what I'm looking for I fall back
> > on .gitmodules (basically Jens' suggestion).  However, my initial
> > copying-to-.git/config approach was mostly done to mimic existing
> > configuration handling in git-submodule.sh.  Since I agree with Jens
> > on configuration precendence, and I now had two options to read
> > (.branch and .remote), I thought I'd pull the logic out into its own
> > function (code included at the end).  While I was shifting the
> > existing submodule config handling over to my new function, I noticed
> > that with this logic, `submodule init` doesn't really do anything
> > important anymore.  Likewise for `submodule sync`, which seems to be
> > quite similar to `init`.
> 
> You need to handle the 'url' setting differently. While I think the
> 'update' setting should not be copied into .git/config at all
> (because it makes it impossible for upstream to change that later
> without the user copying that himself as 'sync' doesn't do that) the
> 'url' setting in .git/config has two important implications:
> 
> 1) It tells the submodule commands that the user wants to have that
>    submodule populated  (which is done in a subsequent "update" after
>    "init" copied the url there).

Good point, but this should depend on submodule.<name>.update; having
it as a side effect of a local submodule.<name>.url makes no sense.
Perhaps `submodule init` should be reduced to just wrap:

  $ git config submodule.<name>.update checkout

where the default update configuration would be 'none'.

> 2) It can be used to follow moving upstreams (think of checking out
>    an earlier commit before the upstream was moved, you won't be able
>    to clone it from there without having the new setting persist).
>    And which repository you follow is a matter of trust, so the extra
>    "git submodule sync" in that case is a good thing to have.
> 
> So I believe 'url' is the only setting that should be copied into
> .git/config while all the others shouldn't.

If you want to override the old repository location for an old commit,
setting submodule.<name>.url makes sense.  My rewritten `sync` updates
the local submodule.<name>.url in the superproject if the
configuration option is already set [1].  Perhaps a `sync --local`
invocation should forcibly populate the local submodule.<name>.url to
make this workflow easier.  Bundling sugar for this special case
should not happen under an extra command called `init`.

> > [snip get_submodule_config()]
>
> Something like that makes sense. You can use it for the settings you add
> first and we can then reuse that for 'update' in a separate patch later.

I'm currently working out the details independently against v1.8.0.
This will be a fairly major shift, so I think it should stay
independent of `update --remote`.  The `update --remote` stuff should
be easy to adjust/rebase if the `init` removal/adjustment develops
into something acceptable.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1250 bytes --]

On Sat, Dec 01, 2012 at 04:56:02PM +0100, Jens Lehmann wrote:
> Am 01.12.2012 00:52, schrieb Phil Hord:
> > If I never 'submodule init' a submodule, it does not get visited by
> > 'git submodule foreach', among others.  I think some people use this
> > behavior explicitly.
> > 
> > On the other hand, I've also notice that a submodule which I have
> > removed does not get de-inited later one.  It causes my 'git submodule
> > foreach' to emit errors.  :-(
> 
> I'm currently hacking on "git submodule deinit" which removes the 'url'
> setting from git/config. This should do the trick for you, right?
> 
> Just removing that submodule automagically would not work that well, as
> it would deinitialize a submodule when you switch to a branch where it
> isn't present and you'd have to reinitialize it when you come back.

I think this is another case where we should be looping through
submodules based on the revision-specific .gitmodules content, and
querying the local config only to determine if the user wants to
update them (to drop into them with foreach, etc.).

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH] submodule: add 'deinit' command

[Jens Lehmann]

With "git submodule init" the user is able to tell git he cares about one
or more submodules and wants to have it populated on the next call to "git
submodule update". But currently there is no easy way he could tell git he
does not care about a submodule anymore and wants to get rid of his local
work tree (except he knows a lot about submodule internals and removes the
"submodule.$name.url" setting from .git/config himself).

Help those users by providing a 'deinit' command. This removes the url
setting from .git/config either for the given submodule(s) or for all
those which have been initialized if none were given. Complain only when
for a submodule given on the command line the url setting can't be found
in .git/config.

Add tests and link the man pages of "git submodule deinit" and "git rm" to
assist the user in deciding whether removing or unregistering the submodule
is the right thing to do for him.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---

Am 01.12.2012 16:56, schrieb Jens Lehmann:
> Am 01.12.2012 00:52, schrieb Phil Hord:
>> If I never 'submodule init' a submodule, it does not get visited by
>> 'git submodule foreach', among others.  I think some people use this
>> behavior explicitly.
>>
>> On the other hand, I've also notice that a submodule which I have
>> removed does not get de-inited later one.  It causes my 'git submodule
>> foreach' to emit errors.  :-(
> 
> I'm currently hacking on "git submodule deinit" which removes the 'url'
> setting from git/config. This should do the trick for you, right?

And here we go ...


 Documentation/git-rm.txt        |  4 ++++
 Documentation/git-submodule.txt | 11 +++++++++
 git-submodule.sh                | 50 ++++++++++++++++++++++++++++++++++++++++-
 t/t7400-submodule-basic.sh      | 11 +++++++++
 4 files changed, 75 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt
index 262436b..ec42bf5 100644
--- a/Documentation/git-rm.txt
+++ b/Documentation/git-rm.txt
@@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree.
 Ignored files are deemed expendable and won't stop a submodule's work
 tree from being removed.

+If you only want to remove the local checkout of a submodule from your
+work tree without committing that use `git submodule deinit` instead
+(see linkgit:git-submodule[1]).
+
 EXAMPLES
 --------
 `git rm Documentation/\*.txt`::
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b1de3ba..fba77f6 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,6 +13,7 @@ SYNOPSIS
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
+'git submodule' [--quiet] deinit [--] [<path>...]
 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
@@ -134,6 +135,16 @@ init::
 	the explicit 'init' step if you do not intend to customize
 	any submodule locations.

+deinit::
+	Unregister the submodules, i.e. remove the `submodule.$name.url`
+	setting from .git/config. Further calls to `git submodule update`,
+	`git submodule foreach` and `git submodule sync` will skip any
+	unregistered submodules until they are initialized again, so use
+	this command if you don't want to have a local checkout of the
+	submodule in your work tree anymore. If you really want to remove
+	a submodule from the repository and commit that use
+	linkgit:git-rm[1] instead.
+
 update::
 	Update the registered submodules, i.e. clone missing submodules and
 	checkout the commit specified in the index of the containing repository.
diff --git a/git-submodule.sh b/git-submodule.sh
index 2365149..4059a2e 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
+   or: $dashless [--quiet] deinit [--] [<path>...]
    or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
@@ -516,6 +517,53 @@ cmd_init()
 }

 #
+# Unregister submodules from .git/config
+#
+# $@ = requested paths (default to all)
+#
+cmd_deinit()
+{
+	# parse $args after "submodule ... init".
+	while test $# -ne 0
+	do
+		case "$1" in
+		-q|--quiet)
+			GIT_QUIET=1
+			;;
+		--)
+			shift
+			break
+			;;
+		-*)
+			usage
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+
+	module_list "$@" |
+	while read mode sha1 stage sm_path
+	do
+		die_if_unmatched "$mode"
+		name=$(module_name "$sm_path") || exit
+		url=$(git config submodule."$name".url)
+		if test -z "$url"
+		then
+			# Only mention uninitialized submodules when its
+			# path have been specified
+			test "$#" != "0" &&
+			say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")"
+			continue
+		fi
+		git config --unset submodule."$name".url &&
+		say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")"
+	done
+}
+
+#
 # Update each submodule path to correct revision, using clone and checkout as needed
 #
 # $@ = requested paths (default to all)
@@ -1108,7 +1156,7 @@ cmd_sync()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | foreach | init | update | status | summary | sync)
+	add | foreach | init | deinit | update | status | summary | sync)
 		command=$1
 		;;
 	-q|--quiet)
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index de7d453..803bda7 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -756,4 +756,15 @@ test_expect_success 'submodule add with an existing name fails unless forced' '
 	)
 '

+test_expect_success 'submodule deinit should unregister submodule url from .git/config' '
+	url=$(git config submodule.example.url) &&
+	git submodule deinit &&
+	test -z "$(git config submodule.example.url)"
+'
+
+test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' '
+	git submodule deinit &&
+	test_must_fail git submodule deinit example
+'
+
 test_done
-- 
1.7.11.7

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]

I'm currently stuck with adding a commit-less existing repository as a
submodule (which happens in t7400-submodule-basic.sh, ../bar/a/b/c
works with relative local path):

  $ mkdir -p super/sub
  $ cd super
  $ git init
  $ (cd sub && git init)
  $ git submodule add ./ sub
  $ git status
  # On branch master
  #
  # Initial commit
  #
  # Changes to be committed:
  #   (use "git rm --cached <file>..." to unstage)
  #
  #       new file:   .gitmodules
  #

What I'm missing is a gitlink form sub for 'Subproject commit
00000...' or some such.  When the subproject has an actual commit,
things work as expected:

  $ mkdir -p super/sub
  $ cd super
  $ git init
  $ (cd sub && git init && echo line-1 > file && git add file && git commit -m file)
  $ git submodule add ./ sub
  $ git status
  # On branch master
  #
  # Initial commit
  #
  # Changes to be committed:
  #   (use "git rm --cached <file>..." to unstage)
  #
  #       new file:   .gitmodules
  #       new file:   sub
  #

This means that module_list isn't aware of the empty submodule, when
the user has just explicitly added it.  Fixing this would seem to need
either 'Subproject commit 00000...' as I suggested earlier, or an
adjustment to module_list that also spits out submodules that are in
.gitmodules but not in the index.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[Jens Lehmann]

Am 01.12.2012 17:30, schrieb W. Trevor King:
> On Sat, Dec 01, 2012 at 04:38:02PM +0100, Jens Lehmann wrote:
>> You need to handle the 'url' setting differently. While I think the 'update' setting should not be copied into .git/config at all (because it makes it impossible for upstream to change that later without the user copying that himself as 'sync' doesn't do that) the 'url' setting in .git/config has two important implications:
>> 
>> 1) It tells the submodule commands that the user wants to have that submodule populated  (which is done in a subsequent "update" after "init" copied the url there).
> 
> Good point, but this should depend on submodule.<name>.update; having it as a side effect of a local submodule.<name>.url makes no sense.

Sorry, but that makes tons of sense: url controls if the submodule
is to be populated and from where, update controls how (and can even
veto populating it if set to "none"). We /could/ do it differently,
but I can't see why we should (and risk severe compatibility issues).

> Perhaps `submodule init` should be reduced to just wrap:
> 
> $ git config submodule.<name>.update checkout
> 
> where the default update configuration would be 'none'.
> 
>> 2) It can be used to follow moving upstreams (think of checking out an earlier commit before the upstream was moved, you won't be able to clone it from there without having the new setting persist). And which repository you follow is a matter of trust, so the extra "git submodule sync" in that case is a good thing to have.
>> 
>> So I believe 'url' is the only setting that should be copied into .git/config while all the others shouldn't.
> 
> If you want to override the old repository location for an old commit, setting submodule.<name>.url makes sense.  My rewritten `sync` updates the local submodule.<name>.url in the superproject if the configuration option is already set [1].  Perhaps a `sync --local` invocation should forcibly populate the local submodule.<name>.url to make this workflow easier.  Bundling sugar for this special case should not happen under an extra command called `init`.

What real world problems do we have with the current init/sync that
this approach would solve?

>>> [snip get_submodule_config()]
>> 
>> Something like that makes sense. You can use it for the settings you add first and we can then reuse that for 'update' in a separate patch later.
> 
> I'm currently working out the details independently against v1.8.0. This will be a fairly major shift, so I think it should stay independent of `update --remote`.  The `update --remote` stuff should be easy to adjust/rebase if the `init` removal/adjustment develops into something acceptable.

I totally agree. Let's get the `update --remote` stuff ready first.

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3824 bytes --]

On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote:
> Am 01.12.2012 17:30, schrieb W. Trevor King:
> > On Sat, Dec 01, 2012 at 04:38:02PM +0100, Jens Lehmann wrote:
> > > 1) It tells the submodule commands that the user wants to have
> > > that submodule populated (which is done in a subsequent "update"
> > > after "init" copied the url there).
> > 
> > Good point, but this should depend on submodule.<name>.update;
> > having it as a side effect of a local submodule.<name>.url makes
> > no sense.
> 
> Sorry, but that makes tons of sense: url controls if the submodule
> is to be populated and from where, update controls how (and can even
> veto populating it if set to "none"). We /could/ do it differently,
> but I can't see why we should (and risk severe compatibility issues).

I think removing `init` will cause some compatibility issues anyway,
so I was re-imaging how you do it.  I don't think update='none' and
"don't populate my submodule" are distinct ideas, while a locally
configured url="somwhere" and "please populate my submodule" are (with
the blank-url case defaulting to the superproject itself).

> > > 2) It can be used to follow moving upstreams (think of checking
> > > out an earlier commit before the upstream was moved, you won't
> > > be able to clone it from there without having the new setting
> > > persist). And which repository you follow is a matter of trust,
> > > so the extra "git submodule sync" in that case is a good thing
> > > to have.
> > > 
> > > So I believe 'url' is the only setting that should be copied
> > > into .git/config while all the others shouldn't.
> > 
> > If you want to override the old repository location for an old
> > commit, setting submodule.<name>.url makes sense.  My rewritten
> > `sync` updates the local submodule.<name>.url in the superproject
> > if the configuration option is already set [1].  Perhaps a `sync
> > --local` invocation should forcibly populate the local
> > submodule.<name>.url to make this workflow easier.  Bundling sugar
> > for this special case should not happen under an extra command
> > called `init`.
> 
> What real world problems do we have with the current init/sync that
> this approach would solve?

I don't have any, but in my `update --remote` series I'm adding two
new config options that are handled differently (define in
.gitmodules, override in superproject .git/config) than existing
submodules options.  I'm trying to avoid confusing users by
standardizing on the more flexible method, which avoids copying stuff
into the superproject's .git/config, and under which the current
`init` functionality doesn't make much sense.

> > > > [snip get_submodule_config()]
> > >
> > > Something like that makes sense. You can use it for the settings
> > > you add first and we can then reuse that for 'update' in a
> > > separate patch later.
> > 
> > I'm currently working out the details independently against
> > v1.8.0. This will be a fairly major shift, so I think it should
> > stay independent of `update --remote`.  The `update --remote`
> > stuff should be easy to adjust/rebase if the `init`
> > removal/adjustment develops into something acceptable.
> 
> I totally agree. Let's get the `update --remote` stuff ready first.

Ok, but we'll have the possible confusion about option setting that I
mention above.  Still, it's good to minimize the number of irons in
the fire, and an `init` removal will probably not get in until 2.0
anyway.  If other people are fine with the different initialization
paths, I'll put the init-removal on hold for now.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[Jens Lehmann]

Am 01.12.2012 18:49, schrieb W. Trevor King:
> I think removing `init` will cause some compatibility issues anyway,
> so I was re-imaging how you do it.  I don't think update='none' and
> "don't populate my submodule" are distinct ideas, while a locally
> configured url="somwhere" and "please populate my submodule" are (with
> the blank-url case defaulting to the superproject itself).

Why would we want to remove "init"? It still has to copy the "url"
setting (and it would be a compatibility nightmare if we would change
that, imagine different git versions used on the same work tree).

>> What real world problems do we have with the current init/sync that
>> this approach would solve?
> 
> I don't have any, but in my `update --remote` series I'm adding two
> new config options that are handled differently (define in
> .gitmodules, override in superproject .git/config) than existing
> submodules options.

No, they're not. They are just handled differently than "url" and
"update", but will behave just like "fetchRecurseSubmodules" and
"ignore" do since day one. And as I explained in another mail I
think "url" is special and "update" should be change to behave like
the other two some day.

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1956 bytes --]

On Sat, Dec 01, 2012 at 07:04:05PM +0100, Jens Lehmann wrote:
> Am 01.12.2012 18:49, schrieb W. Trevor King:
> > I think removing `init` will cause some compatibility issues anyway,
> > so I was re-imaging how you do it.  I don't think update='none' and
> > "don't populate my submodule" are distinct ideas, while a locally
> > configured url="somwhere" and "please populate my submodule" are (with
> > the blank-url case defaulting to the superproject itself).
> 
> Why would we want to remove "init"? It still has to copy the "url"
> setting (and it would be a compatibility nightmare if we would change
> that, imagine different git versions used on the same work tree).

In my init-less rewrite, it doesn't have to copy the url setting.
People using older versions of Git would need to run `init` using
their old version.  Having the url defined in .git/config won't break
my init-less submodule commands, it just means that the value in
.gitmodules will be masked.

> >> What real world problems do we have with the current init/sync that
> >> this approach would solve?
> > 
> > I don't have any, but in my `update --remote` series I'm adding two
> > new config options that are handled differently (define in
> > .gitmodules, override in superproject .git/config) than existing
> > submodules options.
> 
> No, they're not. They are just handled differently than "url" and
> "update", but will behave just like "fetchRecurseSubmodules" and
> "ignore" do since day one. And as I explained in another mail I
> think "url" is special and "update" should be change to behave like
> the other two some day.

I somehow missed those earlier.  Thanks for correcting my tunnel
vision.  This makes me much happier about postponing the init-removal.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH] submodule: add 'deinit' command

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> With "git submodule init" the user is able to tell git he cares about one
> or more submodules and wants to have it populated on the next call to "git
> submodule update". But currently there is no easy way he could tell git he
> does not care about a submodule anymore and wants to get rid of his local
> work tree (except he knows a lot about submodule internals and removes the
> "submodule.$name.url" setting from .git/config himself).
>
> Help those users by providing a 'deinit' command. This removes the url
> setting from .git/config either for the given submodule(s) or for all
> those which have been initialized if none were given. Complain only when
> for a submodule given on the command line the url setting can't be found
> in .git/config.
>
> Add tests and link the man pages of "git submodule deinit" and "git rm" to
> assist the user in deciding whether removing or unregistering the submodule
> is the right thing to do for him.
>
> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
> ---

I fully agree with your analysis on the reason why the "url" element
is special and has to be copied to $GIT_DIR/config, but when you
deinit (or uninit) a submodule to say you are no longer interested
in it and do not want it populated in the context of the
superproject, I am not sure if removing only submodule.$name.url (so
that when you later decide to "init" it again, you will keep the
values for submodule.$name.update and other things from the previous
life) is the sane thing to do, or it is better to remove
submodule.$name.* altogether as if an earlier "init" has never
happened.  Would it be worth analyzing the pros-and-cons here?

> Am 01.12.2012 16:56, schrieb Jens Lehmann:
>> Am 01.12.2012 00:52, schrieb Phil Hord:
>>> If I never 'submodule init' a submodule, it does not get visited by
>>> 'git submodule foreach', among others.  I think some people use this
>>> behavior explicitly.
>>>
>>> On the other hand, I've also notice that a submodule which I have
>>> removed does not get de-inited later one.  It causes my 'git submodule
>>> foreach' to emit errors.  :-(
>> 
>> I'm currently hacking on "git submodule deinit" which removes the 'url'
>> setting from git/config. This should do the trick for you, right?
>
> And here we go ...
>
>
>  Documentation/git-rm.txt        |  4 ++++
>  Documentation/git-submodule.txt | 11 +++++++++
>  git-submodule.sh                | 50 ++++++++++++++++++++++++++++++++++++++++-
>  t/t7400-submodule-basic.sh      | 11 +++++++++
>  4 files changed, 75 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt
> index 262436b..ec42bf5 100644
> --- a/Documentation/git-rm.txt
> +++ b/Documentation/git-rm.txt
> @@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree.
>  Ignored files are deemed expendable and won't stop a submodule's work
>  tree from being removed.
>
> +If you only want to remove the local checkout of a submodule from your
> +work tree without committing that use `git submodule deinit` instead
> +(see linkgit:git-submodule[1]).
> +
>  EXAMPLES
>  --------
>  `git rm Documentation/\*.txt`::
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index b1de3ba..fba77f6 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -13,6 +13,7 @@ SYNOPSIS
>  	      [--reference <repository>] [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
> +'git submodule' [--quiet] deinit [--] [<path>...]
>  'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
>  	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
> @@ -134,6 +135,16 @@ init::
>  	the explicit 'init' step if you do not intend to customize
>  	any submodule locations.
>
> +deinit::
> +	Unregister the submodules, i.e. remove the `submodule.$name.url`
> +	setting from .git/config. Further calls to `git submodule update`,
> +	`git submodule foreach` and `git submodule sync` will skip any
> +	unregistered submodules until they are initialized again, so use
> +	this command if you don't want to have a local checkout of the
> +	submodule in your work tree anymore. If you really want to remove
> +	a submodule from the repository and commit that use
> +	linkgit:git-rm[1] instead.
> +
>  update::
>  	Update the registered submodules, i.e. clone missing submodules and
>  	checkout the commit specified in the index of the containing repository.
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 2365149..4059a2e 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
>  USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
>     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] init [--] [<path>...]
> +   or: $dashless [--quiet] deinit [--] [<path>...]
>     or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
>     or: $dashless [--quiet] foreach [--recursive] <command>
> @@ -516,6 +517,53 @@ cmd_init()
>  }
>
>  #
> +# Unregister submodules from .git/config
> +#
> +# $@ = requested paths (default to all)
> +#
> +cmd_deinit()
> +{
> +	# parse $args after "submodule ... init".
> +	while test $# -ne 0
> +	do
> +		case "$1" in
> +		-q|--quiet)
> +			GIT_QUIET=1
> +			;;
> +		--)
> +			shift
> +			break
> +			;;
> +		-*)
> +			usage
> +			;;
> +		*)
> +			break
> +			;;
> +		esac
> +		shift
> +	done
> +
> +	module_list "$@" |
> +	while read mode sha1 stage sm_path
> +	do
> +		die_if_unmatched "$mode"
> +		name=$(module_name "$sm_path") || exit
> +		url=$(git config submodule."$name".url)
> +		if test -z "$url"
> +		then
> +			# Only mention uninitialized submodules when its
> +			# path have been specified
> +			test "$#" != "0" &&
> +			say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")"
> +			continue
> +		fi
> +		git config --unset submodule."$name".url &&
> +		say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")"
> +	done
> +}
> +
> +#
>  # Update each submodule path to correct revision, using clone and checkout as needed
>  #
>  # $@ = requested paths (default to all)
> @@ -1108,7 +1156,7 @@ cmd_sync()
>  while test $# != 0 && test -z "$command"
>  do
>  	case "$1" in
> -	add | foreach | init | update | status | summary | sync)
> +	add | foreach | init | deinit | update | status | summary | sync)
>  		command=$1
>  		;;
>  	-q|--quiet)
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index de7d453..803bda7 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -756,4 +756,15 @@ test_expect_success 'submodule add with an existing name fails unless forced' '
>  	)
>  '
>
> +test_expect_success 'submodule deinit should unregister submodule url from .git/config' '
> +	url=$(git config submodule.example.url) &&
> +	git submodule deinit &&
> +	test -z "$(git config submodule.example.url)"
> +'
> +
> +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' '
> +	git submodule deinit &&
> +	test_must_fail git submodule deinit example
> +'
> +
>  test_done

[PATCH v6 0/4] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote:
> On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote:
> > I've always felt that the "origin" defaults are broken and are simply
> > being ignored because most users do not trip over them.  But ISTR that
> > submodule commands use the remote indicated by the superproject's
> > current remote-tracking configuration, with a fallback to 'origin' if
> > there is none.  Sort of a "best effort" algorithm, I think.  Am I
> > remembering that wrong?
>
> The current code uses a bare "git-fetch".  I'm not sure what that
> defaults to if you're on a detached head.  If it bothers you, I'm fine
> adding the submodule.<name>.remote option in v6.

Here it is.  Now the remote defaults to $(get_default_remote) with an
optional override via submodule.<name>.remote.

Changes since v5:

* New patch 1 for easy config variable setup.
* Minor tweaks and rewordings in patches 2 and 3 (v5 patches 1 and 2).
* New patch 4 adding submodule.<name>.remote.

I'm fine with squashing patches 1, 2, and 4 together, if people prefer
a more compact series.

W. Trevor King (4):
  submodule: add get_submodule_config helper funtion
  submodule update: add --remote for submodule's upstream changes
  submodule add: If --branch is given, record it in .gitmodules
  submodule update: add submodule.<name>.remote config option

 Documentation/config.txt        |  8 ++++-
 Documentation/git-submodule.txt | 27 ++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++
 git-submodule.sh                | 74 ++++++++++++++++++++++++++++++++++++++---
 t/t7400-submodule-basic.sh      |  1 +
 t/t7406-submodule-update.sh     | 49 +++++++++++++++++++++++++++
 6 files changed, 158 insertions(+), 6 deletions(-)

-- 
1.8.0.4.gf74b0fc.dirty

[PATCH v6 1/4] submodule: add get_submodule_config helper funtion

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

Several submodule configuration variables
(e.g. fetchRecurseSubmodules) are read from .gitmodules with local
overrides from the usual git config files.  This shell function mimics
that logic to help initialize configuration variables in
git-submodule.sh.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 git-submodule.sh | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..97ce5e4 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -152,6 +152,33 @@ die_if_unmatched ()
 }
 
 #
+# Print a submodule configuration setting
+#
+# $1 = submodule name
+# $2 = option name
+# $3 = default value
+#
+# Checks in the usual git-config places first (for overrides),
+# otherwise it falls back on .gitmodules.  This allows you to
+# distribute project-wide defaults in .gitmodules, while still
+# customizing individual repositories if necessary.  If the option is
+# not in .gitmodules either, print a default value.
+#
+get_submodule_config()
+{
+	name="$1"
+	option="$2"
+	default="$3"
+	value=$(git config submodule."$name"."$option")
+	if test -z "$value"
+	then
+		value=$(git config -f .gitmodules submodule."$name"."$option")
+	fi
+	printf '%s' "${value:-$default}"
+}
+
+
+#
 # Map submodule path to submodule name
 #
 # $1 = path
-- 
1.8.0.4.gf74b0fc.dirty

[PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1).  Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1.  This helps you keep up with
changes in the upstream superproject.

However, it's also useful to stay up to date with changes in the
upstream subproject.  Previous workflows for incorporating such
changes include the ungainly:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates.  When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch.  If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`.  You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).

Previous use of submodule.<name>.branch
=======================================

Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option.  The foreach-pull
example above was described by Ævar in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.

Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same.  The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls.  The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`.  Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.

Implementation details
======================

In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1.  However, I didn't change the logic guarding the existing fetch:

  if test -z "$nofetch"
  then
    # Run fetch only if $sha1 isn't present or it
    # is not reachable from a ref.
    (clear_local_git_env; cd "$path" &&
      ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
       test -z "$rev") || git-fetch)) ||
    die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
  fi

There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository.  If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/config.txt        |  7 ++++++-
 Documentation/git-submodule.txt | 25 ++++++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 22 +++++++++++++++++++++-
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
 5 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 11f320b..6f4663c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1998,7 +1998,12 @@ submodule.<name>.update::
 	for a submodule.  These variables are initially populated
 	by 'git submodule init'; edit them to override the
 	URL and other values found in the `.gitmodules` file.  See
-	linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
+
+submodule.<name>.branch::
+	The remote branch name for a submodule, used by `git submodule
+	update --remote`.  Set this option to override the value found in
+	the `.gitmodules` file.  See linkgit:git-submodule[1] and
+	linkgit:gitmodules[5] for details.
 
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..72dd52f 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,7 +13,7 @@ SYNOPSIS
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
-'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
+'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
@@ -236,6 +236,29 @@ OPTIONS
 	(the default). This limit only applies to modified submodules. The
 	size is always limited to 1 for added/deleted/typechanged submodules.
 
+--remote::
+	This option is only valid for the update command.  Instead of using
+	the superproject's recorded SHA-1 to update the submodule, use the
+	status of the submodule's remote tracking branch.  The remote used
+	is branch's remote (`branch.<name>.remote`), defaulting to `origin`.
+	The remote branch used defaults to `master`, but the branch name may
+	be overridden by setting the `submodule.<name>.branch` option in
+	either `.gitmodules` or `.git/config` (with `.git/config` taking
+	precedence).
++
+This works for any of the supported update procedures (`--checkout`,
+`--rebase`, etc.).  The only change is the source of the target SHA-1.
+For example, `submodule update --remote --merge` will merge upstream
+submodule changes into the submodules, while `submodule update
+--merge` will merge superproject gitlink changes into the submodules.
++
+In order to ensure a current tracking branch state, `update --remote`
+fetches the submodule's remote repository before calculating the
+SHA-1.  This makes `submodule update --remote --merge` similar to
+running `git pull` in the submodule.  If you don't want to fetch (for
+something closer to `git merge`), you should use `submodule update
+--remote --no-fetch --merge`.
+
 -N::
 --no-fetch::
 	This option is only valid for the update command.
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index 4effd78..4004fa6 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -47,6 +47,11 @@ submodule.<name>.update::
 	This config option is overridden if 'git submodule update' is given
 	the '--merge', '--rebase' or '--checkout' options.
 
+submodule.<name>.branch::
+	A remote branch name for tracking updates in the upstream submodule.
+	If the option is not specified, it defaults to 'master'.  See the
+	`--remote` documentation in linkgit:git-submodule[1] for details.
+
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
 	submodule. If this option is also present in the submodules entry in
diff --git a/git-submodule.sh b/git-submodule.sh
index 97ce5e4..104b5de 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
-   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+ges
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--] [<path>...]"
@@ -26,6 +27,7 @@ cached=
 recursive=
 init=
 files=
+remote=
 nofetch=
 update=
 prefix=
@@ -536,6 +538,9 @@ cmd_update()
 		-i|--init)
 			init=1
 			;;
+		--remote)
+			remote=1
+			;;
 		-N|--no-fetch)
 			nofetch=1
 			;;
@@ -596,6 +601,7 @@ cmd_update()
 		fi
 		name=$(module_name "$sm_path") || exit
 		url=$(git config submodule."$name".url)
+		branch=$(get_submodule_config "$name" branch master)
 		if ! test -z "$update"
 		then
 			update_module=$update
@@ -630,6 +636,20 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
 		fi
 
+		if test -n "$remote"
+		then
+			if test -z "$nofetch"
+			then
+				# Fetch remote before determining tracking $sha1
+				(clear_local_git_env; cd "$sm_path" && git-fetch) ||
+				die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
+			fi
+			remote_name=$(get_default_remote)
+			sha1=$(clear_local_git_env; cd "$sm_path" &&
+				git rev-parse --verify "${remote_name}/${branch}") ||
+			die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")"
+		fi
+
 		if test "$subsha1" != "$sha1" -o -n "$force"
 		then
 			subforce=$force
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 1542653..a567834 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' '
 	)
 '
 
+test_expect_success 'submodule update --remote should fetch upstream changes' '
+	(cd submodule &&
+	 echo line4 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line4"
+	) &&
+	(cd super &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)"
+	)
+'
+
+test_expect_success 'local config should override .gitmodules branch' '
+	(cd submodule &&
+	 git checkout -b test-branch &&
+	 echo line5 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line5" &&
+	 git checkout master
+	) &&
+	(cd super &&
+	 git config submodule.submodule.branch test-branch &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)"
+	)
+'
+
 test_expect_success 'submodule update --rebase staying on master' '
 	(cd super/submodule &&
 	  git checkout master
-- 
1.8.0.4.gf74b0fc.dirty

[PATCH v6 3/4] submodule add: If --branch is given, record it in .gitmodules

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This allows you to easily record a submodule.<name>.branch option in
.gitmodules when you add a new submodule.  With this patch,

  $ git submodule add -b <branch> <repository> [<path>]
  $ git config -f .gitmodules submodule.<path>.branch <branch>

reduces to

  $ git submodule add -b <branch> <repository> [<path>]

This means that future calls to

  $ git submodule update --remote ...

will get updates from the same branch that you used to initialize the
submodule, which is usually what you want.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 2 ++
 git-submodule.sh                | 4 ++++
 t/t7400-submodule-basic.sh      | 1 +
 3 files changed, 7 insertions(+)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 72dd52f..988bba9 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -208,6 +208,8 @@ OPTIONS
 -b::
 --branch::
 	Branch of repository to add as submodule.
+	The name of the branch is recorded as `submodule.<path>.branch` in
+	`.gitmodules` for `update --remote`.
 
 -f::
 --force::
diff --git a/git-submodule.sh b/git-submodule.sh
index 104b5de..27b02fe 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -395,6 +395,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
+	if test -n "$branch"
+	then
+		git config -f .gitmodules submodule."$sm_path".branch "$branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5397037..90e2915 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test "initial" = "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
 		git submodule init
 	) &&
 
-- 
1.8.0.4.gf74b0fc.dirty

[PATCH v6 4/4] submodule update: add submodule.<name>.remote config option

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

Don't force the user to clone from the tracked repository
(branch.<name>.remote) or `origin`.  By setting
submodule.<name>.remote in .gitmodules or the usual git config files,
you can easily point a submodule at a different remote when using
`submodule update --remote`.

The configured remote name is also used in `submodule sync` to
determine which remote.<name>.url is updated with the submodule's
origin URL.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/config.txt        |  7 ++++---
 Documentation/git-submodule.txt | 10 +++++-----
 git-submodule.sh                | 27 +++++++++++++++++++++------
 t/t7406-submodule-update.sh     | 18 ++++++++++++++++++
 4 files changed, 48 insertions(+), 14 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 6f4663c..c54b9b4 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1999,10 +1999,11 @@ submodule.<name>.update::
 	by 'git submodule init'; edit them to override the
 	URL and other values found in the `.gitmodules` file.  See
 
+submodule.<name>.remote::
 submodule.<name>.branch::
-	The remote branch name for a submodule, used by `git submodule
-	update --remote`.  Set this option to override the value found in
-	the `.gitmodules` file.  See linkgit:git-submodule[1] and
+	The remote repository and branch names for a submodule, used by `git
+	submodule update --remote`.  Set these options to override the value
+	found in the `.gitmodules` file.  See linkgit:git-submodule[1] and
 	linkgit:gitmodules[5] for details.
 
 submodule.<name>.fetchRecurseSubmodules::
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 988bba9..1d8d5f1 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -242,11 +242,11 @@ OPTIONS
 	This option is only valid for the update command.  Instead of using
 	the superproject's recorded SHA-1 to update the submodule, use the
 	status of the submodule's remote tracking branch.  The remote used
-	is branch's remote (`branch.<name>.remote`), defaulting to `origin`.
-	The remote branch used defaults to `master`, but the branch name may
-	be overridden by setting the `submodule.<name>.branch` option in
-	either `.gitmodules` or `.git/config` (with `.git/config` taking
-	precedence).
+	is branch's remote (`branch.<name>.remote`, defaulting to `origin`),
+	and the remote branch used defaults to `master`, but either may be
+	overridden by setting the `submodule.<name>.remote` or
+	`submodule.<name>.branch` option in `.gitmodules` or `.git/config`
+	(with `.git/config` taking precedence).
 +
 This works for any of the supported update procedures (`--checkout`,
 `--rebase`, etc.).  The only change is the source of the target SHA-1.
diff --git a/git-submodule.sh b/git-submodule.sh
index 27b02fe..3e39e29 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -179,6 +179,21 @@ get_submodule_config()
 	printf '%s' "${value:-$default}"
 }
 
+#
+# Print the name of a submodule's configured remote
+#
+# $1 = submodule name
+#
+get_submodule_remote()
+{
+	name="$1"
+	remote=$(get_submodule_config "$name" remote)
+	if test -z "$remote"
+	then
+		remote=$(get_default_remote)
+	fi
+	printf '%s' "${remote}"
+}
 
 #
 # Map submodule path to submodule name
@@ -605,6 +620,7 @@ cmd_update()
 		fi
 		name=$(module_name "$sm_path") || exit
 		url=$(git config submodule."$name".url)
+		remote_name=$(get_submodule_remote "$name")
 		branch=$(get_submodule_config "$name" branch master)
 		if ! test -z "$update"
 		then
@@ -645,10 +661,9 @@ Maybe you want to use 'update --init'?")"
 			if test -z "$nofetch"
 			then
 				# Fetch remote before determining tracking $sha1
-				(clear_local_git_env; cd "$sm_path" && git-fetch) ||
-				die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
+				(clear_local_git_env; cd "$sm_path" && git-fetch "$remote_name") ||
+				die "$(eval_gettext "Unable to fetch '\$remote_name' in submodule path '\$sm_path'")"
 			fi
-			remote_name=$(get_default_remote)
 			sha1=$(clear_local_git_env; cd "$sm_path" &&
 				git rev-parse --verify "${remote_name}/${branch}") ||
 			die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")"
@@ -669,8 +684,8 @@ Maybe you want to use 'update --init'?")"
 				# is not reachable from a ref.
 				(clear_local_git_env; cd "$sm_path" &&
 					( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
-					 test -z "$rev") || git-fetch)) ||
-				die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
+					 test -z "$rev") || git-fetch "$remote_name")) ||
+				die "$(eval_gettext "Unable to fetch '\$remote_name' in submodule path '\$sm_path'")"
 			fi
 
 			# Is this something we just cloned?
@@ -1110,7 +1125,7 @@ cmd_sync()
 			(
 				clear_local_git_env
 				cd "$sm_path"
-				remote=$(get_default_remote)
+				remote=$(get_submodule_remote "$name")
 				git config remote."$remote".url "$sub_origin_url"
 			)
 			fi
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index a567834..86c85f8 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -149,6 +149,24 @@ test_expect_success 'submodule update --remote should fetch upstream changes' '
 	)
 '
 
+test_expect_success 'local config should override .gitmodules remote' '
+	(cd submodule &&
+	 echo line5-master >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line5-master"
+	) &&
+	(cd super/submodule &&
+	 git remote rename origin test-remote
+	) &&
+	(cd super &&
+	 git config submodule.submodule.remote test-remote &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)"
+	)
+'
+
 test_expect_success 'local config should override .gitmodules branch' '
 	(cd submodule &&
 	 git checkout -b test-branch &&
-- 
1.8.0.4.gf74b0fc.dirty

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 5933 bytes --]

Before I get into the details, I'd like to point out that I actually
understand the purpose of `submodule init` now ;).  To avoid further
confusion, my current one-line command summaries would be:

  init:   mark a submodule as active for future submodule operation
  deinit: mark a submodule as inactive for future submodule operation
  sync:   update remote.<name>.origin in submodules to reflect changes
          in .gitmodules or the superproject's remote URL.

I don't think we disagree on that, we just don't agree on how to
implement it.

Currently, Git uses submodule.<name>.url in the superproject's local
configuration as a marker for submodule activation.  This is not (as
far as I know) discussed in the docs, which is why I initially
missunderstood the purpose of `init` to be “setup the superproject's
local configuration so we don't have to keep resolving the submodules
URL relative to the superproject's upstream URL”.  With the proposed
`deinit` docs, this role for the submodule.<name>.url is mentioned,
but not in a place where casual users will be able to easily connect
it to the purpose of `init`.

I floated using submodule.<name>.update (with 'none' for inactive and
anything else for active) as an alternative marker:

On Sat, Dec 01, 2012 at 01:16:43PM -0500, W. Trevor King wrote:
> On Sat, Dec 01, 2012 at 07:04:05PM +0100, Jens Lehmann wrote:
> > Am 01.12.2012 18:49, schrieb W. Trevor King:
> > > I think removing `init` will cause some compatibility issues anyway,
> > > so I was re-imaging how you do it.  I don't think update='none' and
> > > "don't populate my submodule" are distinct ideas, while a locally
> > > configured url="somwhere" and "please populate my submodule" are (with
> > > the blank-url case defaulting to the superproject itself).
> > 
> > Why would we want to remove "init"? It still has to copy the "url"
> > setting (and it would be a compatibility nightmare if we would change
> > that, imagine different git versions used on the same work tree).
> 
> In my init-less rewrite, it doesn't have to copy the url setting.
> People using older versions of Git would need to run `init` using
> their old version.  Having the url defined in .git/config won't break
> my init-less submodule commands, it just means that the value in
> .gitmodules will be masked.

but that doesn't seem to be going over very well.  Junio may have been
weighing in obliquely with:

On Sat, Dec 01, 2012 at 06:00:06PM -0800, Junio C Hamano wrote:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> > [snip v1 deinit commit message]
> 
> I fully agree with your analysis on the reason why the "url" element
> is special and has to be copied to $GIT_DIR/config, but when you
> deinit (or uninit) a submodule to say you are no longer interested
> in it and do not want it populated in the context of the
> superproject, I am not sure if removing only submodule.$name.url (so
> that when you later decide to "init" it again, you will keep the
> values for submodule.$name.update and other things from the previous
> life) is the sane thing to do, or it is better to remove
> submodule.$name.* altogether as if an earlier "init" has never
> happened.  Would it be worth analyzing the pros-and-cons here?

Let me take another stab at presenting my argument in favor of a
different activity marker.

Proposal:

Add a new boolean option, submodule.<name>.active, to explicitly mark
submodules as active (with “active” defined as “to be returned by
module_list()”).  Strip down `init` (and the --init part of `update
--init`) to just setting this option to true.  `deinit` only sets this
option to false (but a `deinit --clean` could remove the whole
submodule.<name> section).

With this in place, extracting URLs for submodule operations be
similar to the extraction of other variables (.gitmodules defaults
with superproject-local .git/config overrides).  This also makes it
easier to track maintenance updates in .gitmodules-defined URLs,
because you aren't forced to bake overrides into your local
.git/config

The upgrade path from earlier versions of Git is easy: if
submodule.<name>.active is unset, use the presence of
submodule.<name>.url to determine its initial value.

In the case where you check out an earlier superproject commit which
is missing a particular submodule (or remove a submodule without
deinit-ing), the presense of an active setting in .git/config should
not cause an error, which they currently seem to:

On Sat, Dec 01, 2012 at 11:37:14AM -0500, W. Trevor King wrote:
> On Sat, Dec 01, 2012 at 04:56:02PM +0100, Jens Lehmann wrote:
> > Am 01.12.2012 00:52, schrieb Phil Hord:
> > > If I never 'submodule init' a submodule, it does not get visited by
> > > 'git submodule foreach', among others.  I think some people use this
> > > behavior explicitly.
> > >
> > > On the other hand, I've also notice that a submodule which I have
> > > removed does not get de-inited later one.  It causes my 'git submodule
> > > foreach' to emit errors.  :-(
> >
> > I'm currently hacking on "git submodule deinit" which removes the 'url'
> > setting from git/config. This should do the trick for you, right?
> >
> > Just removing that submodule automagically would not work that well, as
> > it would deinitialize a submodule when you switch to a branch where it
> > isn't present and you'd have to reinitialize it when you come back.
>
> I think this is another case where we should be looping through
> submodules based on the revision-specific .gitmodules content, and
> querying the local config only to determine if the user wants to
> update them (to drop into them with foreach, etc.).

Thoughts?

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes

[Jens Lehmann]

Am 30.11.2012 04:27, schrieb W. Trevor King:
> On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote:
>> On Thu, Nov 29, 2012 at 2:13 PM, W. Trevor King <wking@tremily.us> wrote:
>>> On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote:
>>>> But I really don't want to figure out how to handle submodule
>>>> collisions during a merge (or rebase!) of my superproject with changes that
>>>> someone else auto-committed in his local $superproject as he and I
>>>> arbitrarily floated up the upstream independently.  There is nothing but
>>>> loathing down that path.
>>>
>>> This is true.  I'm not sure how gitlink collisions are currently
>>> handled…
>>
>> They've always been trouble for me.  But it may be that I am ignorant.
> 
> I haven't dealt with any gitlink merges, but I think that supporting
> easy gitlink merges is orthogonal to this --remote option.  For simple
> cases like "autocommitted submodule floats", one of the conflicting
> gitlinks will be an ancestor of the other, so it should be easy to
> automate that merge.

Submodule merges where one submodule commit is the ancestor of the
other are already resolved automatically in recent git. So Phil's
example will just work as long as only fast-forward merges are needed.

Re: [PATCH] submodule: add 'deinit' command

[Jens Lehmann]

Am 02.12.2012 03:00, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> With "git submodule init" the user is able to tell git he cares about one
>> or more submodules and wants to have it populated on the next call to "git
>> submodule update". But currently there is no easy way he could tell git he
>> does not care about a submodule anymore and wants to get rid of his local
>> work tree (except he knows a lot about submodule internals and removes the
>> "submodule.$name.url" setting from .git/config himself).
>>
>> Help those users by providing a 'deinit' command. This removes the url
>> setting from .git/config either for the given submodule(s) or for all
>> those which have been initialized if none were given. Complain only when
>> for a submodule given on the command line the url setting can't be found
>> in .git/config.
>>
>> Add tests and link the man pages of "git submodule deinit" and "git rm" to
>> assist the user in deciding whether removing or unregistering the submodule
>> is the right thing to do for him.
>>
>> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
>> ---
> 
> I fully agree with your analysis on the reason why the "url" element
> is special and has to be copied to $GIT_DIR/config, but when you
> deinit (or uninit) a submodule to say you are no longer interested
> in it and do not want it populated in the context of the
> superproject, I am not sure if removing only submodule.$name.url (so
> that when you later decide to "init" it again, you will keep the
> values for submodule.$name.update and other things from the previous
> life) is the sane thing to do, or it is better to remove
> submodule.$name.* altogether as if an earlier "init" has never
> happened.  Would it be worth analyzing the pros-and-cons here?

Sure. I was worried about throwing away other settings the user
might have set in the submodule.$name section and the first reflex
was to protect them. But thinking about that again I noticed we are
already throwing away a possibly user customized "url" setting, so
we already remove a possibly customized setting.

Maybe the principle of least surprise is better followed when we
nuke the whole section, as it might surprise the user more to have
a setting resurrected he customized in the last life cycle of the
submodule than seeing that after an deinit followed by an init all
former customizations are consistently gone. So I tend to think now
that removing the whole section would be the better solution here.

Opinions by other submodule users?

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[Jens Lehmann]

Am 02.12.2012 20:09, schrieb W. Trevor King:
> Before I get into the details, I'd like to point out that I actually
> understand the purpose of `submodule init` now ;).  To avoid further
> confusion, my current one-line command summaries would be:
> 
>   init:   mark a submodule as active for future submodule operation
>   deinit: mark a submodule as inactive for future submodule operation
>   sync:   update remote.<name>.origin in submodules to reflect changes
>           in .gitmodules or the superproject's remote URL.
> 
> I don't think we disagree on that, we just don't agree on how to
> implement it.

Nope, it is already implemented and you are arguing to change the
current implementation. To quote from another mail:

Am 01.12.2012 18:49, schrieb W. Trevor King:
> On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote:
>> What real world problems do we have with the current init/sync that
>> this approach would solve?
>
> I don't have any, ...

We don't want to change working code and cause compatibility issues
just because we /could/ do things differently, no?

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>
To: Jens Lehmann <Jens.Lehmann@web.de>, Junio C Hamano <gitster@pobox.com>
Cc: Phil Hord <phil.hord@gmail.com>, Git <git@vger.kernel.org>,
	Heiko Voigt <hvoigt@hvoigt.net>, Jeff King <peff@peff.net>,
	Shawn Pearce <spearce@spearce.org>, Nahor <nahor.j+gmane@gmail.com>
Bcc: 
Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync'
Reply-To: 
In-Reply-To: <50BBBA29.2000106@web.de>
 <50BBB22A.7050901@web.de>
 <20121202190929.GG9401@odin.tremily.us>
OpenPGP: id=39A2F3FA2AB17E5D8764F388FC29BDCDF15F5BE8;
 url=http://tremily.us/pubkey.txt

On Sun, Dec 02, 2012 at 09:29:29PM +0100, Jens Lehmann wrote:
> Am 02.12.2012 20:09, schrieb W. Trevor King:
> > Before I get into the details, I'd like to point out that I actually
> > understand the purpose of `submodule init` now ;).  To avoid further
> > confusion, my current one-line command summaries would be:
> > 
> >   init:   mark a submodule as active for future submodule operation
> >   deinit: mark a submodule as inactive for future submodule operation
> >   sync:   update remote.<name>.origin in submodules to reflect changes
> >           in .gitmodules or the superproject's remote URL.
> > 
> > I don't think we disagree on that, we just don't agree on how to
> > implement it.
> 
> Nope, it is already implemented and you are arguing to change the
> current implementation.

Agreed.

> To quote from another mail:
> 
> Am 01.12.2012 18:49, schrieb W. Trevor King:
> > On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote:
> >> What real world problems do we have with the current init/sync that
> >> this approach would solve?
> >
> > I don't have any, ...
> 
> We don't want to change working code and cause compatibility issues
> just because we /could/ do things differently, no?

In principle, yes, but in this case I think changing the
implementation does not risk much in the way of compatibility issues
(it only hurts users who rely on `submodule init` setting
submodule.<name>.url for reasons of their own.  A few of the existing
tests explictly check the url setting, so perhaps there are a number
of users who do require this side effect?

I think this risk is outweighed by the benefits of having a clearer
activation option.  For example:

On Sun, Dec 02, 2012 at 08:55:22PM +0100, Jens Lehmann wrote:
> Sure. I was worried about throwing away other settings the user
> might have set in the submodule.$name section and the first reflex
> was to protect them. But thinking about that again I noticed we are
> already throwing away a possibly user customized "url" setting, so
> we already remove a possibly customized setting.

With submodule.<name>.active, there's nothing customized that you'd
have to nuke on deinit (except 'active' iteself, which the user is
explicitly asking for).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

Re: [PATCH] submodule: add 'deinit' command

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Maybe the principle of least surprise is better followed when we
> nuke the whole section, as it might surprise the user more to have
> a setting resurrected he customized in the last life cycle of the
> submodule than seeing that after an deinit followed by an init all
> former customizations are consistently gone. So I tend to think now
> that removing the whole section would be the better solution here.

I tend to agree; I suspect that a "deinit" would be mostly done
either to

 (1) correct mistakes the user made during a recent "init" and
     perhaps "sync"; or

 (2) tell Git that the user has finished woing with this particular
     submodule and does not intend to use it for quite a while.

For both (1) and (2), I think it would be easier to users if we gave
them a clean slate, the same state as the one the user who never had
ran "init" on it would be in.  A user in situation (1) is asking for
a clean slate, and a user in situation (2) is better served if he
does not have to worry about leftover entries in $GIT_DIR/config he
has long forgotten from many months ago (during which time the way
the project uses the particular submodule may well have changed)
giving non-standard experience different from what other project
participants would get.

If there were a sane workflow where it makes sense to frequently run
"deinit" followed by some operation followed by "init", it may be
helpful to have an option to keep the other customization.  And one
consideration when implementing that "deinit --keep-customization"
option might be to introduce the submodule.$name.activated boolean;
that way, the operation can keep the customized upstream URL.

In any case, it needs to be shown that such a workflow exists in the
first place to justify "deinit --keep-customization".  I think the
default should be to remove the submodule.$name section.

Re: [RFC] remove/deprecate 'submodule init' and 'sync'

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 3385 bytes --]

On Sun, Dec 02, 2012 at 04:11:59PM -0500, W. Trevor King wrote:
> On Sun, Dec 02, 2012 at 09:29:29PM +0100, Jens Lehmann wrote:
> > Am 01.12.2012 18:49, schrieb W. Trevor King:
> > > On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote:
> > >> What real world problems do we have with the current init/sync that
> > >> this approach would solve?
> > >
> > > I don't have any, ...
> > 
> > We don't want to change working code and cause compatibility issues
> > just because we /could/ do things differently, no?
> 
> In principle, yes, but in this case I think changing the
> implementation does not risk much in the way of compatibility issues
> (it only hurts users who rely on `submodule init` setting
> submodule.<name>.url for reasons of their own.  A few of the existing
> tests explictly check the url setting, so perhaps there are a number
> of users who do require this side effect?
> 
> I think this risk is outweighed by the benefits of having a clearer
> activation option.

For anyone interested in an implementation of my
submodule.<name>.active proposal, I've posted an initial version:

  git://github.com/wking/git.git wtk/submodule.name.active

I can re-post it here as a PATCH series, but I don't think we're at
the level of patch-specific feedback yet.

I'm currently pretty happy with it except for the last commit:

  HACK work around missing index entry for existing empty submodules

To solve that cleanly, I'd need a solution to the commit-less existing
repository problem which I mentioned earlier:

On Sat, Dec 01, 2012 at 11:54:04AM -0500, W. Trevor King wrote:
> I'm currently stuck with adding a commit-less existing repository as a
> submodule (which happens in t7400-submodule-basic.sh, ../bar/a/b/c
> works with relative local path):
> 
>   $ mkdir -p super/sub
>   $ cd super
>   $ git init
>   $ (cd sub && git init)
>   $ git submodule add ./ sub
>   $ git status
>   # On branch master
>   #
>   # Initial commit
>   #
>   # Changes to be committed:
>   #   (use "git rm --cached <file>..." to unstage)
>   #
>   #       new file:   .gitmodules
>   #
> 
> What I'm missing is a gitlink form sub for 'Subproject commit
> 00000...' or some such.  When the subproject has an actual commit,
> things work as expected:
> 
>   $ mkdir -p super/sub
>   $ cd super
>   $ git init
>   $ (cd sub && git init && echo line-1 > file && git add file && git commit -m file)
>   $ git submodule add ./ sub
>   $ git status
>   # On branch master
>   #
>   # Initial commit
>   #
>   # Changes to be committed:
>   #   (use "git rm --cached <file>..." to unstage)
>   #
>   #       new file:   .gitmodules
>   #       new file:   sub
>   #
> 
> This means that module_list isn't aware of the empty submodule, when
> the user has just explicitly added it.  Fixing this would seem to need
> either 'Subproject commit 00000...' as I suggested earlier, or an
> adjustment to module_list that also spits out submodules that are in
> .gitmodules but not in the index.

Other than that, I think all the changes in the test suite are
logically sound and unlikely to cause problems with existing usage.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> From: "W. Trevor King" <wking@tremily.us>
>
> The current `update` command incorporates the superproject's gitlinked
> SHA-1 ($sha1) into the submodule HEAD ($subsha1).  Depending on the
> options you use, it may checkout $sha1, rebase the $subsha1 onto
> $sha1, or merge $sha1 into $subsha1.  This helps you keep up with
> changes in the upstream superproject.
>
> However, it's also useful to stay up to date with changes in the
> upstream subproject.  Previous workflows for incorporating such
> changes include the ungainly:
>
>   $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
>
> With this patch, all of the useful functionality for incorporating
> superproject changes can be reused to incorporate upstream subproject
> updates.  When you specify --remote, the target $sha1 is replaced with
> a $sha1 of the submodule's origin/master tracking branch.  If you want
> to merge a different tracking branch, you can configure the
> `submodule.<name>.branch` option in `.gitmodules`.  You can override
> the `.gitmodules` configuration setting for a particular superproject
> by configuring the option in that superproject's default configuration
> (using the usual configuration hierarchy, e.g. `.git/config`,
> `~/.gitconfig`, etc.).
>
> Previous use of submodule.<name>.branch
> =======================================
>
> Because we're adding a new configuration option, it's a good idea to
> check if anyone else is already using the option.  The foreach-pull
> example above was described by Ævar in
>
>   commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
>   Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>   Date:   Fri May 21 16:10:10 2010 +0000
>
>     git-submodule foreach: Add $toplevel variable
>
> Gerrit uses the same interpretation for the setting, but because
> Gerrit has direct access to the subproject repositories, it updates
> the superproject repositories automatically when a subproject changes.
> Gerrit also accepts the special value '.', which it expands into the
> superproject's branch name.
>
> Although the --remote functionality is using `submodule.<name>.branch`
> slightly differently, the effect is the same.  The foreach-pull
> example uses the option to record the name of the local branch to
> checkout before pulls.  The tracking branch to be pulled is recorded
> in `.git/modules/<name>/config`, which was initialized by the module
> clone during `submodule add` or `submodule init`.  Because the branch
> name stored in `submodule.<name>.branch` was likely the same as the
> branch name used during the initial `submodule add`, the same branch
> will be pulled in each workflow.
>
> Implementation details
> ======================
>
> In order to ensure a current tracking branch state, `update --remote`
> fetches the submodule's remote repository before calculating the
> SHA-1.  However, I didn't change the logic guarding the existing fetch:
>
>   if test -z "$nofetch"
>   then
>     # Run fetch only if $sha1 isn't present or it
>     # is not reachable from a ref.
>     (clear_local_git_env; cd "$path" &&
>       ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
>        test -z "$rev") || git-fetch)) ||
>     die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
>   fi
>
> There will not be a double-fetch, because the new $sha1 determined
> after the `--remote` triggered fetch should always exist in the
> repository.  If it doesn't, it's because some racy process removed it
> from the submodule's repository and we *should* be re-fetching.

As you hinted in the first paragraph, you could flip between merge,
rebase, and detach with a command line option when running the
"update" subcommand, but I would imagine that the expected use
pattern is that for a particular project, you would choose one mode
and consistently stick to that mode.  To make it easier, the user
can set submodule.$name.update once and run "update" without having
to give any flags.

And this is about adding another mode to the "update" subcommand
where the HEAD is not detached, nor merged, nor rebased, but is set
to follow whatever commit a remote branch points at.

Shouldn't the patch add a way for the user to set a configuration
variable to signal that this new mode is always used when "update"
is run without a command line flag?

As the user has to configure submodule.$name.branch in order to use
this mode anyway, I have a feeling that taking that as a signal, and
ignoring submodule.$name.update altogether, might be a simpler
interface from the end user's point of view.  That is,

 (1) if you are not interested in the submodule $name, you do not
     "init" it; you "init" it for all of the following.

 (2) if you want to have the tree state as recorded in the
     superproject, you do "update" without any option to make the
     HEAD of the submodule detached at the commit the superproject's
     tree records;

 (3) if you want to follow the upstream project of the submodule,
     you set submodule.$name.branch to the branch you want to
     follow, and you do "update"---submodule.$name.update is ignored
     and you will make the HEAD of the submodule detached at the tip
     of the branch at the remote (using remote-tracking branch);

 (4) if you want to --merge or --rebase, you give them from the
     command line, or use submodule.$name.update.

I may be oversimplifying a bit, but a separate
submodule.$name.remote feels very wrong; if it were a new token
"remote" that can be set as the value of submodule.$name.update (in
addition to existing "none", "rebase" and "merge"), it might be a
bit more understandable, though.

How does this compare with the floating submodules Heiko has been
working on?

Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 5148 bytes --]

On Mon, Dec 03, 2012 at 08:46:46AM -0800, Junio C Hamano wrote:
> As you hinted in the first paragraph, you could flip between merge,
> rebase, and detach with a command line option when running the
> "update" subcommand, but I would imagine that the expected use
> pattern is that for a particular project, you would choose one mode
> and consistently stick to that mode.  To make it easier, the user
> can set submodule.$name.update once and run "update" without having
> to give any flags.

Sure.

> And this is about adding another mode to the "update" subcommand
> where the HEAD is not detached, nor merged, nor rebased, but is set
> to follow whatever commit a remote branch points at.

This is about adding another suite of modes.  Currently you can
rebase/merge/checkout the superproject-recorded $sha1.  I'm adding the
ability to rebase/merge/checkout a submodule-upstream branch.  I
discuss this explicitly in Documentation/git-submodule.txt when
describing --remote.

> Shouldn't the patch add a way for the user to set a configuration
> variable to signal that this new mode is always used when "update"
> is run without a command line flag?

How about a new submodule.<name>.update-source with (which can be
either superproject-gitlink or submodule-upstream)?  Or to be a bit
simpler and less explicit, a submodule.<name>.update-remote boolean?
For lack of a better name, I'll call this submodule.<name>.<something>
below.

> As the user has to configure submodule.$name.branch in order to use
> this mode anyway, I have a feeling that taking that as a signal, and
> ignoring submodule.$name.update altogether, might be a simpler
> interface from the end user's point of view.  That is,

As I mention earlier, submodule.<name>.update is still important.  I
think it's good to add a new submodule.<name>.<something> config and a
--no-remote option (to override a configured
submodule.<name>.<something>).  This way a user that generally updates
via the superproject's gitlink can still configure a branch to update
from when they use --remote.

>  (1) if you are not interested in the submodule $name, you do not
>      "init" it; you "init" it for all of the following.
> 
>  (2) if you want to have the tree state as recorded in the
>      superproject, you do "update" without any option to make the
>      HEAD of the submodule detached at the commit the superproject's
>      tree records;
> 
>  (3) if you want to follow the upstream project of the submodule,
>      you set submodule.$name.branch to the branch you want to
>      follow, and you do "update"---submodule.$name.update is ignored
>      and you will make the HEAD of the submodule detached at the tip
>      of the branch at the remote (using remote-tracking branch);
> 
>  (4) if you want to --merge or --rebase, you give them from the
>      command line, or use submodule.$name.update.

But what if your whant to merge the upstream project into a currently
checked out submodule branch?  Or rebase a currently detached head
against the upstream branch?

> I may be oversimplifying a bit, but a separate
> submodule.$name.remote feels very wrong;

I use submodule.<name>.remote in patch 4 to specify the name of the
superproject's remote (for when get_default_remote doesn't give the
value you want), but I think you're referring to the potential
submodule.<name>.<something> and the presense of the --remote option.

> How does this compare with the floating submodules Heiko has been
> working on?

Heiko's older hv/floating_submodules also uses submodule.<name>.branch
(with a similar interpretation).  There's also a --branch option to
`update` for command-line overrides (which I don't have, perhaps I
should add them?).  He reverts to the original behavior in the
presense of submodule.<name>.branch with `update --checkout`, or when
submodule.<name>.branch=HEAD.

He also fetches all remotes, while I fetch just the explicitly
configured submodule.<name>.remote falling back on
$(get_default_remote.  His submodule.<name>.branch is the full local
ref for the branch (e.g. 'origin/master'), while mine is just the
remote branch (e.g. 'master').  I split the remote (e.g. 'origin')
into submodule.<name>.remote in patch 4 so you can explicitly fetch
just that remote (and not all the remotes you may have configured for
that submodule).

For reasons that I don't understand, he only supports the `checkout`
update logic for remote branches.

Heiko's newer hv/floating_submodules_draft builds on my earlier v4
--local-branch option, but he uses his own processing logic.  He pulls
the existing 'update to $sha1' logic out into a new
handle_on_demand_update() and uses the stored submodule.<name>.branch
as the name of a local submodule branch to check out when tracking.
Then he pulls that local branches default upstream (configured in
.git/modules/<name>/config) with --ff-only.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

As an example to make this clearer:

  $ cat .gitmodules
  [submodule "sub1"]
    path = sub1
    url = git://example.com/sub1.git
    remote = remote1
    branch = branch1
    update-source = submodule-upstream
    update = rebase
  [submodule "sub2"]
  ...

Means that `git submodule update sub1` will fetch remote1 and rebase
the current sub1 checkout against the tip of remote1/branch1.  The
git://example.com/sub1.git URL is not actually used during this
update.  Presumably the user setup remote1 intentionally in the
submodule, and wants to use the URL they've configured there.

Perhaps I need to ammend my

  submodule update: add submodule.<name>.remote config option

patch (#4) to adjust the remote that has it's URL changed by `sync`?

I may also want to append some form of the following commit (from my
submodule.<name>.active proposal):

  submodule add: configure existing submodule url if not set [1]

Cheers,
Trevor

[1]: https://github.com/wking/git/commit/b045c16cffe6eb86c157a6c7397166a46e147442

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v6 1/4] submodule: add get_submodule_config helper funtion

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> From: "W. Trevor King" <wking@tremily.us>
>
> Several submodule configuration variables
> (e.g. fetchRecurseSubmodules) are read from .gitmodules with local
> overrides from the usual git config files.  This shell function mimics
> that logic to help initialize configuration variables in
> git-submodule.sh.
>
> Signed-off-by: W. Trevor King <wking@tremily.us>
> ---
>  git-submodule.sh | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/git-submodule.sh b/git-submodule.sh
> index ab6b110..97ce5e4 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -152,6 +152,33 @@ die_if_unmatched ()
>  }
>  
>  #
> +# Print a submodule configuration setting
> +#
> +# $1 = submodule name
> +# $2 = option name
> +# $3 = default value
> +#
> +# Checks in the usual git-config places first (for overrides),
> +# otherwise it falls back on .gitmodules.  This allows you to
> +# distribute project-wide defaults in .gitmodules, while still
> +# customizing individual repositories if necessary.  If the option is
> +# not in .gitmodules either, print a default value.
> +#
> +get_submodule_config()
> +{

style (see CodingGuidelines):

	get_submodule_config ()	{

> +	name="$1"
> +	option="$2"
> +	default="$3"
> +	value=$(git config submodule."$name"."$option")

This will get unwieldy quickly once we have submodule.$name.$var
that takes a boolean option, as there are different ways to spell
boolean and "git config --bool" is the way to ask for canonicalized
"true" or "false".

If we never query any boolean via this helper function, it is
obviously not an issue, though.

> +	if test -z "$value"
> +	then
> +		value=$(git config -f .gitmodules submodule."$name"."$option")
> +	fi
> +	printf '%s' "${value:-$default}"
> +}
> +
> +
> +#
>  # Map submodule path to submodule name
>  #
>  # $1 = path

Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> As an example to make this clearer:
>
>   $ cat .gitmodules
>   [submodule "sub1"]
>     path = sub1
>     url = git://example.com/sub1.git
>     remote = remote1
>     branch = branch1
>     update-source = submodule-upstream
>     update = rebase
>   [submodule "sub2"]
>   ...

Maybe it is just me but that "remote = remote1" sticks out like a
sore thumb.

If you are showing the .gitmodules file to be shared as hints to
project participants, why does it even need to have both URL and
remote1?  If remote1 points at some other repository, the recipient
of this .gitmodules file would not have any clue where it is.  If
remote1 points at the same repository as the URL, why should it be
there in the first place?  The superproject is in no business to
force what local remote name each participant would call in their
submodule checkout, and more importantly, there is no _need_ to do
so.

We could extend that reasoning to the branch name (which is also a
local matter, at least technically), but this is a lot more
justifiable.  If the upstream of the superproject is the same
organization as the upstream of the submodule project, which is
often the case when a large project is organized as a forest of
submodules bound at the top-level with a superproject, the
superproject commit on a particular superproject branch may want any
update necessary to complete the superproject made to submodules on
specific branches at the central meeting place.  The superproject's
Milestone22 branch may want to bind commits that is on submodule's
Milestone22 branch.

While a participant locally *can* create M22 branch in the submodule
and set it to build upon Milestone22 branch taken from the central
repository, most people don't.  They use the same branch names
between local and remote (i.e. refs/heads/*:refs/remotes/origin/* to
keep the remote-tracking branches under the same name, and the local
branch $any builds upon the corresponding remote-tracking branch
refs/remotes/origin/$any.  Most importantly, the work done on local
branch $any is pushed out to refs/heads/$any at the remote of the
submodule).  Because of how people use "push" to push $any branch to
the branch of the same name $any at the central meeting place, and
because the upstream wants participants to use a particular branch
name in the submodule at the central meeting place, the set-up ends
up dictating what local branch name should be used.

But I do not see any reason to require or even suggest any local
nickname that is to be used to call the remote.  It really is a
local matter.  Why should .gitmodules have "remote = ..." line?

On the other hand, if you meant the above as an excerpt from
$GIT_DIR/config, it also does not make sense.  At that point, the
participant own the file and updating url to point at whatever
different repository without changing the remote name is sufficient.

It looks way over-engineered for unclear/dubious benefit.

Re: [PATCH v6 1/4] submodule: add get_submodule_config helper funtion

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 4793 bytes --]

On Mon, Dec 03, 2012 at 11:30:12AM -0800, Junio C Hamano wrote:
> > +get_submodule_config()
> > +{
> 
> style (see CodingGuidelines):
> 
> 	get_submodule_config ()	{

Will fix.  I was generally just copying the surrounding code.

> > +	name="$1"
> > +	option="$2"
> > +	default="$3"
> > +	value=$(git config submodule."$name"."$option")
> 
> This will get unwieldy quickly once we have submodule.$name.$var
> that takes a boolean option, as there are different ways to spell
> boolean and "git config --bool" is the way to ask for canonicalized
> "true" or "false".
> 
> If we never query any boolean via this helper function, it is
> obviously not an issue, though.

We do in my submodule.<name>.active branch, and I adjusted the
function in

  submodule: add submodule.<name>.active [1]

to add additional options passed through to `git config`.  You do have
to pick a default to use the extra options though.  If that becomes a
problem, I'd suggest extending git config itself to add a file above
or below the usual series of files.  Then get_submodule_config could
be

  git config --bottom-file .gitmodules submodule."$name"."$option"

or something, without needing a separate shell function.

On Mon, Dec 03, 2012 at 12:29:05PM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > As an example to make this clearer:
> >
> >   $ cat .gitmodules
> >   [submodule "sub1"]
> >     path = sub1
> >     url = git://example.com/sub1.git
> >     remote = remote1
> >     branch = branch1
> >     update-source = submodule-upstream
> >     update = rebase
> >   [submodule "sub2"]
> >   ...
> 
> Maybe it is just me but that "remote = remote1" sticks out like a
> sore thumb.
> 
> If you are showing the .gitmodules file to be shared as hints to
> project participants, why does it even need to have both URL and
> remote1?

The remote name will probably only ever get configured locally in
.git/config.  I put it in (as a separate patch) mostly because Phil
suggested something like it:

On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote:
> On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote:
> > I've always felt that the "origin" defaults are broken and are simply
> > being ignored because most users do not trip over them.  But ISTR that
> > submodule commands use the remote indicated by the superproject's
> > current remote-tracking configuration, with a fallback to 'origin' if
> > there is none.  Sort of a "best effort" algorithm, I think.  Am I
> > remembering that wrong?
>
> The current code uses a bare "git-fetch".  I'm not sure what that
> defaults to if you're on a detached head.  If it bothers you, I'm fine
> adding the submodule.<name>.remote option in v6.

and I hadn't heard any comments against it.  I'm not really attached
to that patch though, so feel free to leave it out (unless Phil chimes
in with stronger motivation?).

On Mon, Dec 03, 2012 at 12:29:05PM -0800, Junio C Hamano wrote:
> But I do not see any reason to require or even suggest any local
> nickname that is to be used to call the remote.  It really is a
> local matter.  Why should .gitmodules have "remote = ..." line?

The idea for configuring it at all probably goes something like “I
don't like where upstream (origin) is taking this submodule.  I want
to follow *my* upstream, but I've called it something besides origin.
Look, a submodule.<name>.remote option!  Now I don't have to rename
my-remote→origin→original-remote.”  I don't think this will come up
all that often.

> On the other hand, if you meant the above as an excerpt from
> $GIT_DIR/config, it also does not make sense.  At that point, the
> participant own the file and updating url to point at whatever
> different repository without changing the remote name is sufficient.

Unless they still want to keep an the origin remote to track the
original submodule upstream.  Maybe they'll want to switch back to
following that remote later.  As I hinted at above, if they have
remotes `alice`, `bob`, etc., it's easier to flip between them by
configuring submodule.<name>.remote

  $ git config submodule.submod.remote alice

than it is to reconfigure the submodule's origin:

  $ cd submod
  $ git remote rename origin charlie
  $ git remote rename alice origin

> It looks way over-engineered for unclear/dubious benefit.

I'm not going to push for submodule.<name>.remote.  Drop at will.

Cheers,
Trevor

[1]: https://github.com/wking/git/commit/fbe2d8419902700a6b0b40defaa5801811b887f7#L0R288

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH v2] submodule: add 'deinit' command

[Jens Lehmann]

With "git submodule init" the user is able to tell git he cares about one
or more submodules and wants to have it populated on the next call to "git
submodule update". But currently there is no easy way he could tell git he
does not care about a submodule anymore and wants to get rid of his local
work tree (except he knows a lot about submodule internals and removes the
"submodule.$name.url" setting from .git/config himself).

Help those users by providing a 'deinit' command. This removes the whole
submodule.<name> section from .git/config either for the given
submodule(s) or for all those which have been initialized if none were
given. Complain only when for a submodule given on the command line the
url setting can't be found in .git/config.

Add tests and link the man pages of "git submodule deinit" and "git rm"
to assist the user in deciding whether removing or unregistering the
submodule is the right thing to do for him.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---

Am 03.12.2012 08:58, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> Maybe the principle of least surprise is better followed when we
>> nuke the whole section, as it might surprise the user more to have
>> a setting resurrected he customized in the last life cycle of the
>> submodule than seeing that after an deinit followed by an init all
>> former customizations are consistently gone. So I tend to think now
>> that removing the whole section would be the better solution here.
> 
> I tend to agree; I suspect that a "deinit" would be mostly done
> either to
> 
>  (1) correct mistakes the user made during a recent "init" and
>      perhaps "sync"; or
> 
>  (2) tell Git that the user has finished woing with this particular
>      submodule and does not intend to use it for quite a while.
> 
> For both (1) and (2), I think it would be easier to users if we gave
> them a clean slate, the same state as the one the user who never had
> ran "init" on it would be in.  A user in situation (1) is asking for
> a clean slate, and a user in situation (2) is better served if he
> does not have to worry about leftover entries in $GIT_DIR/config he
> has long forgotten from many months ago (during which time the way
> the project uses the particular submodule may well have changed)
> giving non-standard experience different from what other project
> participants would get.

Changes in v2:
- Remove the whole submodule section instead of only removing the
  "url" setting and explain why we do that in a comment
- Reworded commit message and git-submodule.txt to reflect that
- Extend the test to check that a custom settings are removed


 Documentation/git-rm.txt        |  4 ++++
 Documentation/git-submodule.txt | 12 ++++++++++
 git-submodule.sh                | 52 ++++++++++++++++++++++++++++++++++++++++-
 t/t7400-submodule-basic.sh      | 12 ++++++++++
 4 files changed, 79 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt
index 262436b..ec42bf5 100644
--- a/Documentation/git-rm.txt
+++ b/Documentation/git-rm.txt
@@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree.
 Ignored files are deemed expendable and won't stop a submodule's work
 tree from being removed.

+If you only want to remove the local checkout of a submodule from your
+work tree without committing that use `git submodule deinit` instead
+(see linkgit:git-submodule[1]).
+
 EXAMPLES
 --------
 `git rm Documentation/\*.txt`::
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b1de3ba..08b55a7 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,6 +13,7 @@ SYNOPSIS
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
+'git submodule' [--quiet] deinit [--] [<path>...]
 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
@@ -134,6 +135,17 @@ init::
 	the explicit 'init' step if you do not intend to customize
 	any submodule locations.

+deinit::
+	Unregister the submodules, i.e. remove the whole `submodule.$name`
+	section from .git/config. Further calls to `git submodule update`,
+	`git submodule foreach` and `git submodule sync` will skip any
+	unregistered submodules until they are initialized again, so use
+	this command if you don't want to have a local checkout of the
+	submodule in your work tree anymore (but note that this command
+	does not remove the submodule work tree). If you really want to
+	remove a submodule from the repository and commit that use
+	linkgit:git-rm[1] instead.
+
 update::
 	Update the registered submodules, i.e. clone missing submodules and
 	checkout the commit specified in the index of the containing repository.
diff --git a/git-submodule.sh b/git-submodule.sh
index 2365149..3f558ed 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
+   or: $dashless [--quiet] deinit [--] [<path>...]
    or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
@@ -516,6 +517,55 @@ cmd_init()
 }

 #
+# Unregister submodules from .git/config
+#
+# $@ = requested paths (default to all)
+#
+cmd_deinit()
+{
+	# parse $args after "submodule ... init".
+	while test $# -ne 0
+	do
+		case "$1" in
+		-q|--quiet)
+			GIT_QUIET=1
+			;;
+		--)
+			shift
+			break
+			;;
+		-*)
+			usage
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+
+	module_list "$@" |
+	while read mode sha1 stage sm_path
+	do
+		die_if_unmatched "$mode"
+		name=$(module_name "$sm_path") || exit
+		url=$(git config submodule."$name".url)
+		if test -z "$url"
+		then
+			# Only mention uninitialized submodules when its
+			# path have been specified
+			test "$#" != "0" &&
+			say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")"
+			continue
+		fi
+		# Remove the whole section so we have a clean state when the user
+		# later decides to init this submodule again
+		git config --remove-section submodule."$name" &&
+		say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")"
+	done
+}
+
+#
 # Update each submodule path to correct revision, using clone and checkout as needed
 #
 # $@ = requested paths (default to all)
@@ -1108,7 +1158,7 @@ cmd_sync()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | foreach | init | update | status | summary | sync)
+	add | foreach | init | deinit | update | status | summary | sync)
 		command=$1
 		;;
 	-q|--quiet)
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index de7d453..ee4f0ab 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -756,4 +756,16 @@ test_expect_success 'submodule add with an existing name fails unless forced' '
 	)
 '

+test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' '
+	git config submodule.example.foo bar &&
+	git submodule deinit &&
+	test -z "$(git config submodule.example.url)" &&
+	test -z "$(git config submodule.example.foo)"
+'
+
+test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' '
+	git submodule deinit &&
+	test_must_fail git submodule deinit example
+'
+
 test_done
-- 
1.8.0.1.348.gc64da69

Re: [PATCH v2] submodule: add 'deinit' command

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> +If you only want to remove the local checkout of a submodule from your
> +work tree without committing that use `git submodule deinit` instead
> +(see linkgit:git-submodule[1]).

I'll add a comma between "without commiting that" and "use X
instead"; it will read better, I think.

> +test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' '
> +	git config submodule.example.foo bar &&
> +	git submodule deinit &&
> +	test -z "$(git config submodule.example.url)" &&
> +	test -z "$(git config submodule.example.foo)"
> +'

This is sufficient, but it might be cleaner to see if

    git config --get-regexp "^submodule\.example\."

results in empty.  Does not make much difference to warrant a re-roll.

> +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' '
> +	git submodule deinit &&
> +	test_must_fail git submodule deinit example
> +'
> +
>  test_done

Thanks; will queue.

[PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

I see that this series has dropped out of "what's cooking?".
Hopefully this reroll gets it back in ;).

Changes since v6 (both in response to Junio's comments):

* Fix style in get_submodule_config definition.
* Drop the submodule.<name>.remote config option (v6's patch 4).

W. Trevor King (3):
  submodule: add get_submodule_config helper funtion
  submodule update: add --remote for submodule's upstream changes
  submodule add: If --branch is given, record it in .gitmodules

 Documentation/config.txt        |  7 +++++-
 Documentation/git-submodule.txt | 27 ++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 ++++
 git-submodule.sh                | 52 ++++++++++++++++++++++++++++++++++++++++-
 t/t7400-submodule-basic.sh      |  1 +
 t/t7406-submodule-update.sh     | 31 ++++++++++++++++++++++++
 6 files changed, 120 insertions(+), 3 deletions(-)

-- 
1.8.0

[PATCH v7 1/3] submodule: add get_submodule_config helper funtion

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

Several submodule configuration variables
(e.g. fetchRecurseSubmodules) are read from .gitmodules with local
overrides from the usual git config files.  This shell function mimics
that logic to help initialize configuration variables in
git-submodule.sh.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 git-submodule.sh | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/git-submodule.sh b/git-submodule.sh
index ab6b110..f969f28 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -152,6 +152,32 @@ die_if_unmatched ()
 }
 
 #
+# Print a submodule configuration setting
+#
+# $1 = submodule name
+# $2 = option name
+# $3 = default value
+#
+# Checks in the usual git-config places first (for overrides),
+# otherwise it falls back on .gitmodules.  This allows you to
+# distribute project-wide defaults in .gitmodules, while still
+# customizing individual repositories if necessary.  If the option is
+# not in .gitmodules either, print a default value.
+#
+get_submodule_config () {
+	name="$1"
+	option="$2"
+	default="$3"
+	value=$(git config submodule."$name"."$option")
+	if test -z "$value"
+	then
+		value=$(git config -f .gitmodules submodule."$name"."$option")
+	fi
+	printf '%s' "${value:-$default}"
+}
+
+
+#
 # Map submodule path to submodule name
 #
 # $1 = path
-- 
1.8.0

[PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1).  Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1.  This helps you keep up with
changes in the upstream superproject.

However, it's also useful to stay up to date with changes in the
upstream subproject.  Previous workflows for incorporating such
changes include the ungainly:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates.  When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch.  If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`.  You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).

Previous use of submodule.<name>.branch
=======================================

Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option.  The foreach-pull
example above was described by Ævar in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.

Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same.  The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls.  The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`.  Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.

Implementation details
======================

In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1.  However, I didn't change the logic guarding the existing fetch:

  if test -z "$nofetch"
  then
    # Run fetch only if $sha1 isn't present or it
    # is not reachable from a ref.
    (clear_local_git_env; cd "$path" &&
      ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
       test -z "$rev") || git-fetch)) ||
    die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
  fi

There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository.  If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/config.txt        |  7 ++++++-
 Documentation/git-submodule.txt | 25 ++++++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 22 +++++++++++++++++++++-
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
 5 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 11f320b..6f4663c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1998,7 +1998,12 @@ submodule.<name>.update::
 	for a submodule.  These variables are initially populated
 	by 'git submodule init'; edit them to override the
 	URL and other values found in the `.gitmodules` file.  See
-	linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
+
+submodule.<name>.branch::
+	The remote branch name for a submodule, used by `git submodule
+	update --remote`.  Set this option to override the value found in
+	the `.gitmodules` file.  See linkgit:git-submodule[1] and
+	linkgit:gitmodules[5] for details.
 
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b4683bb..72dd52f 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,7 +13,7 @@ SYNOPSIS
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
-'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
+'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
@@ -236,6 +236,29 @@ OPTIONS
 	(the default). This limit only applies to modified submodules. The
 	size is always limited to 1 for added/deleted/typechanged submodules.
 
+--remote::
+	This option is only valid for the update command.  Instead of using
+	the superproject's recorded SHA-1 to update the submodule, use the
+	status of the submodule's remote tracking branch.  The remote used
+	is branch's remote (`branch.<name>.remote`), defaulting to `origin`.
+	The remote branch used defaults to `master`, but the branch name may
+	be overridden by setting the `submodule.<name>.branch` option in
+	either `.gitmodules` or `.git/config` (with `.git/config` taking
+	precedence).
++
+This works for any of the supported update procedures (`--checkout`,
+`--rebase`, etc.).  The only change is the source of the target SHA-1.
+For example, `submodule update --remote --merge` will merge upstream
+submodule changes into the submodules, while `submodule update
+--merge` will merge superproject gitlink changes into the submodules.
++
+In order to ensure a current tracking branch state, `update --remote`
+fetches the submodule's remote repository before calculating the
+SHA-1.  This makes `submodule update --remote --merge` similar to
+running `git pull` in the submodule.  If you don't want to fetch (for
+something closer to `git merge`), you should use `submodule update
+--remote --no-fetch --merge`.
+
 -N::
 --no-fetch::
 	This option is only valid for the update command.
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index 4effd78..4004fa6 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -47,6 +47,11 @@ submodule.<name>.update::
 	This config option is overridden if 'git submodule update' is given
 	the '--merge', '--rebase' or '--checkout' options.
 
+submodule.<name>.branch::
+	A remote branch name for tracking updates in the upstream submodule.
+	If the option is not specified, it defaults to 'master'.  See the
+	`--remote` documentation in linkgit:git-submodule[1] for details.
+
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
 	submodule. If this option is also present in the submodules entry in
diff --git a/git-submodule.sh b/git-submodule.sh
index f969f28..1395079 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
-   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+ges
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--] [<path>...]"
@@ -26,6 +27,7 @@ cached=
 recursive=
 init=
 files=
+remote=
 nofetch=
 update=
 prefix=
@@ -535,6 +537,9 @@ cmd_update()
 		-i|--init)
 			init=1
 			;;
+		--remote)
+			remote=1
+			;;
 		-N|--no-fetch)
 			nofetch=1
 			;;
@@ -595,6 +600,7 @@ cmd_update()
 		fi
 		name=$(module_name "$sm_path") || exit
 		url=$(git config submodule."$name".url)
+		branch=$(get_submodule_config "$name" branch master)
 		if ! test -z "$update"
 		then
 			update_module=$update
@@ -629,6 +635,20 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
 		fi
 
+		if test -n "$remote"
+		then
+			if test -z "$nofetch"
+			then
+				# Fetch remote before determining tracking $sha1
+				(clear_local_git_env; cd "$sm_path" && git-fetch) ||
+				die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
+			fi
+			remote_name=$(get_default_remote)
+			sha1=$(clear_local_git_env; cd "$sm_path" &&
+				git rev-parse --verify "${remote_name}/${branch}") ||
+			die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")"
+		fi
+
 		if test "$subsha1" != "$sha1" -o -n "$force"
 		then
 			subforce=$force
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 1542653..a567834 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' '
 	)
 '
 
+test_expect_success 'submodule update --remote should fetch upstream changes' '
+	(cd submodule &&
+	 echo line4 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line4"
+	) &&
+	(cd super &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)"
+	)
+'
+
+test_expect_success 'local config should override .gitmodules branch' '
+	(cd submodule &&
+	 git checkout -b test-branch &&
+	 echo line5 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line5" &&
+	 git checkout master
+	) &&
+	(cd super &&
+	 git config submodule.submodule.branch test-branch &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)"
+	)
+'
+
 test_expect_success 'submodule update --rebase staying on master' '
 	(cd super/submodule &&
 	  git checkout master
-- 
1.8.0

[PATCH v7 3/3] submodule add: If --branch is given, record it in .gitmodules

[W. Trevor King]

From: "W. Trevor King" <wking@tremily.us>

This allows you to easily record a submodule.<name>.branch option in
.gitmodules when you add a new submodule.  With this patch,

  $ git submodule add -b <branch> <repository> [<path>]
  $ git config -f .gitmodules submodule.<path>.branch <branch>

reduces to

  $ git submodule add -b <branch> <repository> [<path>]

This means that future calls to

  $ git submodule update --remote ...

will get updates from the same branch that you used to initialize the
submodule, which is usually what you want.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 2 ++
 git-submodule.sh                | 4 ++++
 t/t7400-submodule-basic.sh      | 1 +
 3 files changed, 7 insertions(+)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 72dd52f..988bba9 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -208,6 +208,8 @@ OPTIONS
 -b::
 --branch::
 	Branch of repository to add as submodule.
+	The name of the branch is recorded as `submodule.<path>.branch` in
+	`.gitmodules` for `update --remote`.
 
 -f::
 --force::
diff --git a/git-submodule.sh b/git-submodule.sh
index 1395079..9f3f437 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -394,6 +394,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
+	if test -n "$branch"
+	then
+		git config -f .gitmodules submodule."$sm_path".branch "$branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5397037..90e2915 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test "initial" = "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
 		git submodule init
 	) &&
 
-- 
1.8.0

Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes

[Junio C Hamano]

What branch did you base this series on?

The preimage of git-submodule.sh in [2/3] does not seem to match
anything I have (I could wiggle the patch, but in general I would
rather prefer not having to).

Re: [PATCH v2] submodule: add 'deinit' command

[Michael J Gruber]

Jens Lehmann venit, vidit, dixit 04.12.2012 22:48:
> With "git submodule init" the user is able to tell git he cares about one
> or more submodules and wants to have it populated on the next call to "git
> submodule update". But currently there is no easy way he could tell git he
> does not care about a submodule anymore and wants to get rid of his local
> work tree (except he knows a lot about submodule internals and removes the
> "submodule.$name.url" setting from .git/config himself).
> 
> Help those users by providing a 'deinit' command. This removes the whole
> submodule.<name> section from .git/config either for the given
> submodule(s) or for all those which have been initialized if none were
> given. Complain only when for a submodule given on the command line the
> url setting can't be found in .git/config.

Whoaaa, so why not have "git rm" remove everything unless I specify a
file to be removed?

I know I'm exaggerating a bit, but defaulting to "--all" for a
destructive operation seems to be a bit harsh, especially when the
command is targeted at "those" users that you mention.

> Add tests and link the man pages of "git submodule deinit" and "git rm"
> to assist the user in deciding whether removing or unregistering the
> submodule is the right thing to do for him.
> 
> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
> ---
> 
> Am 03.12.2012 08:58, schrieb Junio C Hamano:
>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>
>>> Maybe the principle of least surprise is better followed when we
>>> nuke the whole section, as it might surprise the user more to have
>>> a setting resurrected he customized in the last life cycle of the
>>> submodule than seeing that after an deinit followed by an init all
>>> former customizations are consistently gone. So I tend to think now
>>> that removing the whole section would be the better solution here.
>>
>> I tend to agree; I suspect that a "deinit" would be mostly done
>> either to
>>
>>  (1) correct mistakes the user made during a recent "init" and
>>      perhaps "sync"; or
>>
>>  (2) tell Git that the user has finished woing with this particular
>>      submodule and does not intend to use it for quite a while.
>>
>> For both (1) and (2), I think it would be easier to users if we gave
>> them a clean slate, the same state as the one the user who never had
>> ran "init" on it would be in.  A user in situation (1) is asking for
>> a clean slate, and a user in situation (2) is better served if he
>> does not have to worry about leftover entries in $GIT_DIR/config he
>> has long forgotten from many months ago (during which time the way
>> the project uses the particular submodule may well have changed)
>> giving non-standard experience different from what other project
>> participants would get.
> 
> Changes in v2:
> - Remove the whole submodule section instead of only removing the
>   "url" setting and explain why we do that in a comment
> - Reworded commit message and git-submodule.txt to reflect that
> - Extend the test to check that a custom settings are removed
> 
> 
>  Documentation/git-rm.txt        |  4 ++++
>  Documentation/git-submodule.txt | 12 ++++++++++
>  git-submodule.sh                | 52 ++++++++++++++++++++++++++++++++++++++++-
>  t/t7400-submodule-basic.sh      | 12 ++++++++++
>  4 files changed, 79 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt
> index 262436b..ec42bf5 100644
> --- a/Documentation/git-rm.txt
> +++ b/Documentation/git-rm.txt
> @@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree.
>  Ignored files are deemed expendable and won't stop a submodule's work
>  tree from being removed.
> 
> +If you only want to remove the local checkout of a submodule from your
> +work tree without committing that use `git submodule deinit` instead
> +(see linkgit:git-submodule[1]).
> +
>  EXAMPLES
>  --------
>  `git rm Documentation/\*.txt`::
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index b1de3ba..08b55a7 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -13,6 +13,7 @@ SYNOPSIS
>  	      [--reference <repository>] [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
> +'git submodule' [--quiet] deinit [--] [<path>...]
>  'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
>  	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
> @@ -134,6 +135,17 @@ init::
>  	the explicit 'init' step if you do not intend to customize
>  	any submodule locations.
> 
> +deinit::
> +	Unregister the submodules, i.e. remove the whole `submodule.$name`
> +	section from .git/config. Further calls to `git submodule update`,
> +	`git submodule foreach` and `git submodule sync` will skip any
> +	unregistered submodules until they are initialized again, so use
> +	this command if you don't want to have a local checkout of the
> +	submodule in your work tree anymore (but note that this command
> +	does not remove the submodule work tree). If you really want to
> +	remove a submodule from the repository and commit that use
> +	linkgit:git-rm[1] instead.
> +
>  update::
>  	Update the registered submodules, i.e. clone missing submodules and
>  	checkout the commit specified in the index of the containing repository.
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 2365149..3f558ed 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
>  USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
>     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] init [--] [<path>...]
> +   or: $dashless [--quiet] deinit [--] [<path>...]
>     or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
>     or: $dashless [--quiet] foreach [--recursive] <command>
> @@ -516,6 +517,55 @@ cmd_init()
>  }
> 
>  #
> +# Unregister submodules from .git/config
> +#
> +# $@ = requested paths (default to all)
> +#
> +cmd_deinit()
> +{
> +	# parse $args after "submodule ... init".
> +	while test $# -ne 0
> +	do
> +		case "$1" in
> +		-q|--quiet)
> +			GIT_QUIET=1
> +			;;
> +		--)
> +			shift
> +			break
> +			;;
> +		-*)
> +			usage
> +			;;
> +		*)
> +			break
> +			;;
> +		esac
> +		shift
> +	done
> +
> +	module_list "$@" |
> +	while read mode sha1 stage sm_path
> +	do
> +		die_if_unmatched "$mode"
> +		name=$(module_name "$sm_path") || exit
> +		url=$(git config submodule."$name".url)
> +		if test -z "$url"
> +		then
> +			# Only mention uninitialized submodules when its
> +			# path have been specified
> +			test "$#" != "0" &&
> +			say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")"
> +			continue
> +		fi
> +		# Remove the whole section so we have a clean state when the user
> +		# later decides to init this submodule again
> +		git config --remove-section submodule."$name" &&
> +		say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")"
> +	done
> +}
> +
> +#
>  # Update each submodule path to correct revision, using clone and checkout as needed
>  #
>  # $@ = requested paths (default to all)
> @@ -1108,7 +1158,7 @@ cmd_sync()
>  while test $# != 0 && test -z "$command"
>  do
>  	case "$1" in
> -	add | foreach | init | update | status | summary | sync)
> +	add | foreach | init | deinit | update | status | summary | sync)
>  		command=$1
>  		;;
>  	-q|--quiet)
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index de7d453..ee4f0ab 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -756,4 +756,16 @@ test_expect_success 'submodule add with an existing name fails unless forced' '
>  	)
>  '
> 
> +test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' '
> +	git config submodule.example.foo bar &&
> +	git submodule deinit &&
> +	test -z "$(git config submodule.example.url)" &&
> +	test -z "$(git config submodule.example.foo)"
> +'
> +
> +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' '
> +	git submodule deinit &&
> +	test_must_fail git submodule deinit example
> +'
> +
>  test_done
> 

Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 789 bytes --]

On Tue, Dec 11, 2012 at 09:42:48PM -0800, Junio C Hamano wrote:
> What branch did you base this series on?

Every version of this series has been based on v1.8.0.

> The preimage of git-submodule.sh in [2/3] does not seem to match
> anything I have (I could wiggle the patch, but in general I would
> rather prefer not having to).

From patch 1/3:

  diff --git a/git-submodule.sh b/git-submodule.sh
  index ab6b110..f969f28 100755

And from patch 2/3:

  diff --git a/git-submodule.sh b/git-submodule.sh
  index f969f28..1395079 100755

ab6b110 is in v1.8.0:

  $ git ls-tree v1.8.0 git-submodule.sh
  100755 blob ab6b1107b6090494f192f361471ed5748ffa7dc1    git-submodule.sh

I can reroll if necessary, but I'm not sure what I've done wrong…

Cheers,
Trevor

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] submodule: add 'deinit' command

[Jens Lehmann]

Am 12.12.2012 16:08, schrieb Michael J Gruber:
> Jens Lehmann venit, vidit, dixit 04.12.2012 22:48:
>> With "git submodule init" the user is able to tell git he cares about one
>> or more submodules and wants to have it populated on the next call to "git
>> submodule update". But currently there is no easy way he could tell git he
>> does not care about a submodule anymore and wants to get rid of his local
>> work tree (except he knows a lot about submodule internals and removes the
>> "submodule.$name.url" setting from .git/config himself).
>>
>> Help those users by providing a 'deinit' command. This removes the whole
>> submodule.<name> section from .git/config either for the given
>> submodule(s) or for all those which have been initialized if none were
>> given. Complain only when for a submodule given on the command line the
>> url setting can't be found in .git/config.
> 
> Whoaaa, so why not have "git rm" remove everything unless I specify a
> file to be removed?

Because "git add" doesn't add any file in that case either?

> I know I'm exaggerating a bit, but defaulting to "--all" for a
> destructive operation seems to be a bit harsh, especially when the
> command is targeted at "those" users that you mention.

All other submodule commands (except add, which only operates on a
single submodule to be) iterate over all submodules if none were
explicitly given on the command line. So I made deinit just behave
like all the others - and especially init - do. But if people really
are surprised by being consistent here I won't argue against adding
such a "--all" option, but currently I'm not convinced it is worth
it. Especially as I suspect the number of submodule users having
customized those in .git/config is not that high ...

Re: [PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes

[Phil Hord]

Thanks for looking after this.


On Tue, Dec 11, 2012 at 1:58 PM, W. Trevor King <wking@tremily.us> wrote:
> From: "W. Trevor King" <wking@tremily.us>
>
> The current `update` command incorporates the superproject's gitlinked
> SHA-1 ($sha1) into the submodule HEAD ($subsha1).  Depending on the
> options you use, it may checkout $sha1, rebase the $subsha1 onto
> $sha1, or merge $sha1 into $subsha1.  This helps you keep up with
> changes in the upstream superproject.
>
> However, it's also useful to stay up to date with changes in the
> upstream subproject.  Previous workflows for incorporating such
> changes include the ungainly:
>
>   $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
>
> With this patch, all of the useful functionality for incorporating
> superproject changes can be reused to incorporate upstream subproject
> updates.  When you specify --remote, the target $sha1 is replaced with
> a $sha1 of the submodule's origin/master tracking branch.  If you want
> to merge a different tracking branch, you can configure the
> `submodule.<name>.branch` option in `.gitmodules`.  You can override
> the `.gitmodules` configuration setting for a particular superproject
> by configuring the option in that superproject's default configuration
> (using the usual configuration hierarchy, e.g. `.git/config`,
> `~/.gitconfig`, etc.).
>
> Previous use of submodule.<name>.branch
> =======================================
>
> Because we're adding a new configuration option, it's a good idea to
> check if anyone else is already using the option.  The foreach-pull
> example above was described by Ævar in
>
>   commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
>   Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>   Date:   Fri May 21 16:10:10 2010 +0000
>
>     git-submodule foreach: Add $toplevel variable
>
> Gerrit uses the same interpretation for the setting, but because
> Gerrit has direct access to the subproject repositories, it updates
> the superproject repositories automatically when a subproject changes.
> Gerrit also accepts the special value '.', which it expands into the
> superproject's branch name.
>
> Although the --remote functionality is using `submodule.<name>.branch`
> slightly differently, the effect is the same.  The foreach-pull
> example uses the option to record the name of the local branch to
> checkout before pulls.  The tracking branch to be pulled is recorded
> in `.git/modules/<name>/config`, which was initialized by the module
> clone during `submodule add` or `submodule init`.  Because the branch
> name stored in `submodule.<name>.branch` was likely the same as the
> branch name used during the initial `submodule add`, the same branch
> will be pulled in each workflow.
>
> Implementation details
> ======================
>
> In order to ensure a current tracking branch state, `update --remote`
> fetches the submodule's remote repository before calculating the
> SHA-1.  However, I didn't change the logic guarding the existing fetch:
>
>   if test -z "$nofetch"
>   then
>     # Run fetch only if $sha1 isn't present or it
>     # is not reachable from a ref.
>     (clear_local_git_env; cd "$path" &&
>       ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
>        test -z "$rev") || git-fetch)) ||
>     die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
>   fi
>
> There will not be a double-fetch, because the new $sha1 determined
> after the `--remote` triggered fetch should always exist in the
> repository.  If it doesn't, it's because some racy process removed it
> from the submodule's repository and we *should* be re-fetching.
>
> Signed-off-by: W. Trevor King <wking@tremily.us>
> ---
>  Documentation/config.txt        |  7 ++++++-
>  Documentation/git-submodule.txt | 25 ++++++++++++++++++++++++-
>  Documentation/gitmodules.txt    |  5 +++++
>  git-submodule.sh                | 22 +++++++++++++++++++++-
>  t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
>  5 files changed, 87 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 11f320b..6f4663c 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1998,7 +1998,12 @@ submodule.<name>.update::
>         for a submodule.  These variables are initially populated
>         by 'git submodule init'; edit them to override the
>         URL and other values found in the `.gitmodules` file.  See
> -       linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
> +
> +submodule.<name>.branch::
> +       The remote branch name for a submodule, used by `git submodule
> +       update --remote`.  Set this option to override the value found in
> +       the `.gitmodules` file.  See linkgit:git-submodule[1] and
> +       linkgit:gitmodules[5] for details.
>
>  submodule.<name>.fetchRecurseSubmodules::
>         This option can be used to control recursive fetching of this
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index b4683bb..72dd52f 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -13,7 +13,7 @@ SYNOPSIS
>               [--reference <repository>] [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
> -'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
> +'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase]
>               [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
>               [commit] [--] [<path>...]
> @@ -236,6 +236,29 @@ OPTIONS
>         (the default). This limit only applies to modified submodules. The
>         size is always limited to 1 for added/deleted/typechanged submodules.
>
> +--remote::
> +       This option is only valid for the update command.  Instead of using
> +       the superproject's recorded SHA-1 to update the submodule, use the
> +       status of the submodule's remote tracking branch.  The remote used
> +       is branch's remote (`branch.<name>.remote`), defaulting to `origin`.
> +       The remote branch used defaults to `master`, but the branch name may
> +       be overridden by setting the `submodule.<name>.branch` option in
> +       either `.gitmodules` or `.git/config` (with `.git/config` taking
> +       precedence).
> ++
> +This works for any of the supported update procedures (`--checkout`,
> +`--rebase`, etc.).  The only change is the source of the target SHA-1.
> +For example, `submodule update --remote --merge` will merge upstream
> +submodule changes into the submodules, while `submodule update
> +--merge` will merge superproject gitlink changes into the submodules.
> ++
> +In order to ensure a current tracking branch state, `update --remote`
> +fetches the submodule's remote repository before calculating the
> +SHA-1.  This makes `submodule update --remote --merge` similar to
> +running `git pull` in the submodule.  If you don't want to fetch (for
> +something closer to `git merge`), you should use `submodule update
> +--remote --no-fetch --merge`.

I assume the same can be said for 'submodue update --remote --rebase',
right?  I wonder if this can be made merge/rebase-agnostic.  Is it
still true if I word it like this?:

   In order to ensure a current tracking branch state, `update --remote`
   fetches the submodule's remote repository before calculating the
   SHA-1.  If you don't want to fetch, you should use `submodule update
    --remote --no-fetch`.


> +
>  -N::
>  --no-fetch::
>         This option is only valid for the update command.
> diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
> index 4effd78..4004fa6 100644
> --- a/Documentation/gitmodules.txt
> +++ b/Documentation/gitmodules.txt
> @@ -47,6 +47,11 @@ submodule.<name>.update::
>         This config option is overridden if 'git submodule update' is given
>         the '--merge', '--rebase' or '--checkout' options.
>
> +submodule.<name>.branch::
> +       A remote branch name for tracking updates in the upstream submodule.
> +       If the option is not specified, it defaults to 'master'.  See the
> +       `--remote` documentation in linkgit:git-submodule[1] for details.
> +
>  submodule.<name>.fetchRecurseSubmodules::
>         This option can be used to control recursive fetching of this
>         submodule. If this option is also present in the submodules entry in
> diff --git a/git-submodule.sh b/git-submodule.sh
> index f969f28..1395079 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
>  USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
>     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] init [--] [<path>...]
> -   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> +   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> +ges

I think there's an unintentionally added line here with "ges".

>     or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
>     or: $dashless [--quiet] foreach [--recursive] <command>
>     or: $dashless [--quiet] sync [--] [<path>...]"
> @@ -26,6 +27,7 @@ cached=
>  recursive=
>  init=
>  files=
> +remote=
>  nofetch=
>  update=
>  prefix=
> @@ -535,6 +537,9 @@ cmd_update()
>                 -i|--init)
>                         init=1
>                         ;;
> +               --remote)
> +                       remote=1
> +                       ;;
>                 -N|--no-fetch)
>                         nofetch=1
>                         ;;
> @@ -595,6 +600,7 @@ cmd_update()
>                 fi
>                 name=$(module_name "$sm_path") || exit
>                 url=$(git config submodule."$name".url)
> +               branch=$(get_submodule_config "$name" branch master)
>                 if ! test -z "$update"
>                 then
>                         update_module=$update
> @@ -629,6 +635,20 @@ Maybe you want to use 'update --init'?")"
>                         die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
>                 fi
>
> +               if test -n "$remote"
> +               then
> +                       if test -z "$nofetch"
> +                       then
> +                               # Fetch remote before determining tracking $sha1
> +                               (clear_local_git_env; cd "$sm_path" && git-fetch) ||

You should 'git fetch $remote_name' here, and of course, initialize
remote_name before this.  But how can we know the remote_name in the
first place?  Is it safe to assume the submodule remote names will
match those in the superproject?

> +                               die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
> +                       fi
> +                       remote_name=$(get_default_remote)

This get_default_remote finds the remote for the remote-tracking
branch for HEAD in the superproject.  It is possible that HEAD !=
$branch, so we have very few clues to go on here to get a more
reasonable answer, so I do not have any good suggestions to improve
this.

One option would be to find the remote given for
submodule."$branch".merge, but this would suppose there is some
remote-tracking branch configured in the submodule, and that is not
likely to be the case.

> +                       sha1=$(clear_local_git_env; cd "$sm_path" &&
> +                               git rev-parse --verify "${remote_name}/${branch}") ||

This does assume the submodule remote names will match those in the
superproject.  Is this safe?

> +                       die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")"
> +               fi
> +
>                 if test "$subsha1" != "$sha1" -o -n "$force"
>                 then
>                         subforce=$force
> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
> index 1542653..a567834 100755
> --- a/t/t7406-submodule-update.sh
> +++ b/t/t7406-submodule-update.sh
> @@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' '
>         )
>  '
>
> +test_expect_success 'submodule update --remote should fetch upstream changes' '
> +       (cd submodule &&
> +        echo line4 >> file &&
> +        git add file &&
> +        test_tick &&
> +        git commit -m "upstream line4"
> +       ) &&
> +       (cd super &&
> +        git submodule update --remote --force submodule &&
> +        cd submodule &&
> +        test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)"
> +       )
> +'
> +
> +test_expect_success 'local config should override .gitmodules branch' '
> +       (cd submodule &&
> +        git checkout -b test-branch &&
> +        echo line5 >> file &&
> +        git add file &&
> +        test_tick &&
> +        git commit -m "upstream line5" &&
> +        git checkout master
> +       ) &&
> +       (cd super &&
> +        git config submodule.submodule.branch test-branch &&
> +        git submodule update --remote --force submodule &&
> +        cd submodule &&
> +        test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)"
> +       )
> +'
> +
>  test_expect_success 'submodule update --rebase staying on master' '
>         (cd super/submodule &&
>           git checkout master
> --
> 1.8.0
>

Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> On Tue, Dec 11, 2012 at 09:42:48PM -0800, Junio C Hamano wrote:
>> What branch did you base this series on?
>
> Every version of this series has been based on v1.8.0.

Thanks.

There were quite a few changes to git-submodule.sh since then to
'master' and I had to either wiggle the patch or know which exact
one 1/3 needs to be applied to in order to allow 2/3 to apply (try
applying these three to 'master' yourself---you will likely to see
that 2/3 will stop with conflicts).

In any case, I ended up applying them by editing the patches, and I
should have a good copy in 'pu'.  Please double check the result.

Thanks.

Re: [PATCH v2] submodule: add 'deinit' command

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> Especially as I suspect the number of submodule users having
> customized those in .git/config is not that high ...

I thought the point of "deinit" was to say "I am not interested in
having a checkout of these submodules in my working tree anymore".
The user could do "rm -fr submodule && mkdir submodule" to remove it
locally and keep "diff" and "status" from noticing the removal, but
the primary reason the user needs an explicit "deinit" is because
many subcommands of "git submodule" are documented to operate on all
submodules that have been "init"ed when given no explicit submodule
names [*1*].

Your "deinit" is documented not to actually remove the submodule
checkout, but that very much goes against my intuition.  What is the
justification behind that choice?  "We'll remove the configuration,
you remove the tree yourself" will invite the mistake of running
"git rm" on it, which you wanted to avoid with the addition to the
"git rm" documentation, no?


[Footnote]

*1* In reality, the code looks at presense of .git in the submodule
path to decide if it has been "init"ed (cf. cmd_update), but this
implementation of "deinit" does not seem to cause that .git to be
removed, leaving the submodule in "init"ed state from these other
command's perspective.

Re: [PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes

[Junio C Hamano]

Phil Hord <phil.hord@gmail.com> writes:

>> +               if test -n "$remote"
>> +               then
>> +                       if test -z "$nofetch"
>> +                       then
>> +                               # Fetch remote before determining tracking $sha1
>> +                               (clear_local_git_env; cd "$sm_path" && git-fetch) ||
>
> You should 'git fetch $remote_name' here, and of course, initialize
> remote_name before this.  But how can we know the remote_name in the
> first place?  Is it safe to assume the submodule remote names will
> match those in the superproject?
>
>> +                               die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
>> +                       fi
>> +                       remote_name=$(get_default_remote)
>
> This get_default_remote finds the remote for the remote-tracking
> branch for HEAD in the superproject.  It is possible that HEAD !=
> $branch, so we have very few clues to go on here to get a more
> reasonable answer, so I do not have any good suggestions to improve
> this.
>
> One option would be to find the remote given for
> submodule."$branch".merge, but this would suppose there is some
> remote-tracking branch configured in the submodule, and that is not
> likely to be the case.
>
>> +                       sha1=$(clear_local_git_env; cd "$sm_path" &&
>> +                               git rev-parse --verify "${remote_name}/${branch}") ||
>
> This does assume the submodule remote names will match those in the
> superproject.  Is this safe?

All good points.  Thanks for reviewing.

Re: [PATCH v2] submodule: add 'deinit' command

[Jens Lehmann]

Am 12.12.2012 20:32, schrieb Junio C Hamano:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> Especially as I suspect the number of submodule users having
>> customized those in .git/config is not that high ...
> 
> I thought the point of "deinit" was to say "I am not interested in
> having a checkout of these submodules in my working tree anymore".

Yes. (But I'm not sure users expect that command to also remove
the work tree)

> The user could do "rm -fr submodule && mkdir submodule" to remove it
> locally and keep "diff" and "status" from noticing the removal, but
> the primary reason the user needs an explicit "deinit" is because
> many subcommands of "git submodule" are documented to operate on all
> submodules that have been "init"ed when given no explicit submodule
> names [*1*].

The real reason we need deinit is that the next run of "submodule
update" will otherwise happily recreate the submodule checkout you
just removed as long as it finds the url setting in .git/config.

> Your "deinit" is documented not to actually remove the submodule
> checkout, but that very much goes against my intuition.  What is the
> justification behind that choice?

I thought it should match what "submodule init" does, which is to do
nothing to the work tree until the next "submodule update" is run.
(But I agree that analogy is somewhat flawed until we teach "update"
to remove a deinitialized submodule - or maybe teach it the --deinit
option which could do both). On the other hand with current git
submodule work trees always stay around anyway until you remove them
by hand (e.g. when you switch to a branch that doesn't have it), so
I'm not sure what would surprise people more here. So I just left
the work tree unchanged.

> "We'll remove the configuration,
> you remove the tree yourself" will invite the mistake of running
> "git rm" on it, which you wanted to avoid with the addition to the
> "git rm" documentation, no?

I think that'll happen only if git would remind them that they
still have a populated work tree, which I believe it shouldn't.

> [Footnote]
> 
> *1* In reality, the code looks at presense of .git in the submodule
> path to decide if it has been "init"ed (cf. cmd_update), but this
> implementation of "deinit" does not seem to cause that .git to be
> removed, leaving the submodule in "init"ed state from these other
> command's perspective.

Nope, cmd_update() checks first if the url is found in .git/config
and skips the submodule if not. I rechecked and only "summary" and
"foreach" still recurse into a deinitialized submodule, which they
shouldn't. But a quick test shows that "git status" and "git diff"
also still inspect a deinitialized submodule, so there's some work
left to do to handle the case where the work tree is not removed.

So unless people agree that deinit should also remove the work
tree I'll prepare some patches teaching all git commands to
consistently ignore deinitialized submodules. Opinions?

Re: [PATCH v2] submodule: add 'deinit' command

[Junio C Hamano]

Jens Lehmann <Jens.Lehmann@web.de> writes:

> So unless people agree that deinit should also remove the work
> tree I'll prepare some patches teaching all git commands to
> consistently ignore deinitialized submodules. Opinions?

While I agree that consistency is good, "deinit" that does not
remove the working tree of the submodule the user explicitly said he
no longer wants to have the checkout for is a bug, and I think these
two are orthogonal issues.

In other words, "Ignore deinitialized submodules even when an
earlier bug in deinit failed to remove the working tree" is a
robustness issue for the other recursing commands, not an excuse for
"deinit" to have such a bug in the first place, I think.

Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 839 bytes --]

On Wed, Dec 12, 2012 at 10:19:32AM -0800, Junio C Hamano wrote:
> In any case, I ended up applying them by editing the patches, and I
> should have a good copy in 'pu'.  Please double check the result.

Your 'pu' branch looks good to me.  Most of the differences with my
initial patch are due to irrelevant context lines.  I would change
patch 3 (commit 2f507f9a in 'pu') to use

  git config -f .gitmodules submodule."$sm_name".branch "$branch"
                                           ^
instead of

  git config -f .gitmodules submodule."$sm_path".branch "$branch"
                                           ^
to match the nearby changes from 73b0898d.

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 4633 bytes --]

On Wed, Dec 12, 2012 at 12:43:23PM -0500, Phil Hord wrote:
> On Tue, Dec 11, 2012 at 1:58 PM, W. Trevor King <wking@tremily.us> wrote:
> > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> > …
> > +--remote::
> > [snip some --remote documentation]
> > +In order to ensure a current tracking branch state, `update --remote`
> > +fetches the submodule's remote repository before calculating the
> > +SHA-1.  This makes `submodule update --remote --merge` similar to
> > +running `git pull` in the submodule.  If you don't want to fetch (for
> > +something closer to `git merge`), you should use `submodule update
> > +--remote --no-fetch --merge`.
> 
> I assume the same can be said for 'submodue update --remote --rebase',
> right?

Yes.

> I wonder if this can be made merge/rebase-agnostic.  Is it still
> true if I word it like this?:
> 
>    In order to ensure a current tracking branch state, `update --remote`
>    fetches the submodule's remote repository before calculating the
>    SHA-1.  If you don't want to fetch, you should use `submodule update
>     --remote --no-fetch`.

Works for me.  Will change in v8 (which I'll base on 'master').

> > diff --git a/git-submodule.sh b/git-submodule.sh
> > index f969f28..1395079 100755
> > --- a/git-submodule.sh
> > +++ b/git-submodule.sh
> > @@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
> >  USAGE="[--quiet] add [-b branch] [-f|--force] [--reference <repository>] [--] <repository> [<path>]
> >     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
> >     or: $dashless [--quiet] init [--] [<path>...]
> > -   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> > +   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> > +ges
> 
> I think there's an unintentionally added line here with "ges".

That is embarrassing :p.  Will fix in v8.

> > +               if test -n "$remote"
> > +               then
> > +                       if test -z "$nofetch"
> > +                       then
> > +                               # Fetch remote before determining tracking $sha1
> > +                               (clear_local_git_env; cd "$sm_path" && git-fetch) ||
> 
> You should 'git fetch $remote_name' here, and of course, initialize
> remote_name before this.  But how can we know the remote_name in the
> first place?  Is it safe to assume the submodule remote names will
> match those in the superproject?

The other git-fetch call from git-submodule.sh is also bare (i.e. no
specified remote).  When the remote needs to be specified, other
portions of git-submodule.sh use $(get_default_remote), which is (I
think) what the user should expect.  v6 of this series had a
configurable remote name, but Junio wasn't keen on the additional
configuration option.  I don't really mind either way.

> 
> > +                               die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
> > +                       fi
> > +                       remote_name=$(get_default_remote)
> 
> This get_default_remote finds the remote for the remote-tracking
> branch for HEAD in the superproject.  It is possible that HEAD !=
> $branch, so we have very few clues to go on here to get a more
> reasonable answer, so I do not have any good suggestions to improve
> this.

For detached HEADs, get_default_remote should fall back to 'origin',
which seems sane.  If the user wants a different default, they've
likely checkout out a branch in the submodule, setup that branch's
remote, and will be using --merge or --rebase.  If anyone expects
users who will be using detached heads to *want* to specify a
different remote than 'origin', that would be a good argument for
reinstating my submodule.<name>.remote patch from v6.

> > +                       sha1=$(clear_local_git_env; cd "$sm_path" &&
> > +                               git rev-parse --verify "${remote_name}/${branch}") ||
> 
> This does assume the submodule remote names will match those in the
> superproject.  Is this safe?

Another good catch.  I should be calling get_default_remote after
cd-ing into the submodule.  Will change in v8.

Thanks for the feedback :)
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] submodule: add 'deinit' command

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

On Wed, Dec 12, 2012 at 02:34:47PM -0800, Junio C Hamano wrote:
> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
> > So unless people agree that deinit should also remove the work
> > tree I'll prepare some patches teaching all git commands to
> > consistently ignore deinitialized submodules. Opinions?
> 
> While I agree that consistency is good, "deinit" that does not
> remove the working tree of the submodule the user explicitly said he
> no longer wants to have the checkout for is a bug, and I think these
> two are orthogonal issues.

Should `deinit` remove the submodule checkout, replace it with the
original gitlink, and clear the .git/config information then?  That
would restore the user to the state they'd be in if they were never
interested in the submodule.

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] submodule: add 'deinit' command

[Junio C Hamano]

"W. Trevor King" <wking@tremily.us> writes:

> Should `deinit` remove the submodule checkout, replace it with the
> original gitlink, and clear the .git/config information then?  That
> would restore the user to the state they'd be in if they were never
> interested in the submodule.

AFAIU, "restore the user to the state" is the goal.  I am not sure
what you meant by "replace it with the original gitlink", though.  A
checkout with a submodule that the user is not interested in would
have an empty directory at that path, no?

Re: [PATCH v2] submodule: add 'deinit' command

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]

On Wed, Dec 12, 2012 at 03:35:59PM -0800, Junio C Hamano wrote:
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > Should `deinit` remove the submodule checkout, replace it with the
> > original gitlink, and clear the .git/config information then?  That
> > would restore the user to the state they'd be in if they were never
> > interested in the submodule.
> 
> AFAIU, "restore the user to the state" is the goal.  I am not sure
> what you meant by "replace it with the original gitlink", though.  A
> checkout with a submodule that the user is not interested in would
> have an empty directory at that path, no?

Ah yes, the gitlink is only in the index.  Sorry for the noise.

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2] submodule: add 'deinit' command

[Marc Branchaud]

On 12-12-12 05:25 PM, Jens Lehmann wrote:
> 
> So unless people agree that deinit should also remove the work
> tree I'll prepare some patches teaching all git commands to
> consistently ignore deinitialized submodules. Opinions?

I agree with Trevor's suggestion that deinit should restore the user to the
state he would be in if he were never interested in the submodule.  So clean
up .git/config and remove the work tree.  (Maybe just issue a warning instead
if the submodule's work tree is dirty.)

Also, given that semantic, I agree with Michael that a bare "git submodule
deinit" should *not* deinitialize all the submodules.  It should require a
"--all" for that.  The bare command should just print a usage summary.

		M.

[PATCH v8 0/3] submodule update: add --remote for submodule's upstream changes

[wking]

From: "W. Trevor King" <wking@tremily.us>

Comments on v7 seem to have petered out, so here's v8.  Changes since
v7:

* Series based on gitster/master instead of v1.8.0.
* In Documentation/config.txt, restored trailing line of
  submodule.<name>.update documentation, which I had accidentally
  removed in v7.
* In Documentation/git-submodule.txt, make --no-fetch example in the
  --remote description more general, following Phil's suggestion.
* In git-submodule.sh:
  * Remove accidental "ges" line.
  * Use the submodule's default remote to determine which tracking
    branch to fetch.  In v7 I'd been using the superproject's default
    remote.
  * In cmd_add(), use sm_name instead of sm_path to store the --branch
    option (catching up with 73b0898).

W. Trevor King (3):
  submodule: add get_submodule_config helper funtion
  submodule update: add --remote for submodule's upstream changes
  submodule add: If --branch is given, record it in .gitmodules

 Documentation/config.txt        |  6 +++++
 Documentation/git-submodule.txt | 25 +++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 ++++
 git-submodule.sh                | 51 ++++++++++++++++++++++++++++++++++++++++-
 t/t7400-submodule-basic.sh      |  1 +
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++
 6 files changed, 117 insertions(+), 2 deletions(-)

-- 
1.8.0

[PATCH v8 1/3] submodule: add get_submodule_config helper funtion

[wking]

From: "W. Trevor King" <wking@tremily.us>

Several submodule configuration variables
(e.g. fetchRecurseSubmodules) are read from .gitmodules with local
overrides from the usual git config files.  This shell function mimics
that logic to help initialize configuration variables in
git-submodule.sh.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 git-submodule.sh | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/git-submodule.sh b/git-submodule.sh
index 2365149..263a60c 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -153,6 +153,32 @@ die_if_unmatched ()
 }
 
 #
+# Print a submodule configuration setting
+#
+# $1 = submodule name
+# $2 = option name
+# $3 = default value
+#
+# Checks in the usual git-config places first (for overrides),
+# otherwise it falls back on .gitmodules.  This allows you to
+# distribute project-wide defaults in .gitmodules, while still
+# customizing individual repositories if necessary.  If the option is
+# not in .gitmodules either, print a default value.
+#
+get_submodule_config () {
+	name="$1"
+	option="$2"
+	default="$3"
+	value=$(git config submodule."$name"."$option")
+	if test -z "$value"
+	then
+		value=$(git config -f .gitmodules submodule."$name"."$option")
+	fi
+	printf '%s' "${value:-$default}"
+}
+
+
+#
 # Map submodule path to submodule name
 #
 # $1 = path
-- 
1.8.0

[PATCH v8 2/3] submodule update: add --remote for submodule's upstream changes

[wking]

From: "W. Trevor King" <wking@tremily.us>

The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1).  Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1.  This helps you keep up with
changes in the upstream superproject.

However, it's also useful to stay up to date with changes in the
upstream subproject.  Previous workflows for incorporating such
changes include the ungainly:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates.  When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch.  If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`.  You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).

Previous use of submodule.<name>.branch
=======================================

Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option.  The foreach-pull
example above was described by Ævar in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.

Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same.  The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls.  The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`.  Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.

Implementation details
======================

In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1.  However, I didn't change the logic guarding the existing fetch:

  if test -z "$nofetch"
  then
    # Run fetch only if $sha1 isn't present or it
    # is not reachable from a ref.
    (clear_local_git_env; cd "$path" &&
      ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
       test -z "$rev") || git-fetch)) ||
    die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
  fi

There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository.  If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/config.txt        |  6 ++++++
 Documentation/git-submodule.txt | 23 ++++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 21 ++++++++++++++++++++-
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
 5 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index bf8f911..7976a6b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1995,6 +1995,12 @@ submodule.<name>.update::
 	URL and other values found in the `.gitmodules` file.  See
 	linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
 
+submodule.<name>.branch::
+	The remote branch name for a submodule, used by `git submodule
+	update --remote`.  Set this option to override the value found in
+	the `.gitmodules` file.  See linkgit:git-submodule[1] and
+	linkgit:gitmodules[5] for details.
+
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
 	submodule. It can be overridden by using the --[no-]recurse-submodules
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index b1de3ba..8bf173a 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -13,7 +13,7 @@ SYNOPSIS
 	      [--reference <repository>] [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
-'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
+'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase]
 	      [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
@@ -236,6 +236,27 @@ OPTIONS
 	(the default). This limit only applies to modified submodules. The
 	size is always limited to 1 for added/deleted/typechanged submodules.
 
+--remote::
+	This option is only valid for the update command.  Instead of using
+	the superproject's recorded SHA-1 to update the submodule, use the
+	status of the submodule's remote tracking branch.  The remote used
+	is branch's remote (`branch.<name>.remote`), defaulting to `origin`.
+	The remote branch used defaults to `master`, but the branch name may
+	be overridden by setting the `submodule.<name>.branch` option in
+	either `.gitmodules` or `.git/config` (with `.git/config` taking
+	precedence).
++
+This works for any of the supported update procedures (`--checkout`,
+`--rebase`, etc.).  The only change is the source of the target SHA-1.
+For example, `submodule update --remote --merge` will merge upstream
+submodule changes into the submodules, while `submodule update
+--merge` will merge superproject gitlink changes into the submodules.
++
+In order to ensure a current tracking branch state, `update --remote`
+fetches the submodule's remote repository before calculating the
+SHA-1.  If you don't want to fetch, you should use `submodule update
+--remote --no-fetch`.
+
 -N::
 --no-fetch::
 	This option is only valid for the update command.
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index ab3e91c..52d7ae4 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -49,6 +49,11 @@ submodule.<name>.update::
 	This config option is overridden if 'git submodule update' is given
 	the '--merge', '--rebase' or '--checkout' options.
 
+submodule.<name>.branch::
+	A remote branch name for tracking updates in the upstream submodule.
+	If the option is not specified, it defaults to 'master'.  See the
+	`--remote` documentation in linkgit:git-submodule[1] for details.
+
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
 	submodule. If this option is also present in the submodules entry in
diff --git a/git-submodule.sh b/git-submodule.sh
index 263a60c..6ae51c6 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -8,7 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
-   or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--recursive] [--] [<path>...]"
@@ -26,6 +26,7 @@ cached=
 recursive=
 init=
 files=
+remote=
 nofetch=
 update=
 prefix=
@@ -559,6 +560,9 @@ cmd_update()
 		-i|--init)
 			init=1
 			;;
+		--remote)
+			remote=1
+			;;
 		-N|--no-fetch)
 			nofetch=1
 			;;
@@ -619,6 +623,7 @@ cmd_update()
 		fi
 		name=$(module_name "$sm_path") || exit
 		url=$(git config submodule."$name".url)
+		branch=$(get_submodule_config "$name" branch master)
 		if ! test -z "$update"
 		then
 			update_module=$update
@@ -653,6 +658,20 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")"
 		fi
 
+		if test -n "$remote"
+		then
+			if test -z "$nofetch"
+			then
+				# Fetch remote before determining tracking $sha1
+				(clear_local_git_env; cd "$sm_path" && git-fetch) ||
+				die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
+			fi
+			remote_name=$(clear_local_git_env; cd "$sm_path" && get_default_remote)
+			sha1=$(clear_local_git_env; cd "$sm_path" &&
+				git rev-parse --verify "${remote_name}/${branch}") ||
+			die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")"
+		fi
+
 		if test "$subsha1" != "$sha1" -o -n "$force"
 		then
 			subforce=$force
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index feaec6c..4975ec0 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' '
 	)
 '
 
+test_expect_success 'submodule update --remote should fetch upstream changes' '
+	(cd submodule &&
+	 echo line4 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line4"
+	) &&
+	(cd super &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)"
+	)
+'
+
+test_expect_success 'local config should override .gitmodules branch' '
+	(cd submodule &&
+	 git checkout -b test-branch &&
+	 echo line5 >> file &&
+	 git add file &&
+	 test_tick &&
+	 git commit -m "upstream line5" &&
+	 git checkout master
+	) &&
+	(cd super &&
+	 git config submodule.submodule.branch test-branch &&
+	 git submodule update --remote --force submodule &&
+	 cd submodule &&
+	 test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)"
+	)
+'
+
 test_expect_success 'submodule update --rebase staying on master' '
 	(cd super/submodule &&
 	  git checkout master
-- 
1.8.0

[PATCH v8 3/3] submodule add: If --branch is given, record it in .gitmodules

[wking]

From: "W. Trevor King" <wking@tremily.us>

This allows you to easily record a submodule.<name>.branch option in
.gitmodules when you add a new submodule.  With this patch,

  $ git submodule add -b <branch> <repository> [<path>]
  $ git config -f .gitmodules submodule.<path>.branch <branch>

reduces to

  $ git submodule add -b <branch> <repository> [<path>]

This means that future calls to

  $ git submodule update --remote ...

will get updates from the same branch that you used to initialize the
submodule, which is usually what you want.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 2 ++
 git-submodule.sh                | 4 ++++
 t/t7400-submodule-basic.sh      | 1 +
 3 files changed, 7 insertions(+)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 8bf173a..b1996f1 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -208,6 +208,8 @@ OPTIONS
 -b::
 --branch::
 	Branch of repository to add as submodule.
+	The name of the branch is recorded as `submodule.<path>.branch` in
+	`.gitmodules` for `update --remote`.
 
 -f::
 --force::
diff --git a/git-submodule.sh b/git-submodule.sh
index 6ae51c6..22ec5b6 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -417,6 +417,10 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_name".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_name".url "$repo" &&
+	if test -n "$branch"
+	then
+		git config -f .gitmodules submodule."$sm_name".branch "$branch"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index de7d453..2683cba 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' '
 	(
 		cd addtest &&
 		git submodule add -b initial "$submodurl" submod-branch &&
+		test "initial" = "$(git config -f .gitmodules submodule.submod-branch.branch)" &&
 		git submodule init
 	) &&
 
-- 
1.8.0

Re: [PATCH v8 3/3] submodule add: If --branch is given, record it in .gitmodules

[Junio C Hamano]

wking@tremily.us writes:

> From: "W. Trevor King" <wking@tremily.us>
>
> This allows you to easily record a submodule.<name>.branch option in
> .gitmodules when you add a new submodule.  With this patch,
>
>   $ git submodule add -b <branch> <repository> [<path>]
>   $ git config -f .gitmodules submodule.<path>.branch <branch>
>
> reduces to
>
>   $ git submodule add -b <branch> <repository> [<path>]
>
> This means that future calls to
>
>   $ git submodule update --remote ...
>
> will get updates from the same branch that you used to initialize the
> submodule, which is usually what you want.

I agree that it would usually be what you want when you are using
the --remote option.

Will replace the previous round with this.  Thanks.

Re: [PATCH v8 0/3] submodule update: add --remote for submodule's upstream changes

[Heiko Voigt]

Hi,

On Wed, Dec 19, 2012 at 11:03:30AM -0500, wking@tremily.us wrote:
> From: "W. Trevor King" <wking@tremily.us>
> 
> Comments on v7 seem to have petered out, so here's v8.  Changes since
> v7:
> 
> * Series based on gitster/master instead of v1.8.0.
> * In Documentation/config.txt, restored trailing line of
>   submodule.<name>.update documentation, which I had accidentally
>   removed in v7.
> * In Documentation/git-submodule.txt, make --no-fetch example in the
>   --remote description more general, following Phil's suggestion.
> * In git-submodule.sh:
>   * Remove accidental "ges" line.
>   * Use the submodule's default remote to determine which tracking
>     branch to fetch.  In v7 I'd been using the superproject's default
>     remote.
>   * In cmd_add(), use sm_name instead of sm_path to store the --branch
>     option (catching up with 73b0898).

Sorry, I was not able to follow the discussion that closely lately but I
like the outcome. For me there is nothing to change or add functionality
wise. Thanks.

Cheers Heiko

Re: [PATCH v8 1/3] submodule: add get_submodule_config helper funtion

[Heiko Voigt]

On Wed, Dec 19, 2012 at 11:03:31AM -0500, wking@tremily.us wrote:
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 2365149..263a60c 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -153,6 +153,32 @@ die_if_unmatched ()
[...]
> +get_submodule_config () {
> +	name="$1"
> +	option="$2"
> +	default="$3"
> +	value=$(git config submodule."$name"."$option")
> +	if test -z "$value"
> +	then
> +		value=$(git config -f .gitmodules submodule."$name"."$option")
> +	fi
> +	printf '%s' "${value:-$default}"
> +}
> +
> +
> +#

Minor nit: For all other functions we only have one newline as
separator.

Cheers Heiko

Re: [PATCH v8 1/3] submodule: add get_submodule_config helper funtion

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1024 bytes --]

On Fri, Dec 21, 2012 at 09:20:33AM +0100, Heiko Voigt wrote:
> On Wed, Dec 19, 2012 at 11:03:31AM -0500, wking@tremily.us wrote:
> > diff --git a/git-submodule.sh b/git-submodule.sh
> > index 2365149..263a60c 100755
> > --- a/git-submodule.sh
> > +++ b/git-submodule.sh
> > @@ -153,6 +153,32 @@ die_if_unmatched ()
> [...]
> > +get_submodule_config () {
> > +	name="$1"
> > +	option="$2"
> > +	default="$3"
> > +	value=$(git config submodule."$name"."$option")
> > +	if test -z "$value"
> > +	then
> > +		value=$(git config -f .gitmodules submodule."$name"."$option")
> > +	fi
> > +	printf '%s' "${value:-$default}"
> > +}
> > +
> > +
> > +#
> 
> Minor nit: For all other functions we only have one newline as
> separator.

I'm fine rerolling this, or Junio can make the change on my behalf, or
it can be left as is ;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]