[wishlist] submodule.update config

[wishlist] submodule.update config

[Yaroslav Halchenko]

Relates (but orthogonal) to my other thread

  [wishlist] git submodule update --reset-hard

ATM, it possible to specify per submodule update strategy via
configuration variable submodule.SUBMODULE.update where SUBMODULE is the name
of the corresponding submodule.  But I see no way to specify default update
strategy for all submodules.

From our conversation in that other thread  I have discovered to myself about
existence of  submodule.recurse  configuration, and there seems to be a few
more (.fetchJobs, .active) where e.g. .active seems to complement per-submodule
submodule.*.active:

	yoh@debian:~/proj/misc/git$ git grep '[^.]submodule\.[a-z]' -- Documentation/
	Documentation/RelNotes/2.14.0.txt: * Many commands learned to pay attention to submodule.recurse
	Documentation/RelNotes/2.15.0.txt: * "git -c submodule.recurse=yes pull" did not work as if the
	Documentation/config.txt:include::config/submodule.txt[]
	Documentation/config/submodule.txt:     update'. If neither submodule.<name>.active or submodule.active are
	Documentation/config/submodule.txt:     interact with submodules; settings like `submodule.active`
	Documentation/config/submodule.txt:     submodule.active config option. See linkgit:gitsubmodules[7] for
	Documentation/config/submodule.txt:     as computed via `submodule.alternateLocation`. Possible values are
	Documentation/git-clone.txt:    of multiple entries.  The resulting clone has `submodule.active` set to
	Documentation/git-clone.txt:    Defaults to the `submodule.fetchJobs` option.
	Documentation/git-submodule.txt:If no path is specified and submodule.active has been configured, submodules
	Documentation/git-submodule.txt:        Defaults to the `submodule.fetchJobs` option.
	Documentation/gitsubmodules.txt:`submodule.foo.path = path/to/bar`.
	Documentation/gitsubmodules.txt:The section `submodule.foo.*` in the `.gitmodules` file gives additional
	Documentation/gitsubmodules.txt:hints to Git's porcelain layer. For example, the `submodule.foo.url`
	Documentation/gitsubmodules.txt:  b. if the submodule's path matches the pathspec in `submodule.active`
	Documentation/gitsubmodules.txt:submodule's path is excluded in the pathspec in `submodule.active`, the
	Documentation/gitsubmodules.txt:  git config --global submodule.recurse true
	Documentation/gitsubmodules.txt:your working tree. Alternatively you can set 'submodule.recurse' to have
	Documentation/technical/api-config.txt:if (!git_configset_get_bool(gm_config, "submodule.frotz.ignore", &b)) {
	Documentation/technical/http-protocol.txt:  $GIT_URL:     http://example.com/git/repo.git/path/submodule.git
	Documentation/technical/http-protocol.txt:  URL request:  http://example.com/git/repo.git/path/submodule.git/info/refs

I wondered, if you think it would be sensible to also add of
submodule.update which would be considered before submodule.SUBMODULE.update
variable possibly defined per submodule.  That would be more inline with desire
to use any of the --merge, --rebase (and hopefully soon --reset-hard)
strategies specified as an option for submodule update, where no per-submodule
handling  is happening.

Thanks in advance for the consideration!
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

Re: [wishlist] submodule.update config

[Stefan Beller]

On Sat, Dec 8, 2018 at 7:45 AM Yaroslav Halchenko <yoh@onerussian.com> wrote:

> I wondered, if you think it would be sensible to also add of
> submodule.update which would be considered before submodule.SUBMODULE.update
> variable possibly defined per submodule.  That would be more inline with desire
> to use any of the --merge, --rebase (and hopefully soon --reset-hard)
> strategies specified as an option for submodule update, where no per-submodule
> handling  is happening.
>
> Thanks in advance for the consideration!

So you are proposing a variable like submodule.update
without the .<name>. that would apply to any submodule?

The precedence in descending order of these
configs that modify the behavior of "git submodule update"
would be:

* the command line flag (--merge/--rebase/--checkout)
* submodule specific instructions (submodule.<name>.update)
* generic submodule config (the new submodule.update)
* default as --checkout

I first hesitated in thinking this would be a good addition,
as there is no plumbing command for submodules,
to easily modify submodules irrespective of the user
config. But that is out already with the submodule
specific update configs.
So I think it may be a good addition.

I wonder if we'd be better off to re-invent the UX instead
of hiding your intentions in a config setting for a command
that is already long to type. What about

  git submodule merge
  git submodule rebase
  git submodule checkout
  git submodule reset (--hard)

as aliases for
  git submodule update (...)

Re: [wishlist] submodule.update config

[Yaroslav Halchenko]

On Mon, 10 Dec 2018, Stefan Beller wrote:
> > I wondered, if you think it would be sensible to also add of
> > submodule.update which would be considered before submodule.SUBMODULE.update
> > variable possibly defined per submodule.  That would be more inline with desire
> > to use any of the --merge, --rebase (and hopefully soon --reset-hard)
> > strategies specified as an option for submodule update, where no per-submodule
> > handling  is happening.

> > Thanks in advance for the consideration!

> So you are proposing a variable like submodule.update
> without the .<name>. that would apply to any submodule?

yes

> The precedence in descending order of these
> configs that modify the behavior of "git submodule update"
> would be:

> * the command line flag (--merge/--rebase/--checkout)
> * submodule specific instructions (submodule.<name>.update)
> * generic submodule config (the new submodule.update)
> * default as --checkout

sound great

> I first hesitated in thinking this would be a good addition,
> as there is no plumbing command for submodules,
> to easily modify submodules irrespective of the user
> config. But that is out already with the submodule
> specific update configs.
> So I think it may be a good addition.

Glad to hear that. Not sure though I would know where to stick my
nose to figure out what to change. ;-)

> I wonder if we'd be better off to re-invent the UX instead
> of hiding your intentions in a config setting for a command
> that is already long to type. What about

>   git submodule merge
>   git submodule rebase
>   git submodule checkout
>   git submodule reset (--hard)

> as aliases for
>   git submodule update (...)

Well, not sure... In the long run, if UX is to be tuned up, I wonder if
it would be more worthwhile to look toward making all those git commands
(git merge, git checkout, git rebase, ..., git revert, git cherry-pick)
support --recurse-submodules with a consistent with the non-recursive
operation by default behavior (e.g.  not introducing detached HEADs or
controlling that via a set of additional options where needed).  I feel
that "git-submodule" ideally should not get its interface extended to
complement everything "git" commands can do, although that might need to
be extended to provide necessary plumbing.  As for the UX, it should
provide only the set of additional commands, which could not be present
in the main API (e.g. pure "git submodule" itself to list
submodules, and "submodule foreach", "init", "deinit").

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

[PATCH] Re: [wishlist] submodule.update config

[Stefan Beller]

Signed-off-by: Stefan Beller <sbeller@google.com>
---

> > So you are proposing a variable like submodule.update
> [...]
>
> Glad to hear that. Not sure though I would know where to stick my
> nose to figure out what to change. ;-)

The update_module is computed via the submodule--helpers
update-module-mode command, which is a small wrapper
around determine_submodule_update_strategy()
which you already touched in the other patch that makes
--reset-hard another mode.

This contains code and tests, but we'd need some docs as well.
I am not sure about this patch as it allows for easier experimentation
with submodules (e.g. "git config submodule.update '!git reset --hard'"
sounds like what you're trying to get) and using them, but as discussed
below this might be too much convenience already and we'd rather want to
have it properly integrated into the real commands.

> Well, not sure... In the long run, if UX is to be tuned up, I wonder if
> it would be more worthwhile to look toward making all those git commands
> (git merge, git checkout, git rebase, ..., git revert, git cherry-pick)
> support --recurse-submodules with a consistent with the non-recursive
> operation by default behavior

That is the end goal, very much.

> (e.g.  not introducing detached HEADs or
> controlling that via a set of additional options where needed).

As with the discussion of the submodule.repoLike option (the patch I
referenced in the other discussion), this is tricky to get the right
behavior, so it takes some more time to do.

Also what is right for a "git merge --recursive" might be totally different
from a "git submodule update --merge" as the former is not centered around
submodules but merging, such that a sensible default would be expected,
whereas the "submodule update" is allowed to have a rough edge.

From what I get from this discussion is that the submodule.repoLike patch 
needs to offer different modes of submodule operation.

Currently the submodules are handled with the "detached HEAD, period" mode,
whereas that patch proposes a "follow the submodule branch, trust me they're
in sync with the superproject magically" mode, but what you'd rather want to
see is a "don't mess with submodules HEAD detachments, but still have
superproject powers come to be".

As soon as we have one of these modes in place, adding another one
"should be easy", famous last words.

Stefan

 builtin/submodule--helper.c |  4 ++++
 t/t7406-submodule-update.sh | 41 +++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d38113a31a..e1aa3a9995 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1472,6 +1472,10 @@ static void determine_submodule_update_strategy(struct repository *r,
 		if (parse_submodule_update_strategy(val, out) < 0)
 			die(_("Invalid update mode '%s' configured for submodule path '%s'"),
 				val, path);
+	} else if (!repo_config_get_string_const(r, "submodule.update", &val)) {
+		if (parse_submodule_update_strategy(val, out) < 0)
+			die(_("Invalid update mode '%s' configured for 'submodule.update'"),
+				val);
 	} else if (sub->update_strategy.type != SM_UPDATE_UNSPECIFIED) {
 		out->type = sub->update_strategy.type;
 		out->command = sub->update_strategy.command;
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index e87164aa8f..05880fd48f 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -322,6 +322,33 @@ test_expect_success 'submodule update - rebase in .git/config' '
 	)
 '
 
+test_expect_success 'submodule update - rebase in generic .git/config' '
+	git -C super config submodule.update rebase &&
+	git -C super/submodule reset --hard HEAD~1 &&
+	(cd super &&
+	 (cd submodule &&
+	  compare_head
+	 ) &&
+	 git submodule update submodule &&
+	 cd submodule &&
+	 compare_head
+	)
+'
+
+test_expect_success 'submodule.<name>.update overrides submodule.update' '
+	git -C super config submodule.update merge &&
+	git -C super config submodule.submodule.update rebase &&
+	git -C super/submodule reset --hard HEAD~1 &&
+	(cd super &&
+	 (cd submodule &&
+	  compare_head
+	 ) &&
+	 git submodule update submodule &&
+	 cd submodule &&
+	 compare_head
+	)
+'
+
 test_expect_success 'submodule update - checkout in .git/config but --rebase given' '
 	(cd super &&
 	 git config submodule.submodule.update checkout
@@ -339,6 +366,20 @@ test_expect_success 'submodule update - checkout in .git/config but --rebase giv
 	)
 '
 
+test_expect_success 'submodule update - checkout in submodule.update in .git/config but --rebase given' '
+	test_when_finished "git -C super config --unset submodule.update" &&
+	git -C super config submodule.update checkout &&
+	git -C super/submodule reset --hard HEAD~1 &&
+	(cd super &&
+	 (cd submodule &&
+	  compare_head
+	 ) &&
+	 git submodule update --rebase submodule &&
+	 cd submodule &&
+	 compare_head
+	)
+'
+
 test_expect_success 'submodule update - merge in .git/config' '
 	(cd super &&
 	 git config submodule.submodule.update merge
-- 
2.20.0.rc2.403.gdbc3b29805-goog

Re: [PATCH] Re: [wishlist] submodule.update config

[Yaroslav O Halchenko]

On Mon, 10 Dec 2018, Stefan Beller wrote:

> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---

> > > So you are proposing a variable like submodule.update
> > [...]

> > Glad to hear that. Not sure though I would know where to stick my
> > nose to figure out what to change. ;-)

> The update_module is computed via the submodule--helpers
> update-module-mode command, which is a small wrapper
> around determine_submodule_update_strategy()
> which you already touched in the other patch that makes
> --reset-hard another mode.

> This contains code and tests, but we'd need some docs as well.
> I am not sure about this patch as it allows for easier experimentation
> with submodules (e.g. "git config submodule.update '!git reset --hard'"
> sounds like what you're trying to get)

;-) it was indeed one of the original approaches I considered instead of
having "update --reset-hard"...

> and using them, but as discussed
> below this might be too much convenience already and we'd rather want to
> have it properly integrated into the real commands.

indeed, having "update --reset-hard" provides necessary to me
convenience for my use cases.  Motivation behind  submodule.update  was
primarily to allow for heterogeneous (but still simple to define)
strategies, where for some subproject I could just define
submodule.update to be "reset-hard" (I do not expect my local commits
matter) and in the others -- "merge" (I carry my changes on top).

But again, I must confess, that either I forgot or just do not see a
clear use-case/demand for submodule.update config myself any longer,
besides providing a potentially useful default over
submodule.MODULE.update config.

> > Well, not sure... In the long run, if UX is to be tuned up, I wonder if
> > it would be more worthwhile to look toward making all those git commands
> > (git merge, git checkout, git rebase, ..., git revert, git cherry-pick)
> > support --recurse-submodules with a consistent with the non-recursive
> > operation by default behavior

> That is the end goal, very much.

> > (e.g.  not introducing detached HEADs or
> > controlling that via a set of additional options where needed).

> As with the discussion of the submodule.repoLike option (the patch I
> referenced in the other discussion), this is tricky to get the right
> behavior, so it takes some more time to do.

> Also what is right for a "git merge --recursive" might be totally different
> from a "git submodule update --merge" as the former is not centered around
> submodules but merging, such that a sensible default would be expected,
> whereas the "submodule update" is allowed to have a rough edge.

Probably I need to try "submodules update --merge" to see what is that
rough edge which makes it different from the potential "merge
--recurse-submodules", or is it easy to describe? ;-)

I wonder if may be instead of pestering you about this config one, I
should ask about pointers on how to accomplish "revert
--recurse-submodules" or where to poke to make it possible to clone
recursively from  http://datasets.datalad.org/ where  we do not place
submodules all under the very top /.git/modules ;-)

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

Re: [PATCH] Re: [wishlist] submodule.update config

[Stefan Beller]

> But again, I must confess, that either I forgot or just do not see a
> clear use-case/demand for submodule.update config myself any longer,

ok, let's drop that patch then.

> Probably I need to try "submodules update --merge" to see what is that
> rough edge which makes it different from the potential "merge
> --recurse-submodules", or is it easy to describe? ;-)

I think the branch handling would be the difference. I'd expect
"merge --recurse-submodules" to be sensible about staying on
the branch both in the superproject and submodule, whereas
"submodule update --merge" is too much plumbing, that we'd
expect a sensible branch handling (detached HEAD is just fine,
right?)

The merge result would be the same, I'd think.

>
> I wonder if may be instead of pestering you about this config one, I
> should ask about pointers on how to accomplish "revert
> --recurse-submodules"

What do you want to do in revert --recurse-submodules?
When you have "revert --recurse-submodules $COMMIT",
would that revert all submodule commits introduced in
that commit as well as the regular superproject revert?

This would require either opening multiple editors
(once per submodule and at last for the superproject)
or we'd have to do fancy snip-snapping of the user input,
e.g. providing a template like:

    Revert "$title"

    This reverts commit $COMMIT.

    # The above is for the superproject commit
    # Please enter the commit message  ...
    #
    # Changes to be committed:
    #       ...
    # --8<-- DO NOT DELETE THIS LINE
    # Below is the commit for submodule $submodule:
    Revert $submodule_range

    This reverts commits $maybe_many

    # The above is for the submodule commit
    # Please ...

I guess it may be easier to just have multiple
editors opened sequentially to give a commit
message.

>  or where to poke to make it possible to clone
> recursively from  http://datasets.datalad.org/ where  we do not place
> submodules all under the very top /.git/modules ;-)

Not sure what you mean there?

Re: [PATCH] Re: [wishlist] submodule.update config

[Yaroslav Halchenko]

On Wed, 12 Dec 2018, Stefan Beller wrote:

> > But again, I must confess, that either I forgot or just do not see a
> > clear use-case/demand for submodule.update config myself any longer,

> ok, let's drop that patch then.

ok, But I will cherish it in my memory so whenever the use case
comes back to me -- I will be back too ;)

> > Probably I need to try "submodules update --merge" to see what is that
> > rough edge which makes it different from the potential "merge
> > --recurse-submodules", or is it easy to describe? ;-)

> I think the branch handling would be the difference. I'd expect
> "merge --recurse-submodules" to be sensible about staying on
> the branch both in the superproject and submodule, whereas
> "submodule update --merge" is too much plumbing, that we'd
> expect a sensible branch handling (detached HEAD is just fine,
> right?)

re "detached HEAD is just fine" -- I guess "it depends"...  E.g.  why
should it get detached if it was not detached to start with?  Why not
just to perform a regular "git merge --recurse-submodules" within
the submodule thus making it all consistent across?

If there is a need in detached HEADs handling of merges etc, get
them detached and then they would stay detached - no surprises.

> The merge result would be the same, I'd think.

it better be ;)

> > I wonder if may be instead of pestering you about this config one, I
> > should ask about pointers on how to accomplish "revert
> > --recurse-submodules"

> What do you want to do in revert --recurse-submodules?
> When you have "revert --recurse-submodules $COMMIT",
> would that revert all submodule commits introduced in
> that commit as well as the regular superproject revert?

That is correct

> This would require either opening multiple editors
> (once per submodule and at last for the superproject)
> or we'd have to do fancy snip-snapping of the user input,
> e.g. providing a template like:

>     Revert "$title"

>     This reverts commit $COMMIT.

>     # The above is for the superproject commit
>     # Please enter the commit message  ...

>     # Changes to be committed:
>     #       ...
>     # --8<-- DO NOT DELETE THIS LINE
>     # Below is the commit for submodule $submodule:
>     Revert $submodule_range

>     This reverts commits $maybe_many

>     # The above is for the submodule commit
>     # Please ...

> I guess it may be easier to just have multiple
> editors opened sequentially to give a commit
> message.

yeap - that would be beautiful.  Now I just need to do that all manually
;)

> >  or where to poke to make it possible to clone
> > recursively from  http://datasets.datalad.org/ where  we do not place
> > submodules all under the very top /.git/modules ;-)

> Not sure what you mean there?

sorry I was not clear... I will start a new thread for a complete
description.

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik