git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
Search results ordered by [date|relevance]  view[summary|threaded|Atom feed]

* Re: Fwd: New Defects reported by Coverity Scan for git
      [irrelevant] ` <CAGZ79kaPFDMn2K0f529-Crzv+vhU3XUMsSM6w3QV4RXKBv5s_g@mail.gmail.com>
@ 2017-07-18 17:23   ` Junio C Hamano
  2017-07-18 17:41     ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-07-18 17:23 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Jeff King, git\, René Scharfe

Stefan Beller <sbeller@google.com> writes:

> I looked at this report for a while. My current understanding:
> * its detection was triggered by including rs/move-array,
>   f331ab9d4c (use MOVE_ARRAY, 2017-07-15)
> * But it is harmless, because the scan logic does not understand
>   how ALLOC_GROW works. It assumes that
>   done_pbase_paths_alloc can be larger
>   than done_pbase_paths_num + 1, while done_pbase_paths
>   is NULL, such that the memory allocation is not triggered.
>   If that were the case, then we have 2 subsequent dereferences
>   of a NULL pointer right after that. But by inspecting the use
>   of _alloc and _num the initial assumption does not seem possible.

Yes, it does appear that way.  ALLOC_GROW() calls REALLOC_ARRAY()
which safely can realloc NULL to specified size via xrealloc().

I'd be more worried about segfault we seem to be getting only on
Windows from this:

    git -C parent grep -e "(1|2)d(3|4)" --recurse-submodules HEAD^ > actual

in https://travis-ci.org/git/git/jobs/254654195 by the way.



^ permalink raw reply	[relevance 15%]

* Re: Fwd: New Defects reported by Coverity Scan for git
  2017-07-18 17:23   ` Fwd: New Defects reported by Coverity Scan for git Junio C Hamano
@ 2017-07-18 17:41     ` Stefan Beller
  2017-07-18 17:55       ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-18 17:41 UTC (permalink / raw)
  To: Junio C Hamano, Brandon Williams; +Cc: Jeff King, git, René Scharfe

On Tue, Jul 18, 2017 at 10:23 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> I looked at this report for a while. My current understanding:
>> * its detection was triggered by including rs/move-array,
>>   f331ab9d4c (use MOVE_ARRAY, 2017-07-15)
>> * But it is harmless, because the scan logic does not understand
>>   how ALLOC_GROW works. It assumes that
>>   done_pbase_paths_alloc can be larger
>>   than done_pbase_paths_num + 1, while done_pbase_paths
>>   is NULL, such that the memory allocation is not triggered.
>>   If that were the case, then we have 2 subsequent dereferences
>>   of a NULL pointer right after that. But by inspecting the use
>>   of _alloc and _num the initial assumption does not seem possible.
>
> Yes, it does appear that way.  ALLOC_GROW() calls REALLOC_ARRAY()
> which safely can realloc NULL to specified size via xrealloc().
>
> I'd be more worried about segfault we seem to be getting only on
> Windows from this:
>
>     git -C parent grep -e "(1|2)d(3|4)" --recurse-submodules HEAD^ > actual
>
> in https://travis-ci.org/git/git/jobs/254654195 by the way.

Thanks for bringing that to my attention, (I do not follow the travis builds
as closely, as there is no yelling email in my inbox).

Windows builds on travis seem to be flaky.
(sometimes they do not start), but there are also
successful builds, including the -rc0, which may indicate
bw/grep-recurse-submodules may be faulty on Windows.

^ permalink raw reply	[relevance 15%]

* Re: Fwd: New Defects reported by Coverity Scan for git
  2017-07-18 17:41     ` Stefan Beller
@ 2017-07-18 17:55       ` Junio C Hamano
  2017-07-18 18:00         ` Brandon Williams
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-07-18 17:55 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Brandon Williams, Jeff King, git\, René Scharfe

Stefan Beller <sbeller@google.com> writes:

>> I'd be more worried about segfault we seem to be getting only on
>> Windows from this:
>>
>>     git -C parent grep -e "(1|2)d(3|4)" --recurse-submodules HEAD^ > actual
>>
>> in https://travis-ci.org/git/git/jobs/254654195 by the way.
>
> Thanks for bringing that to my attention, (I do not follow the travis builds
> as closely, as there is no yelling email in my inbox).
>
> Windows builds on travis seem to be flaky.
> (sometimes they do not start), but there are also
> successful builds, including the -rc0, which may indicate
> bw/grep-recurse-submodules may be faulty on Windows.

I can get valgrind complaints locally from

    $ cd t && sh t7814-grep*.sh --valgrind -i -v

so this may not be Windows only.  Can repo_worktree_path() return a NULL
in repo_read_gitmodules() to cause git_config_from_file() barf on a NULL
gitmodule_path?

==20383== Syscall param open(filename) points to unaddressable byte(s)
==20383==    at 0x5153140: __open_nocancel (/build/eglibc-SvCtMH/eglibc-2.19/io/../sysdeps/unix/syscall-template.S:81)
==20383==    by 0x50DDED7: _IO_file_fopen@@GLIBC_2.2.5 (/build/eglibc-SvCtMH/eglibc-2.19/libio/fileops.c:228)
==20383==    by 0x50D23D3: __fopen_internal (/build/eglibc-SvCtMH/eglibc-2.19/libio/iofopen.c:90)
==20383==    by 0x569107: git_fopen (/home/gitster/git.git/compat/fopen.c:22)
==20383==    by 0x55B1ED: fopen_or_warn (/home/gitster/git.git/wrapper.c:439)
==20383==    by 0x4A2A32: git_config_from_file (/home/gitster/git.git/config.c:1442)
==20383==    by 0x540317: repo_read_gitmodules (/home/gitster/git.git/submodule.c:269)
==20383==    by 0x434389: grep_submodule (/home/gitster/git.git/builtin/grep.c:422)
==20383==    by 0x4348C8: grep_tree (/home/gitster/git.git/builtin/grep.c:580)
==20383==    by 0x434867: grep_tree (/home/gitster/git.git/builtin/grep.c:576)
==20383==    by 0x436034: cmd_grep (/home/gitster/git.git/builtin/grep.c:622)
==20383==    by 0x4063DC: handle_builtin (/home/gitster/git.git/git.c:330)
==20383==  Address 0x0 is not stack'd, malloc'd or (recently) free'd



^ permalink raw reply	[relevance 14%]

* Re: Fwd: New Defects reported by Coverity Scan for git
  2017-07-18 17:55       ` Junio C Hamano
@ 2017-07-18 18:00         ` Brandon Williams
  2017-07-18 18:06           ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Brandon Williams @ 2017-07-18 18:00 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Stefan Beller, Jeff King, git, René Scharfe

On 07/18, Junio C Hamano wrote:
> Stefan Beller <sbeller@google.com> writes:
> 
> >> I'd be more worried about segfault we seem to be getting only on
> >> Windows from this:
> >>
> >>     git -C parent grep -e "(1|2)d(3|4)" --recurse-submodules HEAD^ > actual
> >>
> >> in https://travis-ci.org/git/git/jobs/254654195 by the way.
> >
> > Thanks for bringing that to my attention, (I do not follow the travis builds
> > as closely, as there is no yelling email in my inbox).
> >
> > Windows builds on travis seem to be flaky.
> > (sometimes they do not start), but there are also
> > successful builds, including the -rc0, which may indicate
> > bw/grep-recurse-submodules may be faulty on Windows.
> 
> I can get valgrind complaints locally from
> 
>     $ cd t && sh t7814-grep*.sh --valgrind -i -v
> 
> so this may not be Windows only.  Can repo_worktree_path() return a NULL
> in repo_read_gitmodules() to cause git_config_from_file() barf on a NULL
> gitmodule_path?

Yep that's most likely the cause.  The issue is if a repository doesn't
have a worktree then what should a worktree path look like?
repo_read_gitmodules() should check if there is a worktree before trying
to load the gitmodules file.  I actually noticed this and have it fixed locally in
another series I'm working on right now.  Looks like I may have to get
that change in first though.  Thanks for finding this.

> 
> ==20383== Syscall param open(filename) points to unaddressable byte(s)
> ==20383==    at 0x5153140: __open_nocancel (/build/eglibc-SvCtMH/eglibc-2.19/io/../sysdeps/unix/syscall-template.S:81)
> ==20383==    by 0x50DDED7: _IO_file_fopen@@GLIBC_2.2.5 (/build/eglibc-SvCtMH/eglibc-2.19/libio/fileops.c:228)
> ==20383==    by 0x50D23D3: __fopen_internal (/build/eglibc-SvCtMH/eglibc-2.19/libio/iofopen.c:90)
> ==20383==    by 0x569107: git_fopen (/home/gitster/git.git/compat/fopen.c:22)
> ==20383==    by 0x55B1ED: fopen_or_warn (/home/gitster/git.git/wrapper.c:439)
> ==20383==    by 0x4A2A32: git_config_from_file (/home/gitster/git.git/config.c:1442)
> ==20383==    by 0x540317: repo_read_gitmodules (/home/gitster/git.git/submodule.c:269)
> ==20383==    by 0x434389: grep_submodule (/home/gitster/git.git/builtin/grep.c:422)
> ==20383==    by 0x4348C8: grep_tree (/home/gitster/git.git/builtin/grep.c:580)
> ==20383==    by 0x434867: grep_tree (/home/gitster/git.git/builtin/grep.c:576)
> ==20383==    by 0x436034: cmd_grep (/home/gitster/git.git/builtin/grep.c:622)
> ==20383==    by 0x4063DC: handle_builtin (/home/gitster/git.git/git.c:330)
> ==20383==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
> 
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 1%]

* Re: Fwd: New Defects reported by Coverity Scan for git
  2017-07-18 18:00         ` Brandon Williams
@ 2017-07-18 18:06           ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-18 18:06 UTC (permalink / raw)
  To: Brandon Williams; +Cc: Junio C Hamano, Jeff King, git, René Scharfe

On Tue, Jul 18, 2017 at 11:00 AM, Brandon Williams <bmwill@google.com> wrote:
> On 07/18, Junio C Hamano wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>
>> >> I'd be more worried about segfault we seem to be getting only on
>> >> Windows from this:
>> >>
>> >>     git -C parent grep -e "(1|2)d(3|4)" --recurse-submodules HEAD^ > actual
>> >>
>> >> in https://travis-ci.org/git/git/jobs/254654195 by the way.
>> >
>> > Thanks for bringing that to my attention, (I do not follow the travis builds
>> > as closely, as there is no yelling email in my inbox).
>> >
>> > Windows builds on travis seem to be flaky.
>> > (sometimes they do not start), but there are also
>> > successful builds, including the -rc0, which may indicate
>> > bw/grep-recurse-submodules may be faulty on Windows.
>>
>> I can get valgrind complaints locally from
>>
>>     $ cd t && sh t7814-grep*.sh --valgrind -i -v
>>
>> so this may not be Windows only.  Can repo_worktree_path() return a NULL
>> in repo_read_gitmodules() to cause git_config_from_file() barf on a NULL
>> gitmodule_path?
>
> Yep that's most likely the cause.  The issue is if a repository doesn't
> have a worktree then what should a worktree path look like?
> repo_read_gitmodules() should check if there is a worktree before trying
> to load the gitmodules file.  I actually noticed this and have it fixed locally in
> another series I'm working on right now.  Looks like I may have to get
> that change in first though.  Thanks for finding this.

If there is no worktree, we could fallback to read it from the tree
HEAD:.gitmodules, if that doesn't exist, then there are no submodules.

Thanks,
Stefan

^ permalink raw reply	[relevance 15%]

* Re: [PATCH] RFC: A new type of symbolic refs
      [irrelevant]   ` <CAGZ79kY=j-sgrAFwAhP-_9EKBsa6FXARaVuCxM6gPHV7M+1OAA@mail.gmail.com>
@ 2017-07-18 19:03     ` Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2017-07-18 19:03 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git\

Stefan Beller <sbeller@google.com> writes:

>> Even if we limit ourselves to pointing at the index of the
>> superproject, there probably are a handful of interesting issues
>> that need to be clarified (not in the sense of "this and that issues
>> exist, so this won't be a useful feature", but in the sense of "we'd
>> be able to do these useful things using this feature, and we need to
>> fill in more details"), such as:
>>
>>  - Making new commits in the submodule available to the upstream.
>>    Just like a detached HEAD in the submodule, this is not tied to
>>    any concrete branch, and it is unclear how a recursive "push"
>>    from the superproject should propagate the changes to the
>>    upstream of the submodule;
>
> In Gerrit land people are trained to use
>   git push HEAD:refs/for/master
> which would work fine when propagated to the submodule as-is.

What I was alluding to is how you find 'master' in 'for/master',
given the submodule HEAD which only says "I point at the index of
the superproject".  Note that 'master' here refers to a branch
in the submodule.  If we further limit ourselves and declare that
a submodule that uses this external-reference as HEAD must always
be on the same branch as the superproject is on, then we can find
what branch the superproject is on, and use that.

^ permalink raw reply	[relevance 17%]

* [GSoC][PATCH 0/8] Update: Week 9
@ 2017-07-18 20:48 Prathamesh Chavan
  2017-07-18 20:48 ` [GSoC][PATCH 1/8] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
                   ` (6 more replies)
  0 siblings, 7 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:48 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

SUMMARY OF MY PROJECT:

Git submodule subcommands are currently implemented by using shell script
'git-submodule.sh'. There are several reasons why we'll prefer not to
use the shell script. My project intends to convert the subcommands into
C code, thus making them builtins. This will increase Git's portability
and hence the efficiency of working with the git-submodule commands.
Link to the complete proposal: [1]

Mentors:
Stefan Beller <sbeller@google.com>
Christian Couder <christian.couder@gmail.com>

UPDATES:

Following are the updates about my ongoing project:

* status: Certain optimization were implemented as they were suggested.
  Also, the new version was posted for review[2]. This update also
  contains the above-stated version of this patch. 

* sync: The lasted version was posted on the mailing list[3].
  This update also contains the above-stated version of this patch.

* summary: This patch is updated after its last review.
  and the updated one is attached with this update.

* add: porting of this submodule subcommand has started.

* foreach: The former patch[4] posted on the mailing list has been split
  into smaller patches, along with certain additional changes which
  were suggested in the reviews. The patch is currently being posted
  discussion with the mentors and I aim to post it on the mailing
  list soon.

PLAN FOR WEEK-10 (18 July 2017 to 24 July 2017):

* foreach: After having a discussion with the mentors about the prepared
  patch, I'll post the patches.

* add: the porting of this subcommand has begun and will aim to finish
  it by the end of this week.

* Apart from that, I also aim to work on getting the rest of the patches
  ('status', 'sync', 'deinit', and other functions) merged.

Apart from this, sorry for posting the update late for this week. I arrived
at my college late yesterday and hence wasn't able to prepare this with
the ongoing classes. Also, I would do my best so that this doesn't occur
again.

[1]: https://docs.google.com/document/d/1krxVLooWl--75Pot3dazhfygR3wCUUWZWzTXtK1L-xU/
[2]: https://public-inbox.org/git/20170713200538.25806-4-pc44800@gmail.com/
[3]: https://public-inbox.org/git/20170713200538.25806-5-pc44800@gmail.com/
[4]: https://public-inbox.org/git/20170603003710.5558-1-sbeller@google.com/

Prathamesh Chavan (8):
  submodule--helper: introduce get_submodule_displaypath()
  submodule--helper: introduce for_each_submodule_list()
  submodule: port set_name_rev() from shell to C
  submodule: port submodule subcommand 'status' from shell to C
  submodule: port submodule subcommand 'sync' from shell to C
  submodule: port submodule subcommand 'deinit' from shell to C
  diff: change scope of the function count_lines()
  submodule: port submodule subcommand 'summary' from shell to C

 builtin/submodule--helper.c | 1070 ++++++++++++++++++++++++++++++++++++++++++-
 diff.c                      |    2 +-
 diff.h                      |    1 +
 git-submodule.sh            |  354 +-------------
 4 files changed, 1056 insertions(+), 371 deletions(-)

-- 
2.13.0


^ permalink raw reply	[relevance 17%]

* [GSoC][PATCH 1/8] submodule--helper: introduce get_submodule_displaypath()
  2017-07-18 20:48 [GSoC][PATCH 0/8] Update: Week 9 Prathamesh Chavan
@ 2017-07-18 20:48 ` Prathamesh Chavan
  2017-07-18 20:48 ` [GSoC][PATCH 2/8] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:48 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function get_submodule_displaypath() to replace the code
occurring in submodule_init() for generating displaypath of the
submodule with a call to it.

This new function will also be used in other parts of the system
in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 6abdad329..7af4de09b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -220,6 +220,27 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+
+	if (prefix && super_prefix) {
+		BUG("cannot have prefix '%s' and superprefix '%s'",
+		    prefix, super_prefix);
+	} else if (prefix) {
+		struct strbuf sb = STRBUF_INIT;
+		char *displaypath = xstrdup(relative_path(path, prefix, &sb));
+		strbuf_release(&sb);
+		return displaypath;
+	} else if (super_prefix) {
+		int len = strlen(super_prefix);
+		const char *format = is_dir_sep(super_prefix[len - 1]) ? "%s%s" : "%s/%s";
+		return xstrfmt(format, super_prefix, path);
+	} else {
+		return xstrdup(path);
+	}
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -339,16 +360,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 	/* Only loads from .gitmodules, no overlay with .git/config */
 	gitmodules_config();
-
-	if (prefix && get_super_prefix())
-		die("BUG: cannot have prefix and superprefix");
-	else if (prefix)
-		displaypath = xstrdup(relative_path(path, prefix, &sb));
-	else if (get_super_prefix()) {
-		strbuf_addf(&sb, "%s%s", get_super_prefix(), path);
-		displaypath = strbuf_detach(&sb, NULL);
-	} else
-		displaypath = xstrdup(path);
+	displaypath = get_submodule_displaypath(path, prefix);
 
 	sub = submodule_from_path(null_sha1, path);
 
@@ -363,7 +375,6 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 * Set active flag for the submodule being initialized
 	 */
 	if (!is_submodule_active(the_repository, path)) {
-		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH 2/8] submodule--helper: introduce for_each_submodule_list()
  2017-07-18 20:48 [GSoC][PATCH 0/8] Update: Week 9 Prathamesh Chavan
  2017-07-18 20:48 ` [GSoC][PATCH 1/8] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
@ 2017-07-18 20:48 ` Prathamesh Chavan
  2017-07-18 20:48 ` [GSoC][PATCH 3/8] submodule: port set_name_rev() from shell to C Prathamesh Chavan
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:48 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function for_each_submodule_list() and
replace a loop in module_init() with a call to it.

The new function will also be used in other parts of the
system in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7af4de09b..e41572f7a 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -14,6 +14,9 @@
 #include "refs.h"
 #include "connect.h"
 
+typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
+				      void *cb_data);
+
 static char *get_default_remote(void)
 {
 	char *dest = NULL, *ret;
@@ -352,17 +355,30 @@ static int module_list(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static void init_submodule(const char *path, const char *prefix, int quiet)
+static void for_each_submodule_list(const struct module_list list,
+				    submodule_list_func_t fn, void *cb_data)
 {
+	int i;
+	for (i = 0; i < list.nr; i++)
+		fn(list.entries[i], cb_data);
+}
+
+struct init_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+};
+#define INIT_CB_INIT { NULL, 0 }
+
+static void init_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct init_cb *info = cb_data;
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	/* Only loads from .gitmodules, no overlay with .git/config */
-	gitmodules_config();
-	displaypath = get_submodule_displaypath(path, prefix);
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
 
-	sub = submodule_from_path(null_sha1, path);
+	sub = submodule_from_path(null_sha1, list_item->name);
 
 	if (!sub)
 		die(_("No url found for submodule path '%s' in .gitmodules"),
@@ -374,7 +390,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_active(the_repository, path)) {
+	if (!is_submodule_active(the_repository, list_item->name)) {
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
@@ -416,7 +432,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 		if (git_config_set_gently(sb.buf, url))
 			die(_("Failed to register url for submodule path '%s'"),
 			    displaypath);
-		if (!quiet)
+		if (!info->quiet)
 			fprintf(stderr,
 				_("Submodule '%s' (%s) registered for path '%s'\n"),
 				sub->name, url, displaypath);
@@ -445,10 +461,10 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 static int module_init(int argc, const char **argv, const char *prefix)
 {
+	struct init_cb info = INIT_CB_INIT;
 	struct pathspec pathspec;
 	struct module_list list = MODULE_LIST_INIT;
 	int quiet = 0;
-	int i;
 
 	struct option module_init_options[] = {
 		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
@@ -473,8 +489,11 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	if (!argc && git_config_get_value_multi("submodule.active"))
 		module_list_active(&list);
 
-	for (i = 0; i < list.nr; i++)
-		init_submodule(list.entries[i]->name, prefix, quiet);
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+
+	gitmodules_config();
+	for_each_submodule_list(list, init_submodule, &info);
 
 	return 0;
 }
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 3/8] submodule: port set_name_rev() from shell to C
  2017-07-18 20:48 [GSoC][PATCH 0/8] Update: Week 9 Prathamesh Chavan
  2017-07-18 20:48 ` [GSoC][PATCH 1/8] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
  2017-07-18 20:48 ` [GSoC][PATCH 2/8] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
@ 2017-07-18 20:48 ` Prathamesh Chavan
  2017-07-18 20:49 ` [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' " Prathamesh Chavan
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:48 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Function set_name_rev() is ported from git-submodule to the
submodule--helper builtin. The function get_name_rev() generates the
value of the revision name as required, and the function
print_name_rev() handles the formating and printing of the obtained
revision name.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 63 +++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 16 ++----------
 2 files changed, 65 insertions(+), 14 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index e41572f7a..80f744407 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -244,6 +244,68 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+static char *get_name_rev(const char *sub_path, const char* object_id)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char ***d;
+
+	static const char *describe_bare[] = {
+		NULL
+	};
+
+	static const char *describe_tags[] = {
+		"--tags", NULL
+	};
+
+	static const char *describe_contains[] = {
+		"--contains", NULL
+	};
+
+	static const char *describe_all_always[] = {
+		"--all", "--always", NULL
+	};
+
+	static const char **describe_argv[] = {
+		describe_bare, describe_tags, describe_contains,
+		describe_all_always, NULL
+	};
+
+	for (d = describe_argv; *d; d++) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+		prepare_submodule_repo_env(&cp.env_array);
+		cp.dir = sub_path;
+		cp.git_cmd = 1;
+		cp.no_stderr = 1;
+
+		argv_array_push(&cp.args, "describe");
+		argv_array_pushv(&cp.args, *d);
+		argv_array_push(&cp.args, object_id);
+
+		if (!capture_command(&cp, &sb, 0) && sb.len) {
+			strbuf_strip_suffix(&sb, "\n");
+			return strbuf_detach(&sb, NULL);
+		}
+
+	}
+
+	strbuf_release(&sb);
+	return NULL;
+}
+
+static int print_name_rev(int argc, const char **argv, const char *prefix)
+{
+	char *namerev;
+	if (argc != 3)
+		die("print-name-rev only accepts two arguments: <path> <sha1>");
+
+	namerev = get_name_rev(argv[1], argv[2]);
+	if (namerev && namerev[0])
+		printf(" (%s)", namerev);
+	printf("\n");
+
+	return 0;
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -1242,6 +1304,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index e131760ee..e988167e0 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -759,18 +759,6 @@ cmd_update()
 	}
 }
 
-set_name_rev () {
-	revname=$( (
-		sanitize_submodule_env
-		cd "$1" && {
-			git describe "$2" 2>/dev/null ||
-			git describe --tags "$2" 2>/dev/null ||
-			git describe --contains "$2" 2>/dev/null ||
-			git describe --all --always "$2"
-		}
-	) )
-	test -z "$revname" || revname=" ($revname)"
-}
 #
 # Show commit summary for submodules in index or working tree
 #
@@ -1042,14 +1030,14 @@ cmd_status()
 		fi
 		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
 		then
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say " $sha1 $displaypath$revname"
 		else
 			if test -z "$cached"
 			then
 				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
 			fi
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say "+$sha1 $displaypath$revname"
 		fi
 
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' from shell to C
  2017-07-18 20:48 [GSoC][PATCH 0/8] Update: Week 9 Prathamesh Chavan
                   ` (2 preceding siblings ...)
  2017-07-18 20:48 ` [GSoC][PATCH 3/8] submodule: port set_name_rev() from shell to C Prathamesh Chavan
@ 2017-07-18 20:49 ` " Prathamesh Chavan
  2017-07-18 21:39   ` Stefan Beller
  2017-07-18 20:49 ` [GSoC][PATCH 5/8] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
                   ` (2 subsequent siblings)
  6 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:49 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

This aims to make git-submodule 'status' a built-in. Hence, the function
cmd_status() is ported from shell to C. This is done by introducing
three functions: module_status(), submodule_status() and print_status().

The function module_status() acts as the front-end of the subcommand.
It parses subcommand's options and then calls the function
module_list_compute() for computing the list of submodules. Then
this functions calls for_each_submodule_list() looping through the
list obtained.

Then for_each_submodule_list() calls submodule_status() for each of the
submodule in its list. The function submodule_status() is responsible
for generating the status each submodule it is called for, and
then calls print_status().

Finally, the function print_status() handles the printing of submodule's
status.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 146 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  49 +--------------
 2 files changed, 147 insertions(+), 48 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 80f744407..9c1630495 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -560,6 +560,151 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct status_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+	unsigned int cached: 1;
+};
+#define STATUS_CB_INIT { NULL, 0, 0, 0 }
+
+static void print_status(struct status_cb *info, char state, const char *path,
+			 char *sub_sha1, char *displaypath)
+{
+	if (info->quiet)
+		return;
+
+	printf("%c%s %s", state, sub_sha1, displaypath);
+
+	if (state == ' ' || state == '+') {
+		struct argv_array name_rev_args = ARGV_ARRAY_INIT;
+
+		argv_array_pushl(&name_rev_args, "print-name-rev",
+				 path, sub_sha1, NULL);
+		print_name_rev(name_rev_args.argc, name_rev_args.argv,
+			       info->prefix);
+	} else {
+		printf("\n");
+	}
+}
+
+static int handle_submodule_head_ref(const char *refname,
+				     const struct object_id *oid, int flags,
+				     void *cb_data)
+{
+	struct strbuf *output = cb_data;
+	if (oid)
+		strbuf_addstr(output, oid_to_hex(oid));
+	return 0;
+}
+
+static void status_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct status_cb *info = cb_data;
+	char *sub_sha1 = xstrdup(oid_to_hex(&list_item->oid));
+	char *displaypath;
+	struct stat st;
+
+	if (!submodule_from_path(null_sha1, list_item->name))
+		die(_("no submodule mapping found in .gitmodules for path '%s'"),
+		      list_item->name);
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (list_item->ce_flags) {
+		print_status(info, 'U', list_item->name,
+			     sha1_to_hex(null_sha1), displaypath);
+		goto cleanup;
+	}
+
+	if (!is_submodule_active(the_repository, list_item->name)) {
+		print_status(info, '-', list_item->name, sub_sha1, displaypath);
+		goto cleanup;
+	}
+
+	if (!lstat(list_item->name, &st) && !ce_match_stat(list_item, &st, 0)) {
+		print_status(info, ' ', list_item->name, sub_sha1, displaypath);
+	} else {
+		if (!info->cached) {
+			struct strbuf sb = STRBUF_INIT;
+			if (head_ref_submodule(list_item->name,
+					       handle_submodule_head_ref, &sb))
+				die(_("could not resolve HEAD ref inside the"
+				      "submodule '%s'"), list_item->name);
+			print_status(info, '+', list_item->name, sb.buf,
+				     displaypath);
+			strbuf_release(&sb);
+		} else {
+			print_status(info, '+', list_item->name, sub_sha1,
+				     displaypath);
+		}
+	}
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "status", "--recursive",
+				 NULL);
+
+		if (info->cached)
+			argv_array_push(&cpr.args, "--cached");
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	free(displaypath);
+	free(sub_sha1);
+}
+
+static int module_status(int argc, const char **argv, const char *prefix)
+{
+	struct status_cb info = STATUS_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int cached = 0;
+	int recursive = 0;
+
+	struct option module_status_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT_BOOL(0, "cached", &cached, N_("Use commit stored in the index instead of the one stored in the submodule HEAD")),
+		OPT_BOOL(0, "recursive", &recursive, N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule status [--quiet] [--cached] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_status_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+	info.cached = !!cached;
+
+	gitmodules_config();
+	for_each_submodule_list(list, status_submodule, &info);
+
+	return 0;
+}
+
 static int module_name(int argc, const char **argv, const char *prefix)
 {
 	const struct submodule *sub;
@@ -1306,6 +1451,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
+	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index e988167e0..51b057d82 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1005,54 +1005,7 @@ cmd_status()
 		shift
 	done
 
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		if test "$stage" = U
-		then
-			say "U$sha1 $displaypath"
-			continue
-		fi
-		if ! git submodule--helper is-active "$sm_path" ||
-		{
-			! test -d "$sm_path"/.git &&
-			! test -f "$sm_path"/.git
-		}
-		then
-			say "-$sha1 $displaypath"
-			continue;
-		fi
-		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
-		then
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say " $sha1 $displaypath$revname"
-		else
-			if test -z "$cached"
-			then
-				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
-			fi
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say "+$sha1 $displaypath$revname"
-		fi
-
-		if test -n "$recursive"
-		then
-			(
-				prefix="$displaypath/"
-				sanitize_submodule_env
-				wt_prefix=
-				cd "$sm_path" &&
-				eval cmd_status
-			) ||
-			die "$(eval_gettext "Failed to recurse into submodule path '\$sm_path'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} "$@"
 }
 #
 # Sync remote urls for submodules
-- 
2.13.0


^ permalink raw reply	[relevance 20%]

* [GSoC][PATCH 5/8] submodule: port submodule subcommand 'sync' from shell to C
  2017-07-18 20:48 [GSoC][PATCH 0/8] Update: Week 9 Prathamesh Chavan
                   ` (3 preceding siblings ...)
  2017-07-18 20:49 ` [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-07-18 20:49 ` " Prathamesh Chavan
  2017-07-18 22:23   ` Stefan Beller
  2017-07-18 20:49 ` [GSoC][PATCH 6/8] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
  2017-07-18 20:49 ` [GSoC][PATCH 8/8] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
  6 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:49 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Port the submodule subcommand 'sync' from shell to C using the same
mechanism as that used for porting submodule subcommand 'status'.
Hence, here the function cmd_sync() is ported from shell to C.
This is done by introducing three functions: module_sync(),
sync_submodule() and print_default_remote().

The function print_default_remote() is introduced for getting
the default remote as stdout.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 179 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  56 +-------------
 2 files changed, 180 insertions(+), 55 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 9c1630495..da91c489b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -44,6 +44,20 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static int print_default_remote(int argc, const char **argv, const char *prefix)
+{
+	const char *remote;
+
+	if (argc != 1)
+		die(_("submodule--helper print-default-remote takes no arguments"));
+
+	remote = get_default_remote();
+	if (remote)
+		puts(remote);
+
+	return 0;
+}
+
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -379,6 +393,25 @@ static void module_list_active(struct module_list *list)
 	*list = active_modules;
 }
 
+static char *get_up_path(const char *path)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	for (i = count_slashes(path); i; i--)
+		strbuf_addstr(&sb, "../");
+
+	/*
+	 * Check if 'path' ends with slash or not
+	 * for having the same output for dir/sub_dir
+	 * and dir/sub_dir/
+	 */
+	if (!is_dir_sep(path[strlen(path) - 1]))
+		strbuf_addstr(&sb, "../");
+
+	return strbuf_detach(&sb, NULL);
+}
+
 static int module_list(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -724,6 +757,150 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct sync_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define SYNC_CB_INIT { NULL, 0, 0 }
+
+static void sync_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct sync_cb *info = cb_data;
+	const struct submodule *sub;
+	char *sub_key, *remote_key;
+	char *sub_origin_url, *super_config_url, *displaypath;
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	if (!is_submodule_active(the_repository, list_item->name))
+		return;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub || !sub->url)
+		die(_("no url found for submodule path '%s' in .gitmodules"),
+		      list_item->name);
+
+	if (starts_with_dot_dot_slash(sub->url) || starts_with_dot_slash(sub->url)) {
+		char *remote_url, *up_path;
+		char *remote = get_default_remote();
+		char *remote_key = xstrfmt("remote.%s.url", remote);
+
+		if (git_config_get_string(remote_key, &remote_url))
+			remote_url = xgetcwd();
+
+		up_path = get_up_path(list_item->name);
+		sub_origin_url = relative_url(remote_url, sub->url, up_path);
+		super_config_url = relative_url(remote_url, sub->url, NULL);
+
+		free(remote);
+		free(remote_key);
+		free(up_path);
+		free(remote_url);
+	} else {
+		sub_origin_url = xstrdup(sub->url);
+		super_config_url = xstrdup(sub->url);
+	}
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (!info->quiet)
+		printf(_("Synchronizing submodule url for '%s'\n"),
+			 displaypath);
+
+	sub_key = xstrfmt("submodule.%s.url", sub->name);
+	if (git_config_set_gently(sub_key, super_config_url))
+		die(_("failed to register url for submodule path '%s'"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.dir = list_item->name;
+	argv_array_pushl(&cp.args, "submodule--helper",
+			 "print-default-remote", NULL);
+	if (capture_command(&cp, &sb, 0))
+		die(_("failed to get the default remote for submodule '%s'"),
+		      list_item->name);
+
+	strbuf_strip_suffix(&sb, "\n");
+	remote_key = xstrfmt("remote.%s.url", sb.buf);
+	strbuf_release(&sb);
+
+	child_process_init(&cp);
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.dir = list_item->name;
+	argv_array_pushl(&cp.args, "config", remote_key, sub_origin_url, NULL);
+	if (run_command(&cp))
+		die(_("failed to update remote for submodule '%s'"),
+		      list_item->name);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "sync", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	free(sub_key);
+	free(super_config_url);
+	free(displaypath);
+	free(sub_origin_url);
+}
+
+static int module_sync(int argc, const char **argv, const char *prefix)
+{
+	struct sync_cb info = SYNC_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_sync_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of synchronizing submodule url")),
+		OPT_BOOL(0, "recursive", &recursive,
+			N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper sync [--quiet] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_sync_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, sync_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1452,6 +1629,8 @@ static struct cmd_struct commands[] = {
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
+	{"print-default-remote", print_default_remote, 0},
+	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 51b057d82..6bfc5e17d 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1037,63 +1037,9 @@ cmd_sync()
 			;;
 		esac
 	done
-	cd_to_toplevel
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-
-		# skip inactive submodules
-		if ! git submodule--helper is-active "$sm_path"
-		then
-			continue
-		fi
-
-		name=$(git submodule--helper name "$sm_path")
-		url=$(git config -f .gitmodules --get submodule."$name".url)
-
-		# Possibly a url relative to parent
-		case "$url" in
-		./*|../*)
-			# rewrite foo/bar as ../.. to find path from
-			# submodule work tree to superproject work tree
-			up_path="$(printf '%s\n' "$sm_path" | sed "s/[^/][^/]*/../g")" &&
-			# guarantee a trailing /
-			up_path=${up_path%/}/ &&
-			# path from submodule work tree to submodule origin repo
-			sub_origin_url=$(git submodule--helper resolve-relative-url "$url" "$up_path") &&
-			# path from superproject work tree to submodule origin repo
-			super_config_url=$(git submodule--helper resolve-relative-url "$url") || exit
-			;;
-		*)
-			sub_origin_url="$url"
-			super_config_url="$url"
-			;;
-		esac
 
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		say "$(eval_gettext "Synchronizing submodule url for '\$displaypath'")"
-		git config submodule."$name".url "$super_config_url"
-
-		if test -e "$sm_path"/.git
-		then
-		(
-			sanitize_submodule_env
-			cd "$sm_path"
-			remote=$(get_default_remote)
-			git config remote."$remote".url "$sub_origin_url"
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 
-			if test -n "$recursive"
-			then
-				prefix="$prefix$sm_path/"
-				eval cmd_sync
-			fi
-		)
-		fi
-	done
 }
 
 cmd_absorbgitdirs()
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH 6/8] submodule: port submodule subcommand 'deinit' from shell to C
  2017-07-18 20:48 [GSoC][PATCH 0/8] Update: Week 9 Prathamesh Chavan
                   ` (4 preceding siblings ...)
  2017-07-18 20:49 ` [GSoC][PATCH 5/8] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-07-18 20:49 ` " Prathamesh Chavan
  2017-07-18 20:49 ` [GSoC][PATCH 8/8] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
  6 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:49 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The same mechanism is used even for porting this submodule
subcommand, as used in the ported subcommands till now.
The function cmd_deinit in split up after porting into three
functions: module_deinit(), for_each_submodule_list() and
deinit_submodule().

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 143 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  55 +----------------
 2 files changed, 144 insertions(+), 54 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index da91c489b..8a679abf6 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -901,6 +901,148 @@ static int module_sync(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct deinit_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int force: 1;
+	unsigned int all: 1;
+};
+#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
+
+static void deinit_submodule(const struct cache_entry *list_item,
+			     void *cb_data)
+{
+	struct deinit_cb *info = cb_data;
+	const struct submodule *sub;
+	char *displaypath = NULL;
+	struct child_process cp_config = CHILD_PROCESS_INIT;
+	struct strbuf sb_config = STRBUF_INIT;
+	char *sm_path = xstrdup(list_item->name);
+	char *sub_git_dir = xstrfmt("%s/.git", sm_path);
+	struct stat st;
+
+	sub = submodule_from_path(null_sha1, sm_path);
+
+	if (!sub || !sub->name)
+		goto cleanup;
+
+	displaypath = get_submodule_displaypath(sm_path, info->prefix);
+
+	/* remove the submodule work tree (unless the user already did it) */
+	if (is_directory(sm_path)) {
+		/* protect submodules containing a .git directory */
+		if (is_git_directory(sub_git_dir))
+			die(_("Submodule work tree '%s' contains a .git "
+			      "directory use 'rm -rf' if you really want "
+			      "to remove it including all of its history"),
+			      displaypath);
+
+		if (!info->force) {
+			struct child_process cp_rm = CHILD_PROCESS_INIT;
+			cp_rm.git_cmd = 1;
+			argv_array_pushl(&cp_rm.args, "rm", "-qn", sm_path,
+					 NULL);
+
+			/* list_item->name is changed by cmd_rm() below */
+			if (run_command(&cp_rm))
+				die(_("Submodule work tree '%s' contains local "
+				      "modifications; use '-f' to discard them"),
+				      displaypath);
+		}
+
+		if (!lstat(sm_path, &st)) {
+			struct strbuf sb_rm = STRBUF_INIT;
+			strbuf_addstr(&sb_rm, sm_path);
+
+			if (!remove_dir_recursively(&sb_rm, 0)) {
+				if (!info->quiet)
+					printf(_("Cleared directory '%s'\n"),
+						 displaypath);
+			} else {
+				if (!info->quiet)
+					printf(_("Could not remove submodule work tree '%s'\n"),
+						 displaypath);
+			}
+			strbuf_release(&sb_rm);
+		}
+	}
+
+	if (mkdir(sm_path, st.st_mode))
+		die(_("could not create empty submodule directory %s"),
+		      displaypath);
+
+	cp_config.git_cmd = 1;
+	argv_array_pushl(&cp_config.args, "config", "--get-regexp", NULL);
+	argv_array_pushf(&cp_config.args, "submodule.%s\\.", sub->name);
+
+	/* remove the .git/config entries (unless the user already did it) */
+	if (!capture_command(&cp_config, &sb_config, 0) && sb_config.len) {
+		char *sub_key = xstrfmt("submodule.%s", sub->name);
+		/*
+		 * remove the whole section so we have a clean state when
+		 * the user later decides to init this submodule again
+		 */
+		git_config_rename_section_in_file(NULL, sub_key, NULL);
+		if (!info->quiet)
+			printf(_("Submodule '%s' (%s) unregistered for path '%s'\n"),
+				 sub->name, sub->url, displaypath);
+		free(sub_key);
+	}
+
+cleanup:
+	free(displaypath);
+	free(sub_git_dir);
+	free(sm_path);
+	strbuf_release(&sb_config);
+}
+
+static int module_deinit(int argc, const char **argv, const char *prefix)
+{
+	struct deinit_cb info = DEINIT_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int force = 0;
+	int all = 0;
+
+	struct option module_deinit_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT__FORCE(&force, N_("Remove submodule working trees even if they contain local changes")),
+		OPT_BOOL(0, "all", &all, N_("Unregister all submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule deinit [--quiet] [-f | --force] [--all | [--] [<path>...]]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_deinit_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.all = !!all;
+	info.force = !!force;
+
+	if (all && argc) {
+		error("pathspec and --all are incompatible");
+		usage_with_options(git_submodule_helper_usage,
+				   module_deinit_options);
+	}
+
+	if (!argc && !all)
+		die(_("Use '--all' if you really want to deinitialize all submodules"));
+
+	gitmodules_config();
+	for_each_submodule_list(list, deinit_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1631,6 +1773,7 @@ static struct cmd_struct commands[] = {
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
+	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 6bfc5e17d..73e6f093f 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -428,60 +428,7 @@ cmd_deinit()
 		shift
 	done
 
-	if test -n "$deinit_all" && test "$#" -ne 0
-	then
-		echo >&2 "$(eval_gettext "pathspec and --all are incompatible")"
-		usage
-	fi
-	if test $# = 0 && test -z "$deinit_all"
-	then
-		die "$(eval_gettext "Use '--all' if you really want to deinitialize all submodules")"
-	fi
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-
-		displaypath=$(git submodule--helper relative-path "$sm_path" "$wt_prefix")
-
-		# Remove the submodule work tree (unless the user already did it)
-		if test -d "$sm_path"
-		then
-			# Protect submodules containing a .git directory
-			if test -d "$sm_path/.git"
-			then
-				die "$(eval_gettext "\
-Submodule work tree '\$displaypath' contains a .git directory
-(use 'rm -rf' if you really want to remove it including all of its history)")"
-			fi
-
-			if test -z "$force"
-			then
-				git rm -qn "$sm_path" ||
-				die "$(eval_gettext "Submodule work tree '\$displaypath' contains local modifications; use '-f' to discard them")"
-			fi
-			rm -rf "$sm_path" &&
-			say "$(eval_gettext "Cleared directory '\$displaypath'")" ||
-			say "$(eval_gettext "Could not remove submodule work tree '\$displaypath'")"
-		fi
-
-		mkdir "$sm_path" || say "$(eval_gettext "Could not create empty submodule directory '\$displaypath'")"
-
-		# Remove the .git/config entries (unless the user already did it)
-		if test -n "$(git config --get-regexp submodule."$name\.")"
-		then
-			# Remove the whole section so we have a clean state when
-			# the user later decides to init this submodule again
-			url=$(git config submodule."$name".url)
-			git config --remove-section submodule."$name" 2>/dev/null &&
-			say "$(eval_gettext "Submodule '\$name' (\$url) unregistered for path '\$displaypath'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${force:+--force} ${deinit_all:+--all} "$@"
 }
 
 is_tip_reachable () (
-- 
2.13.0


^ permalink raw reply	[relevance 20%]

* [GSoC][PATCH 8/8] submodule: port submodule subcommand 'summary' from shell to C
  2017-07-18 20:48 [GSoC][PATCH 0/8] Update: Week 9 Prathamesh Chavan
                   ` (5 preceding siblings ...)
  2017-07-18 20:49 ` [GSoC][PATCH 6/8] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
@ 2017-07-18 20:49 ` " Prathamesh Chavan
  6 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-18 20:49 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The submodule subcommand 'summary' is ported in the process of
making git-submodule a builtin. The function cmd_summary() from
git-submodule.sh is ported to functions module_summary(),
compute_summary_module_list(), prepare_submodule_summary() and
print_submodule_summary().

The first function module_summary() parses the options of submodule
subcommand and also acts as the front-end of this subcommand.
After parsing them, it calls the compute_summary_module_list()

The functions compute_summary_module_list() runs the diff_cmd,
and generates the modules list, as required by the subcommand.
The generation of this module list is done by the using the
callback function submodule_summary_callback(), and stored in the
structure module_cb.

Once the module list is generated, prepare_submodule_summary()
further goes through the list and filters the list, for
eventually calling the print_submodule_summary() function.

Finally, the print_submodule_summary() takes care of generating
and printing the summary for each submodule.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version of patch,
instead of adding the GIT_DIR to the env_variables of child_process,
we use the function prepare_submodule_repo_env().

Complete build report of this series is available at:
https://travis-ci.org/pratham-pc/git/builds/
Branch: week-9
Build #135

 builtin/submodule--helper.c | 469 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 182 +----------------
 2 files changed, 470 insertions(+), 181 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 8a679abf6..c438a922d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -13,6 +13,9 @@
 #include "remote.h"
 #include "refs.h"
 #include "connect.h"
+#include "revision.h"
+#include "diffcore.h"
+#include "diff.h"
 
 typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
 				      void *cb_data);
@@ -757,6 +760,471 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct module_cb {
+	unsigned int mod_src;
+	unsigned int mod_dst;
+	struct object_id oid_src;
+	struct object_id oid_dst;
+	char status;
+	const char *sm_path;
+};
+#define MODULE_CB_INIT { 0, 0, NULL, NULL, '\0', NULL }
+
+struct module_cb_list {
+	struct module_cb **entries;
+	int alloc, nr;
+};
+#define MODULE_CB_LIST_INIT { NULL, 0, 0 }
+
+struct summary_cb {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	char *diff_cmd;
+	unsigned int cached: 1;
+	unsigned int for_status: 1;
+	unsigned int quiet: 1;
+	unsigned int files: 1;
+	int summary_limits;
+};
+#define SUMMARY_CB_INIT { 0, NULL, NULL, NULL, 0, 0, 0, 0, 0 }
+
+static void print_submodule_summary(struct summary_cb *info,
+				    struct module_cb *p)
+{
+	int missing_src = 0;
+	int missing_dst = 0;
+	char *displaypath;
+	char *sha1_abbr_src;
+	char *sha1_abbr_dst;
+	int errmsg = 0;
+	int total_commits = -1;
+	struct strbuf sb_sha1_src = STRBUF_INIT;
+	struct strbuf sb_sha1_dst = STRBUF_INIT;
+	char *sha1_dst = oid_to_hex(&p->oid_dst);
+	char *sha1_src = oid_to_hex(&p->oid_src);
+	char *sm_git_dir = xstrfmt("%s/.git", p->sm_path);
+	int is_sm_git_dir = 0;
+
+	if (!info->cached && !strcmp(sha1_dst, sha1_to_hex(null_sha1))) {
+		if (S_ISGITLINK(p->mod_dst)) {
+			struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_parse = STRBUF_INIT;
+
+			cp_rev_parse.git_cmd = 1;
+			cp_rev_parse.no_stderr = 1;
+			cp_rev_parse.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+			argv_array_pushl(&cp_rev_parse.args,
+					 "rev-parse", "HEAD", NULL);
+			if (!capture_command(&cp_rev_parse, &sb_rev_parse, 0)) {
+				strbuf_strip_suffix(&sb_rev_parse, "\n");
+				sha1_dst = xstrdup(sb_rev_parse.buf);
+			}
+			strbuf_release(&sb_rev_parse);
+		} else if (S_ISLNK(p->mod_dst) || S_ISREG(p->mod_dst)) {
+			struct child_process cp_hash_object = CHILD_PROCESS_INIT;
+			struct strbuf sb_hash_object = STRBUF_INIT;
+
+			cp_hash_object.git_cmd = 1;
+			argv_array_pushl(&cp_hash_object.args,
+					 "hash-object", p->sm_path,
+					 NULL);
+			if (!capture_command(&cp_hash_object,
+					     &sb_hash_object, 0)) {
+				strbuf_strip_suffix(&sb_hash_object, "\n");
+				sha1_dst = xstrdup(sb_hash_object.buf);
+			}
+			strbuf_release(&sb_hash_object);
+		} else {
+			if (p->mod_dst != 0)
+				die(_("unexpected mode %d\n"), p->mod_dst);
+		}
+	}
+
+	if (is_git_directory(sm_git_dir))
+		is_sm_git_dir = 1;
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_src)) {
+		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+		cp_rev_parse.git_cmd = 1;
+		cp_rev_parse.no_stdout = 1;
+		cp_rev_parse.dir = p->sm_path;
+		prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+		argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
+				 "--verify", NULL);
+		argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1_src);
+
+		if (run_command(&cp_rev_parse))
+			missing_src = 1;
+	}
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_dst)) {
+		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+		cp_rev_parse.git_cmd = 1;
+		cp_rev_parse.no_stdout = 1;
+		cp_rev_parse.dir = p->sm_path;
+		prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+		argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
+				 "--verify", NULL);
+		argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1_dst);
+
+		if (run_command(&cp_rev_parse))
+			missing_dst = 1;
+	}
+
+	displaypath = get_submodule_displaypath(p->sm_path, info->prefix);
+
+	if (!missing_dst && !missing_src) {
+		if (is_sm_git_dir) {
+			struct child_process cp_rev_list = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_list = STRBUF_INIT;
+			const char *range;
+
+			if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst))
+				range = xstrfmt("%s...%s", sha1_src, sha1_dst);
+			else if (S_ISGITLINK(p->mod_src))
+				range = xstrdup(sha1_src);
+			else
+				range = xstrdup(sha1_dst);
+
+			cp_rev_list.git_cmd = 1;
+			cp_rev_list.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_list.env_array);
+
+			argv_array_pushl(&cp_rev_list.args, "rev-list",
+					 "--first-parent", range, "--", NULL);
+			if (!capture_command(&cp_rev_list, &sb_rev_list, 0)) {
+				if (sb_rev_list.len)
+					total_commits = count_lines(sb_rev_list.buf,
+								    sb_rev_list.len);
+				else
+					total_commits = 0;
+			}
+			strbuf_release(&sb_rev_list);
+		}
+	} else {
+		errmsg = 1;
+	}
+
+	strbuf_addstr(&sb_sha1_src, sha1_src);
+	strbuf_addstr(&sb_sha1_dst, sha1_dst);
+
+	strbuf_remove(&sb_sha1_src, 7, 33);
+	strbuf_remove(&sb_sha1_dst, 7, 33);
+
+	sha1_abbr_src = strbuf_detach(&sb_sha1_src, NULL);
+	sha1_abbr_dst = strbuf_detach(&sb_sha1_dst, NULL);
+
+	if (p->status == 'T') {
+		if (S_ISGITLINK(p->mod_dst)) {
+			if (total_commits < 0)
+				printf(_("* %s %s(blob)->%s(submodule):\n"),
+					 displaypath, sha1_abbr_src,
+					 sha1_abbr_dst);
+			else
+				printf(_("* %s %s(blob)->%s(submodule) (%d):\n"),
+					 displaypath, sha1_abbr_src,
+					 sha1_abbr_dst, total_commits);
+		} else {
+			if (total_commits < 0)
+				printf(_("* %s %s(submodule)->%s(blob):\n"),
+					 displaypath, sha1_abbr_src,
+					 sha1_abbr_dst);
+			else
+				printf(_("* %s %s(submodule)->%s(blob) (%d):\n"),
+					 displaypath, sha1_abbr_src,
+					 sha1_abbr_dst, total_commits);
+		}
+	} else {
+		if (total_commits < 0)
+			printf("* %s %s...%s:\n", displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+		else
+			printf("* %s %s...%s (%d):\n", displaypath,
+				 sha1_abbr_src, sha1_abbr_dst, total_commits);
+	}
+
+	if (errmsg) {
+		/*
+		 * Don't give error msg for modification whose dst is not
+		 * submodule, i.e. deleted or changed to blob
+		 */
+		if (S_ISGITLINK(p->mod_src)) {
+			if (missing_src && missing_dst) {
+				printf(_("  Warn: %s doesn't contain commits %s and %s\n"),
+				 displaypath, sha1_src, sha1_dst);
+			} else if (missing_src) {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, sha1_src);
+			} else {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, sha1_dst);
+			}
+		}
+
+	} else if (is_sm_git_dir) {
+		if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst)) {
+			struct child_process cp_log = CHILD_PROCESS_INIT;
+			char *limit = NULL;
+
+			if (info->summary_limits > 0)
+				limit = xstrfmt("-%d", info->summary_limits);
+
+			cp_log.git_cmd = 1;
+			cp_log.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_log.env_array);
+
+			argv_array_pushl(&cp_log.args, "log", NULL);
+			if (limit)
+				argv_array_push(&cp_log.args, limit);
+			argv_array_pushl(&cp_log.args, "--pretty=  %m %s",
+					 "--first-parent", NULL);
+			argv_array_pushf(&cp_log.args, "%s...%s", sha1_src,
+					 sha1_dst);
+
+			run_command(&cp_log);
+
+		} else if (S_ISGITLINK(p->mod_dst)) {
+			struct child_process cp_log = CHILD_PROCESS_INIT;
+
+			cp_log.git_cmd = 1;
+			cp_log.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_log.env_array);
+
+			argv_array_pushl(&cp_log.args, "log",
+					 "--pretty=  > %s", "-1",
+					 sha1_dst, NULL);
+
+			run_command(&cp_log);
+		} else {
+			struct child_process cp_log = CHILD_PROCESS_INIT;
+
+			cp_log.git_cmd = 1;
+			cp_log.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_log.env_array);
+
+			argv_array_pushl(&cp_log.args, "log",
+					 "--pretty=  < %s",
+					 "-1", sha1_src, NULL);
+
+			run_command(&cp_log);
+		}
+	}
+	printf("\n");
+}
+
+static void prepare_submodule_summary(struct summary_cb *info,
+				      struct module_cb_list *list)
+{
+	int i;
+	for (i = 0; i < list->nr; i++) {
+		struct module_cb *p = list->entries[i];
+		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+		if (p->status == 'D' || p->status == 'T') {
+			print_submodule_summary(info, p);
+			continue;
+		}
+
+		if (info->for_status) {
+			char *config_key;
+			const char *ignore_config = "none";
+			const char *value;
+			const struct submodule *sub = submodule_from_path(null_sha1, p->sm_path);
+
+			if (sub) {
+				config_key = xstrfmt("submodule.%s.ignore",
+						     sub->name);
+				if (!git_config_get_value(config_key, &value))
+					ignore_config = value;
+				else if (sub->ignore)
+					ignore_config = sub->ignore;
+
+				if (p->status != 'A' && !strcmp(ignore_config,
+								"all"))
+					continue;
+			}
+		}
+
+		/* Also show added or modified modules which are checked out */
+		cp_rev_parse.dir = p->sm_path;
+		cp_rev_parse.git_cmd = 1;
+		cp_rev_parse.no_stderr = 1;
+		cp_rev_parse.no_stdout = 1;
+
+		argv_array_pushl(&cp_rev_parse.args, "rev-parse",
+				 "--git-dir", NULL);
+
+		if (!run_command(&cp_rev_parse))
+			print_submodule_summary(info, p);
+	}
+}
+
+static void submodule_summary_callback(struct diff_queue_struct *q,
+				       struct diff_options *options,
+				       void *data)
+{
+	int i;
+	struct module_cb_list *list = data;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		struct module_cb *temp;
+
+		if (!S_ISGITLINK(p->one->mode) && !S_ISGITLINK(p->two->mode))
+			continue;
+		temp = (struct module_cb*)malloc(sizeof(struct module_cb));
+		temp->mod_src = p->one->mode;
+		temp->mod_dst = p->two->mode;
+		temp->oid_src = p->one->oid;
+		temp->oid_dst = p->two->oid;
+		temp->status = p->status;
+		temp->sm_path = xstrdup(p->one->path);
+
+		ALLOC_GROW(list->entries, list->nr + 1, list->alloc);
+		list->entries[list->nr++] = temp;
+	}
+}
+
+static int compute_summary_module_list(char *head, struct summary_cb *info)
+{
+	struct argv_array diff_args = ARGV_ARRAY_INIT;
+	struct rev_info rev;
+	struct module_cb_list list = MODULE_CB_LIST_INIT;
+
+	argv_array_push(&diff_args, info->diff_cmd);
+	if (info->cached)
+		argv_array_push(&diff_args, "--cached");
+	argv_array_pushl(&diff_args, "--ignore-submodules=dirty", "--raw",
+			 NULL);
+	if (head)
+		argv_array_push(&diff_args, head);
+	argv_array_push(&diff_args, "--");
+	if (info->argc)
+		argv_array_pushv(&diff_args, info->argv);
+
+	git_config(git_diff_basic_config, NULL);
+	init_revisions(&rev, info->prefix);
+	gitmodules_config();
+	rev.abbrev = 0;
+	precompose_argv(diff_args.argc, diff_args.argv);
+
+	diff_args.argc = setup_revisions(diff_args.argc, diff_args.argv,
+					 &rev, NULL);
+	rev.diffopt.output_format = DIFF_FORMAT_NO_OUTPUT | DIFF_FORMAT_CALLBACK;
+	rev.diffopt.format_callback = submodule_summary_callback;
+	rev.diffopt.format_callback_data = &list;
+
+	if (!info->cached) {
+		if (!strcmp(info->diff_cmd, "diff-index"))
+			setup_work_tree();
+		if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
+			perror("read_cache_preload");
+			return -1;
+		}
+	} else if (read_cache() < 0) {
+		perror("read_cache");
+		return -1;
+	}
+
+	if (!strcmp(info->diff_cmd, "diff-index"))
+		run_diff_index(&rev, info->cached);
+	else
+		run_diff_files(&rev, 0);
+	prepare_submodule_summary(info, &list);
+	return 0;
+
+}
+
+static int module_summary(int argc, const char **argv, const char *prefix)
+{
+	struct summary_cb info = SUMMARY_CB_INIT;
+	int cached = 0;
+	char *diff_cmd = "diff-index";
+	int for_status = 0;
+	int quiet = 0;
+	int files = 0;
+	int summary_limits = -1;
+	char *head = NULL;
+	struct child_process cp_rev = CHILD_PROCESS_INIT;
+	struct strbuf sb_rev = STRBUF_INIT;
+
+	struct option module_summary_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
+		OPT_BOOL(0, "cached", &cached, N_("Use the commit stored in the index instead of the submodule HEAD")),
+		OPT_BOOL(0, "files", &files, N_("To compares the commit in the index with that in the submodule HEAD")),
+		OPT_BOOL(0, "for-status", &for_status, N_("Skip submodules with 'all' ignore_config value")),
+		OPT_INTEGER('n', "summary-limits", &summary_limits, N_("Limit the summary size")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper summary [<options>] [--] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_summary_options,
+			     git_submodule_helper_usage, 0);
+
+	if (!summary_limits)
+		return 0;
+
+	cp_rev.git_cmd = 1;
+	argv_array_pushl(&cp_rev.args, "rev-parse", "-q", "--verify",
+			 NULL);
+	if (argc)
+		argv_array_push(&cp_rev.args, argv[0]);
+	else
+		argv_array_pushl(&cp_rev.args, "HEAD", NULL);
+
+	if (!capture_command(&cp_rev, &sb_rev, 0)) {
+		strbuf_strip_suffix(&sb_rev, "\n");
+		head = xstrdup(sb_rev.buf);
+		if (argc) {
+			argv++;
+			argc--;
+		}
+		strbuf_release(&sb_rev);
+	} else if (!argc || !strcmp(argv[0], "HEAD")) {
+		/* before the first commit: compare with an empty tree */
+		struct stat st;
+		unsigned char sha1[20];
+		if (fstat(0, &st) < 0 || index_fd(sha1, 0, &st, 2, prefix, 3))
+			die("Unable to add %s to database", sha1);
+		head = sha1_to_hex(sha1);
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else {
+		head = "HEAD";
+	}
+
+	if (files) {
+		if (cached)
+			die(_("The --cached option cannot be used with the --files option"));
+		diff_cmd = "diff-files";
+		head = NULL;
+	}
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.cached = cached;
+	info.for_status = for_status;
+	info.quiet = quiet;
+	info.files = files;
+	info.summary_limits = summary_limits;
+	info.diff_cmd = diff_cmd;
+
+	return compute_summary_module_list(head, &info);
+}
+
 struct sync_cb {
 	const char *prefix;
 	unsigned int quiet: 1;
@@ -1774,6 +2242,7 @@ static struct cmd_struct commands[] = {
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
+	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 73e6f093f..a427ddafd 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -51,31 +51,6 @@ die_if_unmatched ()
 	fi
 }
 
-#
-# Print a submodule configuration setting
-#
-# $1 = submodule name
-# $2 = option name
-# $3 = default value
-#
-# Checks in the usual git-config places first (for overrides),
-# otherwise it falls back on .gitmodules.  This allows you to
-# distribute project-wide defaults in .gitmodules, while still
-# customizing individual repositories if necessary.  If the option is
-# not in .gitmodules either, print a default value.
-#
-get_submodule_config () {
-	name="$1"
-	option="$2"
-	default="$3"
-	value=$(git config submodule."$name"."$option")
-	if test -z "$value"
-	then
-		value=$(git config -f .gitmodules submodule."$name"."$option")
-	fi
-	printf '%s' "${value:-$default}"
-}
-
 isnumber()
 {
 	n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
@@ -755,163 +730,8 @@ cmd_summary() {
 		shift
 	done
 
-	test $summary_limit = 0 && return
-
-	if rev=$(git rev-parse -q --verify --default HEAD ${1+"$1"})
-	then
-		head=$rev
-		test $# = 0 || shift
-	elif test -z "$1" || test "$1" = "HEAD"
-	then
-		# before the first commit: compare with an empty tree
-		head=$(git hash-object -w -t tree --stdin </dev/null)
-		test -z "$1" || shift
-	else
-		head="HEAD"
-	fi
-
-	if [ -n "$files" ]
-	then
-		test -n "$cached" &&
-		die "$(gettext "The --cached option cannot be used with the --files option")"
-		diff_cmd=diff-files
-		head=
-	fi
-
-	cd_to_toplevel
-	eval "set $(git rev-parse --sq --prefix "$wt_prefix" -- "$@")"
-	# Get modified modules cared by user
-	modules=$(git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- "$@" |
-		sane_egrep '^:([0-7]* )?160000' |
-		while read -r mod_src mod_dst sha1_src sha1_dst status sm_path
-		do
-			# Always show modules deleted or type-changed (blob<->module)
-			if test "$status" = D || test "$status" = T
-			then
-				printf '%s\n' "$sm_path"
-				continue
-			fi
-			# Respect the ignore setting for --for-status.
-			if test -n "$for_status"
-			then
-				name=$(git submodule--helper name "$sm_path")
-				ignore_config=$(get_submodule_config "$name" ignore none)
-				test $status != A && test $ignore_config = all && continue
-			fi
-			# Also show added or modified modules which are checked out
-			GIT_DIR="$sm_path/.git" git-rev-parse --git-dir >/dev/null 2>&1 &&
-			printf '%s\n' "$sm_path"
-		done
-	)
-
-	test -z "$modules" && return
-
-	git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- $modules |
-	sane_egrep '^:([0-7]* )?160000' |
-	cut -c2- |
-	while read -r mod_src mod_dst sha1_src sha1_dst status name
-	do
-		if test -z "$cached" &&
-			test $sha1_dst = 0000000000000000000000000000000000000000
-		then
-			case "$mod_dst" in
-			160000)
-				sha1_dst=$(GIT_DIR="$name/.git" git rev-parse HEAD)
-				;;
-			100644 | 100755 | 120000)
-				sha1_dst=$(git hash-object $name)
-				;;
-			000000)
-				;; # removed
-			*)
-				# unexpected type
-				eval_gettextln "unexpected mode \$mod_dst" >&2
-				continue ;;
-			esac
-		fi
-		missing_src=
-		missing_dst=
-
-		test $mod_src = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_src^0 >/dev/null &&
-		missing_src=t
-
-		test $mod_dst = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_dst^0 >/dev/null &&
-		missing_dst=t
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${for_status:+--for-status} ${files:+--files} ${cached:+--cached} ${summary_limit:+-n $summary_limit} "$@"
 
-		display_name=$(git submodule--helper relative-path "$name" "$wt_prefix")
-
-		total_commits=
-		case "$missing_src,$missing_dst" in
-		t,)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_src")"
-			;;
-		,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_dst")"
-			;;
-		t,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commits \$sha1_src and \$sha1_dst")"
-			;;
-		*)
-			errmsg=
-			total_commits=$(
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				range="$sha1_src...$sha1_dst"
-			elif test $mod_src = 160000
-			then
-				range=$sha1_src
-			else
-				range=$sha1_dst
-			fi
-			GIT_DIR="$name/.git" \
-			git rev-list --first-parent $range -- | wc -l
-			)
-			total_commits=" ($(($total_commits + 0)))"
-			;;
-		esac
-
-		sha1_abbr_src=$(echo $sha1_src | cut -c1-7)
-		sha1_abbr_dst=$(echo $sha1_dst | cut -c1-7)
-		if test $status = T
-		then
-			blob="$(gettext "blob")"
-			submodule="$(gettext "submodule")"
-			if test $mod_dst = 160000
-			then
-				echo "* $display_name $sha1_abbr_src($blob)->$sha1_abbr_dst($submodule)$total_commits:"
-			else
-				echo "* $display_name $sha1_abbr_src($submodule)->$sha1_abbr_dst($blob)$total_commits:"
-			fi
-		else
-			echo "* $display_name $sha1_abbr_src...$sha1_abbr_dst$total_commits:"
-		fi
-		if test -n "$errmsg"
-		then
-			# Don't give error msg for modification whose dst is not submodule
-			# i.e. deleted or changed to blob
-			test $mod_dst = 160000 && echo "$errmsg"
-		else
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				limit=
-				test $summary_limit -gt 0 && limit="-$summary_limit"
-				GIT_DIR="$name/.git" \
-				git log $limit --pretty='format:  %m %s' \
-				--first-parent $sha1_src...$sha1_dst
-			elif test $mod_dst = 160000
-			then
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  > %s' -1 $sha1_dst
-			else
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  < %s' -1 $sha1_src
-			fi
-			echo
-		fi
-		echo
-	done
 }
 #
 # List all submodules, prefixed with:
-- 
2.13.0


^ permalink raw reply	[relevance 15%]

* Re: [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' from shell to C
  2017-07-18 20:49 ` [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-07-18 21:39   ` Stefan Beller
  2017-07-18 22:32     ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-18 21:39 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Tue, Jul 18, 2017 at 1:49 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> This aims to make git-submodule 'status' a built-in. Hence, the function
> cmd_status() is ported from shell to C. This is done by introducing
> three functions: module_status(), submodule_status() and print_status().
>
> The function module_status() acts as the front-end of the subcommand.
> It parses subcommand's options and then calls the function
> module_list_compute() for computing the list of submodules. Then
> this functions calls for_each_submodule_list() looping through the
> list obtained.
>
> Then for_each_submodule_list() calls submodule_status() for each of the
> submodule in its list. The function submodule_status() is responsible
> for generating the status each submodule it is called for, and
> then calls print_status().
>
> Finally, the function print_status() handles the printing of submodule's
> status.
>
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
>  builtin/submodule--helper.c | 146 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            |  49 +--------------
>  2 files changed, 147 insertions(+), 48 deletions(-)
>
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 80f744407..9c1630495 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -560,6 +560,151 @@ static int module_init(int argc, const char **argv, const char *prefix)
>         return 0;
>  }
>
> +struct status_cb {
> +       const char *prefix;
> +       unsigned int quiet: 1;
> +       unsigned int recursive: 1;
> +       unsigned int cached: 1;
> +};
> +#define STATUS_CB_INIT { NULL, 0, 0, 0 }
> +
> +static void print_status(struct status_cb *info, char state, const char *path,
> +                        char *sub_sha1, char *displaypath)
> +{
> +       if (info->quiet)
> +               return;
> +
> +       printf("%c%s %s", state, sub_sha1, displaypath);
> +
> +       if (state == ' ' || state == '+') {
> +               struct argv_array name_rev_args = ARGV_ARRAY_INIT;
> +
> +               argv_array_pushl(&name_rev_args, "print-name-rev",
> +                                path, sub_sha1, NULL);
> +               print_name_rev(name_rev_args.argc, name_rev_args.argv,
> +                              info->prefix);
> +       } else {
> +               printf("\n");
> +       }
> +}
> +
> +static int handle_submodule_head_ref(const char *refname,
> +                                    const struct object_id *oid, int flags,
> +                                    void *cb_data)
> +{
> +       struct strbuf *output = cb_data;
> +       if (oid)
> +               strbuf_addstr(output, oid_to_hex(oid));
> +       return 0;
> +}
> +
> +static void status_submodule(const struct cache_entry *list_item, void *cb_data)
> +{
> +       struct status_cb *info = cb_data;
> +       char *sub_sha1 = xstrdup(oid_to_hex(&list_item->oid));
> +       char *displaypath;
> +       struct stat st;
> +
> +       if (!submodule_from_path(null_sha1, list_item->name))
> +               die(_("no submodule mapping found in .gitmodules for path '%s'"),
> +                     list_item->name);
> +
> +       displaypath = get_submodule_displaypath(list_item->name, info->prefix);
> +
> +       if (list_item->ce_flags) {
> +               print_status(info, 'U', list_item->name,
> +                            sha1_to_hex(null_sha1), displaypath);
> +               goto cleanup;
> +       }
> +
> +       if (!is_submodule_active(the_repository, list_item->name)) {
> +               print_status(info, '-', list_item->name, sub_sha1, displaypath);
> +               goto cleanup;
> +       }
> +
> +       if (!lstat(list_item->name, &st) && !ce_match_stat(list_item, &st, 0)) {
> +               print_status(info, ' ', list_item->name, sub_sha1, displaypath);

The question from the last round still stands
https://public-inbox.org/git/CAGZ79kb18z5zc9iu3Vv5aVZWJmoZzmwbMVpy89VC-t-ei2M+bw@mail.gmail.com/

  I am not an expert in the diff area  and wonder how
  the cmd_diff_files functionality is achieved with just a stat call
  and then comparing it to  ce_match_stat. 'Using "dirty" ignores
  all changes to the work tree of submodules, only changes to the
  commits stored in the superproject are shown.' So I'd have
  expected ce->oid to be compared (is there an index entry differing,
  i.e. more than one stage?)

^ permalink raw reply	[relevance 25%]

* Re: [GSoC][PATCH 5/8] submodule: port submodule subcommand 'sync' from shell to C
  2017-07-18 20:49 ` [GSoC][PATCH 5/8] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-07-18 22:23   ` Stefan Beller
      [irrelevant]     ` <CAME+mvV9-spX0DrToZdtydhZ5cXA5A7R7JCca4f87Rn3tdLUpw@mail.gmail.com>
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-18 22:23 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Tue, Jul 18, 2017 at 1:49 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> Port the submodule subcommand 'sync' from shell to C using the same
> mechanism as that used for porting submodule subcommand 'status'.
> Hence, here the function cmd_sync() is ported from shell to C.
> This is done by introducing three functions: module_sync(),
> sync_submodule() and print_default_remote().
>
> The function print_default_remote() is introduced for getting
> the default remote as stdout.
>
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
>  builtin/submodule--helper.c | 179 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            |  56 +-------------
>  2 files changed, 180 insertions(+), 55 deletions(-)
>
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 9c1630495..da91c489b 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -44,6 +44,20 @@ static char *get_default_remote(void)
>         return ret;
>  }
>
> +static int print_default_remote(int argc, const char **argv, const char *prefix)
> +{
> +       const char *remote;
> +
> +       if (argc != 1)
> +               die(_("submodule--helper print-default-remote takes no arguments"));
> +
> +       remote = get_default_remote();
> +       if (remote)
> +               puts(remote);
> +
> +       return 0;
> +}
> +
>  static int starts_with_dot_slash(const char *str)
>  {
>         return str[0] == '.' && is_dir_sep(str[1]);
> @@ -379,6 +393,25 @@ static void module_list_active(struct module_list *list)
>         *list = active_modules;
>  }
>
> +static char *get_up_path(const char *path)
> +{
> +       int i;
> +       struct strbuf sb = STRBUF_INIT;
> +
> +       for (i = count_slashes(path); i; i--)
> +               strbuf_addstr(&sb, "../");
> +
> +       /*
> +        * Check if 'path' ends with slash or not
> +        * for having the same output for dir/sub_dir
> +        * and dir/sub_dir/
> +        */
> +       if (!is_dir_sep(path[strlen(path) - 1]))
> +               strbuf_addstr(&sb, "../");
> +
> +       return strbuf_detach(&sb, NULL);
> +}
> +
>  static int module_list(int argc, const char **argv, const char *prefix)
>  {
>         int i;
> @@ -724,6 +757,150 @@ static int module_name(int argc, const char **argv, const char *prefix)
>         return 0;
>  }
>
> +struct sync_cb {
> +       const char *prefix;
> +       unsigned int quiet: 1;
> +       unsigned int recursive: 1;
> +};
> +#define SYNC_CB_INIT { NULL, 0, 0 }
> +
> +static void sync_submodule(const struct cache_entry *list_item, void *cb_data)
> +{
> +       struct sync_cb *info = cb_data;
> +       const struct submodule *sub;
> +       char *sub_key, *remote_key;
> +       char *sub_origin_url, *super_config_url, *displaypath;
> +       struct strbuf sb = STRBUF_INIT;
> +       struct child_process cp = CHILD_PROCESS_INIT;
> +
> +       if (!is_submodule_active(the_repository, list_item->name))
> +               return;

We can use the_repository here, as we also use child processes to
recurse, such that we always operate on the_repository as the
superproject.


> +
> +       sub = submodule_from_path(null_sha1, list_item->name);
> +
> +       if (!sub || !sub->url)
> +               die(_("no url found for submodule path '%s' in .gitmodules"),
> +                     list_item->name);

We do not die in the shell script when the url is missing in the
.gitmodules file.

> +
> +       if (starts_with_dot_dot_slash(sub->url) || starts_with_dot_slash(sub->url)) {
> +               char *remote_url, *up_path;
> +               char *remote = get_default_remote();
> +               char *remote_key = xstrfmt("remote.%s.url", remote);
> +
> +               if (git_config_get_string(remote_key, &remote_url))
> +                       remote_url = xgetcwd();
> +
> +               up_path = get_up_path(list_item->name);
> +               sub_origin_url = relative_url(remote_url, sub->url, up_path);
> +               super_config_url = relative_url(remote_url, sub->url, NULL);
> +
> +               free(remote);
> +               free(remote_key);
> +               free(up_path);
> +               free(remote_url);
> +       } else {
> +               sub_origin_url = xstrdup(sub->url);
> +               super_config_url = xstrdup(sub->url);
> +       }
> +
> +       displaypath = get_submodule_displaypath(list_item->name, info->prefix);
> +
> +       if (!info->quiet)
> +               printf(_("Synchronizing submodule url for '%s'\n"),
> +                        displaypath);
> +
> +       sub_key = xstrfmt("submodule.%s.url", sub->name);
> +       if (git_config_set_gently(sub_key, super_config_url))
> +               die(_("failed to register url for submodule path '%s'"),
> +                     displaypath);
> +
> +       if (!is_submodule_populated_gently(list_item->name, NULL))
> +               goto cleanup;
> +
> +       prepare_submodule_repo_env(&cp.env_array);
> +       cp.git_cmd = 1;
> +       cp.dir = list_item->name;
> +       argv_array_pushl(&cp.args, "submodule--helper",
> +                        "print-default-remote", NULL);
> +       if (capture_command(&cp, &sb, 0))
> +               die(_("failed to get the default remote for submodule '%s'"),
> +                     list_item->name);
> +
> +       strbuf_strip_suffix(&sb, "\n");
> +       remote_key = xstrfmt("remote.%s.url", sb.buf);
> +       strbuf_release(&sb);
> +
> +       child_process_init(&cp);
> +       prepare_submodule_repo_env(&cp.env_array);
> +       cp.git_cmd = 1;
> +       cp.dir = list_item->name;
> +       argv_array_pushl(&cp.args, "config", remote_key, sub_origin_url, NULL);
> +       if (run_command(&cp))
> +               die(_("failed to update remote for submodule '%s'"),
> +                     list_item->name);

While it is a strict conversion from the shell script, we could also
try to do this in-process:
1) we'd find out the submodules git dir using submodule_to_gitdir
2) construct the path the the config file as "%s/.gitconfig"
3) using git_config_set_in_file (which presumably takes file name,
  key and value) the value can be set

^ permalink raw reply	[relevance 25%]

* Re: [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' from shell to C
  2017-07-18 21:39   ` Stefan Beller
@ 2017-07-18 22:32     ` Junio C Hamano
  2017-07-18 22:44       ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-07-18 22:32 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Prathamesh Chavan, git\, Christian Couder

Stefan Beller <sbeller@google.com> writes:

>> +       if (!lstat(list_item->name, &st) && !ce_match_stat(list_item, &st, 0)) {
>> +               print_status(info, ' ', list_item->name, sub_sha1, displaypath);
>
> The question from the last round still stands
> https://public-inbox.org/git/CAGZ79kb18z5zc9iu3Vv5aVZWJmoZzmwbMVpy89VC-t-ei2M+bw@mail.gmail.com/
>
>   I am not an expert in the diff area  and wonder how
>   the cmd_diff_files functionality is achieved with just a stat call
>   and then comparing it to  ce_match_stat. 'Using "dirty" ignores
>   all changes to the work tree of submodules, only changes to the
>   commits stored in the superproject are shown.' So I'd have
>   expected ce->oid to be compared (is there an index entry differing,
>   i.e. more than one stage?)

ce_match_stat() calls into ce_compare_gitlink() for a 160000 entry,
which would resolve HEAD ref there and compares ce->oid with it.

But as you said, this is probably insufficient to emulate the
original.  Shouldn't it call into run_diff_files(), which is the
in-core way to run the equivalent of "diff-files"?

^ permalink raw reply	[relevance 18%]

* Re: [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' from shell to C
  2017-07-18 22:32     ` Junio C Hamano
@ 2017-07-18 22:44       ` Stefan Beller
  2017-07-18 22:47         ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-18 22:44 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Prathamesh Chavan, git, Christian Couder

On Tue, Jul 18, 2017 at 3:32 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>> +       if (!lstat(list_item->name, &st) && !ce_match_stat(list_item, &st, 0)) {
>>> +               print_status(info, ' ', list_item->name, sub_sha1, displaypath);
>>
>> The question from the last round still stands
>> https://public-inbox.org/git/CAGZ79kb18z5zc9iu3Vv5aVZWJmoZzmwbMVpy89VC-t-ei2M+bw@mail.gmail.com/
>>
>>   I am not an expert in the diff area  and wonder how
>>   the cmd_diff_files functionality is achieved with just a stat call
>>   and then comparing it to  ce_match_stat. 'Using "dirty" ignores
>>   all changes to the work tree of submodules, only changes to the
>>   commits stored in the superproject are shown.' So I'd have
>>   expected ce->oid to be compared (is there an index entry differing,
>>   i.e. more than one stage?)
>
> ce_match_stat() calls into ce_compare_gitlink() for a 160000 entry,
> which would resolve HEAD ref there and compares ce->oid with it.

Oh in that case this should be fine, as in the original we did
"git diff-files --ignore-submodules=dirty <path>",
which did precisely that.

> But as you said, this is probably insufficient to emulate the
> original.  Shouldn't it call into run_diff_files(), which is the
> in-core way to run the equivalent of "diff-files"?

Oh, your comment in [1] was related to cmd_diff_files,
which is more complicated than run_diff_files?
run_diff_files also iterates over all cache entries.
I think we need to be looking at match_stat_with_submodule
to figure out what we need to do.

[1] https://public-inbox.org/git/xmqq60fdoyyt.fsf@gitster.mtv.corp.google.com/

^ permalink raw reply	[relevance 25%]

* Re: [GSoC][PATCH 4/8] submodule: port submodule subcommand 'status' from shell to C
  2017-07-18 22:44       ` Stefan Beller
@ 2017-07-18 22:47         ` Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2017-07-18 22:47 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Prathamesh Chavan, git, Christian Couder

On Tue, Jul 18, 2017 at 3:44 PM, Stefan Beller <sbeller@google.com> wrote:
> On Tue, Jul 18, 2017 at 3:32 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>
>>>> +       if (!lstat(list_item->name, &st) && !ce_match_stat(list_item, &st, 0)) {
>>>> +               print_status(info, ' ', list_item->name, sub_sha1, displaypath);
>>>
>>> The question from the last round still stands
>>> https://public-inbox.org/git/CAGZ79kb18z5zc9iu3Vv5aVZWJmoZzmwbMVpy89VC-t-ei2M+bw@mail.gmail.com/
>>>
>>>   I am not an expert in the diff area  and wonder how
>>>   the cmd_diff_files functionality is achieved with just a stat call
>>>   and then comparing it to  ce_match_stat. 'Using "dirty" ignores
>>>   all changes to the work tree of submodules, only changes to the
>>>   commits stored in the superproject are shown.' So I'd have
>>>   expected ce->oid to be compared (is there an index entry differing,
>>>   i.e. more than one stage?)
>>
>> ce_match_stat() calls into ce_compare_gitlink() for a 160000 entry,
>> which would resolve HEAD ref there and compares ce->oid with it.
>
> Oh in that case this should be fine, as in the original we did
> "git diff-files --ignore-submodules=dirty <path>",
> which did precisely that.

Are you sure that this will work correctly when ce is unmerged?
run_diff_files() has quite a lot of code for that case.

^ permalink raw reply	[relevance 18%]

* Re: subrepo vs submodule
      [irrelevant] <CAHd499BovefnyWjauE0vNaofmvL8JBZCSzJ7w-cJ8Sm3eoJ4=w@mail.gmail.com>
@ 2017-07-19 17:43 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-19 17:43 UTC (permalink / raw)
  To: Robert Dailey; +Cc: Git

On Wed, Jul 19, 2017 at 7:27 AM, Robert Dailey <rcdailey.lists@gmail.com> wrote:
> So I found out about "subrepo" today: https://github.com/ingydotnet/git-subrepo
>
> I'm still reading about how it works internally, but what do you guys
> think about it?

Nice find!

From reading the toplevel ReadMe.pod, I have the impression
that it is closer to subtree than submodules as the content is
stored in the main repository using a .gitrepo file to indicate that
it is a special directory.

> Is it a more or less perfect alternative to submodules?

Maybe to subtrees. It does not offer the workflows that
submodules allow you:
  " Users get your repo and all your subrepos just
  by cloning your repo."

Submodules allow you to *not* get the content of a submodule.
There are multiple reasons for not wanting the content, including
* not needing it (e.g. documentation, hardware specific code
  for an architecture you don't have)
* not wanting it (e.g. large binary files yield bandwidth concerns)

"Upstream history (clone/pull) is condensed into a single commit."

Same for submodules, not so(?) for subtrees.

Depending on the point of view this is either a feature or a bug.
For clean history you may view this as a feature, but as soon as
you want to git-bisect, you'd wish for the most fine grained history
available.

> What would be a reason not to use it?

There are multiple things that should drive a decision. some:
* What do you need in terms of workflow (even just talking about
  "submodule vs subtree" you get different answers for different
  workflows), and then seeing how subrepo compares to the others.
* Can you use it? (does it work on your platform? subrepo is bash,
  whereas submodule, subtree are posix shell; how is it distributed?
  Some people are uneasy installing from random github repos on
  the web, so do you install it via the package manager of your
  distribution, has your organisation a package to use?)
* Ease of use, do you need to explain the workflow
  to collaborateurs?

^ permalink raw reply	[relevance 24%]

* Re: [GSoC][PATCH 5/8] submodule: port submodule subcommand 'sync' from shell to C
      [irrelevant]     ` <CAME+mvV9-spX0DrToZdtydhZ5cXA5A7R7JCca4f87Rn3tdLUpw@mail.gmail.com>
@ 2017-07-20 19:57       ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-20 19:57 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Thu, Jul 20, 2017 at 12:36 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> Firstly, thanks for reviewing my patches. I have even checked out the
> other reviews
> and improvised the other patches according to reviews as well.
> I had a few doubts about this one though.
>
>>> +       const struct submodule *sub;
>>> +       char *sub_key, *remote_key;
>>> +       char *sub_origin_url, *super_config_url, *displaypath;
>>> +       struct strbuf sb = STRBUF_INIT;
>>> +       struct child_process cp = CHILD_PROCESS_INIT;
>>> +
>>> +       if (!is_submodule_active(the_repository, list_item->name))
>>> +               return;
>>
>> We can use the_repository here, as we also use child processes to
>> recurse, such that we always operate on the_repository as the
>> superproject.
>>
>
> Sorry, but can you explain this a bit more?

Well that was more thinking out out, in the sense of explaining why
it is the correct thing to do.

As the recursion is handled via spawning processes, each process
has the_repository pointing at a different repository (and the correct
repository for each process), at least to my understanding.

>
>>
>>> +
>>> +       sub = submodule_from_path(null_sha1, list_item->name);
>>> +
>>> +       if (!sub || !sub->url)
>>> +               die(_("no url found for submodule path '%s' in .gitmodules"),
>>> +                     list_item->name);
>>
>> We do not die in the shell script when the url is missing in the
>> .gitmodules file.
>>
>
> Then to have a faithful conversion, IMO, deleting the above lines
> would be the correct way?

Well, then we may run into segfaults due to dereferencing a NULL pointer.
So we have to figure out, what the code actually does when there is
no URL set. According to my understanding this would

    url=$(git config -f .gitmodules --get submodule."$name".url)
    # second case, but empty vars:
    sub_origin_url="$url"
    super_config_url="$url"

    ....

The issue with this shell script is that there is no difference between
"" and NULL, so the place where we do

    sub_origin_url="$url"
    super_config_url="$url"

we would need to translate NULL -> empty string

>
>>> +
>>> +       prepare_submodule_repo_env(&cp.env_array);
>>> +       cp.git_cmd = 1;
>>> +       cp.dir = list_item->name;
>>> +       argv_array_pushl(&cp.args, "submodule--helper",
>>> +                        "print-default-remote", NULL);
>>> +       if (capture_command(&cp, &sb, 0))
>>> +               die(_("failed to get the default remote for submodule '%s'"),
>>> +                     list_item->name);
>>> +
>>> +       strbuf_strip_suffix(&sb, "\n");
>>> +       remote_key = xstrfmt("remote.%s.url", sb.buf);
>>> +       strbuf_release(&sb);
>>> +
>>> +       child_process_init(&cp);
>>> +       prepare_submodule_repo_env(&cp.env_array);
>>> +       cp.git_cmd = 1;
>>> +       cp.dir = list_item->name;
>>> +       argv_array_pushl(&cp.args, "config", remote_key, sub_origin_url, NULL);
>>> +       if (run_command(&cp))
>>> +               die(_("failed to update remote for submodule '%s'"),
>>> +                     list_item->name);
>>
>> While it is a strict conversion from the shell script, we could also
>> try to do this in-process:
>> 1) we'd find out the submodules git dir using submodule_to_gitdir
>> 2) construct the path the the config file as "%s/.gitconfig"
>> 3) using git_config_set_in_file (which presumably takes file name,
>>   key and value) the value can be set
>
> Thanks for pointing that out. That surely reduced a child_process.
> Although the path of the config file for the case of submodules
> would be constructed by "%s/config".

Ah yes, that is correct.


>
> Thanks,
> Prathamesh Chavan

^ permalink raw reply	[relevance 24%]

* [PATCH] recursive submodules: detach HEAD from new state
@ 2017-07-24 17:36 Stefan Beller
  2017-07-24 18:03 ` Jonathan Nieder
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-24 17:36 UTC (permalink / raw)
  To: git; +Cc: gitster, jrnieder, Stefan Beller

When a submodule is on a branch and in its superproject you run a
recursive checkout, the branch of the submodule is updated to what the
superproject checks out. This is very unexpected in the current model of
Git as e.g. 'submodule update' always detaches the submodule HEAD.

Despite having plans to have submodule HEADS not detached in the future,
the current behavior is really bad as it doesn't match user expectations
and it is not checking for loss of commits (only to be recovered via the
reflog).

Detach the HEAD unconditionally in the submodule when updating it.

Signed-off-by: Stefan Beller <sbeller@google.com>
---

This is a resend of [1], which did not receive any attention.
I improved the commit message laying out the current state of affairs,
arguing that any future plan should not weigh in as much as the current
possible data loss.

[1] https://public-inbox.org/git/20170630003851.17288-1-sbeller@google.com/

Thanks,
Stefan

 submodule.c               |  3 ++-
 t/lib-submodule-update.sh | 17 +++++++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/submodule.c b/submodule.c
index ef83c2a9ee..4b7c0a4c82 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1653,7 +1653,8 @@ int submodule_move_head(const char *path,
 			cp.dir = path;
 
 			prepare_submodule_repo_env(&cp.env_array);
-			argv_array_pushl(&cp.args, "update-ref", "HEAD", new, NULL);
+			argv_array_pushl(&cp.args, "update-ref", "HEAD",
+					 "--no-deref", new, NULL);
 
 			if (run_command(&cp)) {
 				ret = -1;
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 2d26f86800..fc406b95d7 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -848,6 +848,23 @@ test_submodule_switch_recursing_with_args () {
 			test_submodule_content sub1 origin/add_sub1
 		)
 	'
+	test_expect_success "$command: submodule branch is not changed, detach HEAD instead" '
+		prolog &&
+		reset_work_tree_to_interested add_sub1 &&
+		(
+			cd submodule_update &&
+			git -C sub1 checkout -b keep_branch &&
+			git -C sub1 rev-parse HEAD >expect &&
+			git branch -t check-keep origin/modify_sub1 &&
+			$command check-keep &&
+			test_superproject_content origin/modify_sub1 &&
+			test_submodule_content sub1 origin/modify_sub1 &&
+			git -C sub1 rev-parse keep_branch >actual &&
+			test_cmp expect actual &&
+			test_must_fail git -C sub1 symbolic-ref HEAD
+		)
+	'
+
 	# Replacing a tracked file with a submodule produces a checked out submodule
 	test_expect_success "$command: replace tracked file with submodule checks out submodule" '
 		prolog &&
-- 
2.14.0.rc0.3.g6c2e499285


^ permalink raw reply	[relevance 30%]

* Re: [PATCH] recursive submodules: detach HEAD from new state
  2017-07-24 17:36 [PATCH] recursive submodules: detach HEAD from new state Stefan Beller
@ 2017-07-24 18:03 ` Jonathan Nieder
  2017-07-24 19:07   ` Stefan Beller
      [irrelevant]   ` <xmqq8tjdcyf1.fsf@gitster.mtv.corp.google.com>
  0 siblings, 2 replies; 200+ results
From: Jonathan Nieder @ 2017-07-24 18:03 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, gitster

Hi,

Stefan Beller wrote:

> When a submodule is on a branch and in its superproject you run a
> recursive checkout, the branch of the submodule is updated to what the
> superproject checks out. This is very unexpected in the current model of
> Git as e.g. 'submodule update' always detaches the submodule HEAD.
>
> Despite having plans to have submodule HEADS not detached in the future,
> the current behavior is really bad as it doesn't match user expectations
> and it is not checking for loss of commits (only to be recovered via the
> reflog).

I think the corrected behavior doesn't match user expectations,
either.

Could this patch include some documentation to help users know what to
expect?

> Detach the HEAD unconditionally in the submodule when updating it.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
> This is a resend of [1], which did not receive any attention.

Yikes.  Yes, this bug looks problematic.  Thanks for working on it.

> I improved the commit message laying out the current state of affairs,
> arguing that any future plan should not weigh in as much as the current
> possible data loss.

Can you elaborate on what you mean about data loss?  At first glance
it would seem to me that detaching HEAD could lead to data loss since
there isn't a branch to keep track of the user's work.  Are you saying
the current behavior of updating whatever branch HEAD is on (which,
don't get me wrong, is a wrong behavior that needs fixing) bypassed
the reflog?

Thanks,
Jonathan

> [1] https://public-inbox.org/git/20170630003851.17288-1-sbeller@google.com/
[...]
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1653,7 +1653,8 @@ int submodule_move_head(const char *path,
>  			cp.dir = path;
>  
>  			prepare_submodule_repo_env(&cp.env_array);
> -			argv_array_pushl(&cp.args, "update-ref", "HEAD", new, NULL);
> +			argv_array_pushl(&cp.args, "update-ref", "HEAD",
> +					 "--no-deref", new, NULL);
>  
>  			if (run_command(&cp)) {
>  				ret = -1;

^ permalink raw reply	[relevance 16%]

* Re: [PATCH] recursive submodules: detach HEAD from new state
  2017-07-24 18:03 ` Jonathan Nieder
@ 2017-07-24 19:07   ` Stefan Beller
  2017-07-24 20:57     ` Junio C Hamano
      [irrelevant]   ` <xmqq8tjdcyf1.fsf@gitster.mtv.corp.google.com>
  1 sibling, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-24 19:07 UTC (permalink / raw)
  To: Jonathan Nieder; +Cc: git, Junio C Hamano

On Mon, Jul 24, 2017 at 11:03 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi,
>
> Stefan Beller wrote:
>
>> When a submodule is on a branch and in its superproject you run a
>> recursive checkout, the branch of the submodule is updated to what the
>> superproject checks out. This is very unexpected in the current model of
>> Git as e.g. 'submodule update' always detaches the submodule HEAD.
>>
>> Despite having plans to have submodule HEADS not detached in the future,
>> the current behavior is really bad as it doesn't match user expectations
>> and it is not checking for loss of commits (only to be recovered via the
>> reflog).
>
> I think the corrected behavior doesn't match user expectations,
> either.

Well, what is the user expectation?

>
> Could this patch include some documentation to help users know what to
> expect?

Sure, once we figured out what is reasonable.

>
>> Detach the HEAD unconditionally in the submodule when updating it.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>> This is a resend of [1], which did not receive any attention.
>
> Yikes.  Yes, this bug looks problematic.  Thanks for working on it.
>
>> I improved the commit message laying out the current state of affairs,
>> arguing that any future plan should not weigh in as much as the current
>> possible data loss.
>
> Can you elaborate on what you mean about data loss?

Assume we have a submodule 'sub' inside the superproject 'super',
then

    git -C super/sub checkout <my-unrelated-branch>
    git -C super checkout <some-tree-ish>

modifies my-unrelated-branch in the submodule, which is not related
to the superproject in any way.

This patch would detach from that branch and have the HEAD contain
the desired sha1. To think that further we'd still have potential data loss:

    git -C super/sub checkout <my-unrelated-branch>
    git -C super checkout <some-tree-ish>
    # fine so far as sub is in detached HEAD, but:
     ... hack hack hack ... in 'sub'
    git -C super/sub commit -m "work"
    git -C super checkout <other-tree-ish>
    # subs work is only to be recovered via reflog!

However this matches the current behavior of
"submodule update" which also tips, that are
not reachable from any ref.

> At first glance
> it would seem to me that detaching HEAD could lead to data loss since
> there isn't a branch to keep track of the user's work.

yes, but that is the same with "submodule update", which is what
people may have in mind?

>  Are you saying
> the current behavior of updating whatever branch HEAD is on (which,
> don't get me wrong, is a wrong behavior that needs fixing) bypassed
> the reflog?

No, I am not saying that.
I am saying that updating an unrelated branch (which is dragged into
the affair just because HEAD points at it) is very subtle thing, as any
commits on that branch can be considered safe (it is on a branch, right?)
but the detached HEAD is the default unsafe mode we currently have.

Thanks,
Stefan

^ permalink raw reply	[relevance 24%]

* [GSoC][PATCH 00/13] Update: Week 10
@ 2017-07-24 20:34 Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
                   ` (12 more replies)
  0 siblings, 13 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

SUMMARY OF MY PROJECT:

Git submodule subcommands are currently implemented by using shell script
'git-submodule.sh'. There are several reasons why we'll prefer not to
use the shell script. My project intends to convert the subcommands into
C code, thus making them builtins. This will increase Git's portability
and hence the efficiency of working with the git-submodule commands.
Link to the complete proposal: [1]

Mentors:
Stefan Beller <sbeller@google.com>
Christian Couder <christian.couder@gmail.com>

UPDATES:

Following are the updates about my ongoing project:

* status: some of the optimization changes which were suggested
  earlier were reverted, as I wasn't sure it fulfilled the
  purpose.

* sync & summary: some changes to the code were suggested after the
  previous review. Those changes were implemented successfully.

* add: the porting of this subcommand is underway. In the function
  cmd_add(), the value of sm_path undergoes quite a few changes
  which are taken care by the sed command. I'm currently working
  on porting them.

* foreach: as said in the previous update, the former patch [2] was
  spilt up into 4 patches, to get a clear picture of different changes
  made to submodule foreach. All of these patches are attached with
  this update. Also, the subcommand foreach is ported in accordance
  with this new changes as well.

PLAN FOR WEEK-11 (25 July 2017 to 31 July 2017):

* As all the patches prepared so far are posted on the mailing list, I'll
  focus on getting these patches merged and do the needful improvisions.

* Apart from this, as stated before, the porting of submodule subcommand
  is underway and will try my best to finish its porting and discuss it
  with the mentors as well in this following week. 

Also, a complete build report of this work is available at [3].
Branch: week-10
Build #140

And the work has also been pushed on github. It is available at [4].

[1]: https://docs.google.com/document/d/1krxVLooWl--75Pot3dazhfygR3wCUUWZWzTXtK1L-xU/
[2]: https://public-inbox.org/git/20170603003710.5558-1-sbeller@google.com/
[3]: https://travis-ci.org/pratham-pc/git/builds/
[4]: https://github.com/pratham-pc/git/commits/week-10

Prathamesh Chavan (13):
  submodule--helper: introduce get_submodule_displaypath()
  submodule--helper: introduce for_each_submodule_list()
  submodule: port set_name_rev() from shell to C
  submodule: port submodule subcommand 'status' from shell to C
  submodule: port submodule subcommand 'sync' from shell to C
  submodule: port submodule subcommand 'deinit' from shell to C
  diff: change scope of the function count_lines()
  submodule: port submodule subcommand 'summary' from shell to C
  submodule foreach: correct '$path' in nested submodules from a
    subdirectory
  submodule foreach: document '$sm_path' instead of '$path'
  submodule foreach: clarify the '$toplevel' variable documentation
  submodule foreach: document variable '$displaypath'
  submodule: port submodule subcommand 'foreach' from shell to C

 Documentation/git-submodule.txt |   15 +-
 builtin/submodule--helper.c     | 1165 ++++++++++++++++++++++++++++++++++++++-
 diff.c                          |    2 +-
 diff.h                          |    1 +
 git-submodule.sh                |  394 +------------
 t/t7407-submodule-foreach.sh    |   38 +-
 6 files changed, 1197 insertions(+), 418 deletions(-)

-- 
2.13.0


^ permalink raw reply	[relevance 18%]

* [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath()
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function get_submodule_displaypath() to replace the code
occurring in submodule_init() for generating displaypath of the
submodule with a call to it.

This new function will also be used in other parts of the system
in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 6abdad329..7af4de09b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -220,6 +220,27 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+
+	if (prefix && super_prefix) {
+		BUG("cannot have prefix '%s' and superprefix '%s'",
+		    prefix, super_prefix);
+	} else if (prefix) {
+		struct strbuf sb = STRBUF_INIT;
+		char *displaypath = xstrdup(relative_path(path, prefix, &sb));
+		strbuf_release(&sb);
+		return displaypath;
+	} else if (super_prefix) {
+		int len = strlen(super_prefix);
+		const char *format = is_dir_sep(super_prefix[len - 1]) ? "%s%s" : "%s/%s";
+		return xstrfmt(format, super_prefix, path);
+	} else {
+		return xstrdup(path);
+	}
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -339,16 +360,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 	/* Only loads from .gitmodules, no overlay with .git/config */
 	gitmodules_config();
-
-	if (prefix && get_super_prefix())
-		die("BUG: cannot have prefix and superprefix");
-	else if (prefix)
-		displaypath = xstrdup(relative_path(path, prefix, &sb));
-	else if (get_super_prefix()) {
-		strbuf_addf(&sb, "%s%s", get_super_prefix(), path);
-		displaypath = strbuf_detach(&sb, NULL);
-	} else
-		displaypath = xstrdup(path);
+	displaypath = get_submodule_displaypath(path, prefix);
 
 	sub = submodule_from_path(null_sha1, path);
 
@@ -363,7 +375,6 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 * Set active flag for the submodule being initialized
 	 */
 	if (!is_submodule_active(the_repository, path)) {
-		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list()
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function for_each_submodule_list() and
replace a loop in module_init() with a call to it.

The new function will also be used in other parts of the
system in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7af4de09b..e41572f7a 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -14,6 +14,9 @@
 #include "refs.h"
 #include "connect.h"
 
+typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
+				      void *cb_data);
+
 static char *get_default_remote(void)
 {
 	char *dest = NULL, *ret;
@@ -352,17 +355,30 @@ static int module_list(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static void init_submodule(const char *path, const char *prefix, int quiet)
+static void for_each_submodule_list(const struct module_list list,
+				    submodule_list_func_t fn, void *cb_data)
 {
+	int i;
+	for (i = 0; i < list.nr; i++)
+		fn(list.entries[i], cb_data);
+}
+
+struct init_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+};
+#define INIT_CB_INIT { NULL, 0 }
+
+static void init_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct init_cb *info = cb_data;
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	/* Only loads from .gitmodules, no overlay with .git/config */
-	gitmodules_config();
-	displaypath = get_submodule_displaypath(path, prefix);
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
 
-	sub = submodule_from_path(null_sha1, path);
+	sub = submodule_from_path(null_sha1, list_item->name);
 
 	if (!sub)
 		die(_("No url found for submodule path '%s' in .gitmodules"),
@@ -374,7 +390,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_active(the_repository, path)) {
+	if (!is_submodule_active(the_repository, list_item->name)) {
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
@@ -416,7 +432,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 		if (git_config_set_gently(sb.buf, url))
 			die(_("Failed to register url for submodule path '%s'"),
 			    displaypath);
-		if (!quiet)
+		if (!info->quiet)
 			fprintf(stderr,
 				_("Submodule '%s' (%s) registered for path '%s'\n"),
 				sub->name, url, displaypath);
@@ -445,10 +461,10 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 static int module_init(int argc, const char **argv, const char *prefix)
 {
+	struct init_cb info = INIT_CB_INIT;
 	struct pathspec pathspec;
 	struct module_list list = MODULE_LIST_INIT;
 	int quiet = 0;
-	int i;
 
 	struct option module_init_options[] = {
 		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
@@ -473,8 +489,11 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	if (!argc && git_config_get_value_multi("submodule.active"))
 		module_list_active(&list);
 
-	for (i = 0; i < list.nr; i++)
-		init_submodule(list.entries[i]->name, prefix, quiet);
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+
+	gitmodules_config();
+	for_each_submodule_list(list, init_submodule, &info);
 
 	return 0;
 }
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
                   ` (9 subsequent siblings)
  12 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Function set_name_rev() is ported from git-submodule to the
submodule--helper builtin. The function get_name_rev() generates the
value of the revision name as required, and the function
print_name_rev() handles the formating and printing of the obtained
revision name.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 63 +++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 16 ++----------
 2 files changed, 65 insertions(+), 14 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index e41572f7a..80f744407 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -244,6 +244,68 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+static char *get_name_rev(const char *sub_path, const char* object_id)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char ***d;
+
+	static const char *describe_bare[] = {
+		NULL
+	};
+
+	static const char *describe_tags[] = {
+		"--tags", NULL
+	};
+
+	static const char *describe_contains[] = {
+		"--contains", NULL
+	};
+
+	static const char *describe_all_always[] = {
+		"--all", "--always", NULL
+	};
+
+	static const char **describe_argv[] = {
+		describe_bare, describe_tags, describe_contains,
+		describe_all_always, NULL
+	};
+
+	for (d = describe_argv; *d; d++) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+		prepare_submodule_repo_env(&cp.env_array);
+		cp.dir = sub_path;
+		cp.git_cmd = 1;
+		cp.no_stderr = 1;
+
+		argv_array_push(&cp.args, "describe");
+		argv_array_pushv(&cp.args, *d);
+		argv_array_push(&cp.args, object_id);
+
+		if (!capture_command(&cp, &sb, 0) && sb.len) {
+			strbuf_strip_suffix(&sb, "\n");
+			return strbuf_detach(&sb, NULL);
+		}
+
+	}
+
+	strbuf_release(&sb);
+	return NULL;
+}
+
+static int print_name_rev(int argc, const char **argv, const char *prefix)
+{
+	char *namerev;
+	if (argc != 3)
+		die("print-name-rev only accepts two arguments: <path> <sha1>");
+
+	namerev = get_name_rev(argv[1], argv[2]);
+	if (namerev && namerev[0])
+		printf(" (%s)", namerev);
+	printf("\n");
+
+	return 0;
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -1242,6 +1304,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index e131760ee..e988167e0 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -759,18 +759,6 @@ cmd_update()
 	}
 }
 
-set_name_rev () {
-	revname=$( (
-		sanitize_submodule_env
-		cd "$1" && {
-			git describe "$2" 2>/dev/null ||
-			git describe --tags "$2" 2>/dev/null ||
-			git describe --contains "$2" 2>/dev/null ||
-			git describe --all --always "$2"
-		}
-	) )
-	test -z "$revname" || revname=" ($revname)"
-}
 #
 # Show commit summary for submodules in index or working tree
 #
@@ -1042,14 +1030,14 @@ cmd_status()
 		fi
 		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
 		then
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say " $sha1 $displaypath$revname"
 		else
 			if test -z "$cached"
 			then
 				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
 			fi
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say "+$sha1 $displaypath$revname"
 		fi
 
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' from shell to C
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (2 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
@ 2017-07-24 20:34 ` " Prathamesh Chavan
  2017-07-24 21:30   ` Brandon Williams
  2017-07-24 20:34 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
                   ` (8 subsequent siblings)
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

This aims to make git-submodule 'status' a built-in. Hence, the function
cmd_status() is ported from shell to C. This is done by introducing
three functions: module_status(), submodule_status() and print_status().

The function module_status() acts as the front-end of the subcommand.
It parses subcommand's options and then calls the function
module_list_compute() for computing the list of submodules. Then
this functions calls for_each_submodule_list() looping through the
list obtained.

Then for_each_submodule_list() calls submodule_status() for each of the
submodule in its list. The function submodule_status() is responsible
for generating the status each submodule it is called for, and
then calls print_status().

Finally, the function print_status() handles the printing of submodule's
status.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version of patch, following changes were made:
* instead of using the ce_match_stat(), cmd_diff_files is used.
* currently, no comment about future scope of optimization wrt the
  cmd_diff_files() usage was added as currently, I'm not fully sure of
  the way to optimize the function.

 builtin/submodule--helper.c | 151 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  49 +-------------
 2 files changed, 152 insertions(+), 48 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 80f744407..b39828174 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -560,6 +560,156 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct status_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+	unsigned int cached: 1;
+};
+#define STATUS_CB_INIT { NULL, 0, 0, 0 }
+
+static void print_status(struct status_cb *info, char state, const char *path,
+			 char *sub_sha1, char *displaypath)
+{
+	if (info->quiet)
+		return;
+
+	printf("%c%s %s", state, sub_sha1, displaypath);
+
+	if (state == ' ' || state == '+') {
+		struct argv_array name_rev_args = ARGV_ARRAY_INIT;
+
+		argv_array_pushl(&name_rev_args, "print-name-rev",
+				 path, sub_sha1, NULL);
+		print_name_rev(name_rev_args.argc, name_rev_args.argv,
+			       info->prefix);
+	} else {
+		printf("\n");
+	}
+}
+
+static int handle_submodule_head_ref(const char *refname,
+				     const struct object_id *oid, int flags,
+				     void *cb_data)
+{
+	struct strbuf *output = cb_data;
+	if (oid)
+		strbuf_addstr(output, oid_to_hex(oid));
+	return 0;
+}
+
+static void status_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct status_cb *info = cb_data;
+	char *sub_sha1 = xstrdup(oid_to_hex(&list_item->oid));
+	char *displaypath;
+	struct argv_array diff_files_args = ARGV_ARRAY_INIT;
+
+	if (!submodule_from_path(null_sha1, list_item->name))
+		die(_("no submodule mapping found in .gitmodules for path '%s'"),
+		      list_item->name);
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (list_item->ce_flags) {
+		print_status(info, 'U', list_item->name,
+			     sha1_to_hex(null_sha1), displaypath);
+		goto cleanup;
+	}
+
+	if (!is_submodule_active(the_repository, list_item->name)) {
+		print_status(info, '-', list_item->name, sub_sha1, displaypath);
+		goto cleanup;
+	}
+
+	argv_array_pushl(&diff_files_args, "diff-files",
+			 "--ignore-submodules=dirty", "--quiet", "--",
+			 list_item->name, NULL);
+
+	if (!cmd_diff_files(diff_files_args.argc, diff_files_args.argv,
+			    info->prefix)) {
+		print_status(info, ' ', list_item->name, sub_sha1, displaypath);
+	} else {
+		if (!info->cached) {
+			struct strbuf sb = STRBUF_INIT;
+			if (head_ref_submodule(list_item->name,
+					       handle_submodule_head_ref, &sb))
+				die(_("could not resolve HEAD ref inside the"
+				      "submodule '%s'"), list_item->name);
+			print_status(info, '+', list_item->name, sb.buf,
+				     displaypath);
+			strbuf_release(&sb);
+		} else {
+			print_status(info, '+', list_item->name, sub_sha1,
+				     displaypath);
+		}
+	}
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "status", "--recursive",
+				 NULL);
+
+		if (info->cached)
+			argv_array_push(&cpr.args, "--cached");
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	free(displaypath);
+	free(sub_sha1);
+}
+
+static int module_status(int argc, const char **argv, const char *prefix)
+{
+	struct status_cb info = STATUS_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int cached = 0;
+	int recursive = 0;
+
+	struct option module_status_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT_BOOL(0, "cached", &cached, N_("Use commit stored in the index instead of the one stored in the submodule HEAD")),
+		OPT_BOOL(0, "recursive", &recursive, N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule status [--quiet] [--cached] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_status_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+	info.cached = !!cached;
+
+	gitmodules_config();
+	for_each_submodule_list(list, status_submodule, &info);
+
+	return 0;
+}
+
 static int module_name(int argc, const char **argv, const char *prefix)
 {
 	const struct submodule *sub;
@@ -1306,6 +1456,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
+	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index e988167e0..51b057d82 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1005,54 +1005,7 @@ cmd_status()
 		shift
 	done
 
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		if test "$stage" = U
-		then
-			say "U$sha1 $displaypath"
-			continue
-		fi
-		if ! git submodule--helper is-active "$sm_path" ||
-		{
-			! test -d "$sm_path"/.git &&
-			! test -f "$sm_path"/.git
-		}
-		then
-			say "-$sha1 $displaypath"
-			continue;
-		fi
-		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
-		then
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say " $sha1 $displaypath$revname"
-		else
-			if test -z "$cached"
-			then
-				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
-			fi
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say "+$sha1 $displaypath$revname"
-		fi
-
-		if test -n "$recursive"
-		then
-			(
-				prefix="$displaypath/"
-				sanitize_submodule_env
-				wt_prefix=
-				cd "$sm_path" &&
-				eval cmd_status
-			) ||
-			die "$(eval_gettext "Failed to recurse into submodule path '\$sm_path'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} "$@"
 }
 #
 # Sync remote urls for submodules
-- 
2.13.0


^ permalink raw reply	[relevance 20%]

* [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' from shell to C
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (3 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-07-24 20:34 ` " Prathamesh Chavan
  2017-07-24 21:52   ` Brandon Williams
  2017-07-24 20:34 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
                   ` (7 subsequent siblings)
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Port the submodule subcommand 'sync' from shell to C using the same
mechanism as that used for porting submodule subcommand 'status'.
Hence, here the function cmd_sync() is ported from shell to C.
This is done by introducing three functions: module_sync(),
sync_submodule() and print_default_remote().

The function print_default_remote() is introduced for getting
the default remote as stdout.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version of patch, following changes were made:
* the code use to die when sub->url was found to be NULL. This was not a
  correct translation of code. It was corrected by using an empty string
  instead of sub->url.
* a process was used in the previous patch for registering the submodule
  url. This was avoided by the suggested changes on the previous patch.
* some nits were also corrected.
 
 builtin/submodule--helper.c | 183 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  56 +-------------
 2 files changed, 184 insertions(+), 55 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index b39828174..2d1d3984d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -44,6 +44,20 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static int print_default_remote(int argc, const char **argv, const char *prefix)
+{
+	const char *remote;
+
+	if (argc != 1)
+		die(_("submodule--helper print-default-remote takes no arguments"));
+
+	remote = get_default_remote();
+	if (remote)
+		puts(remote);
+
+	return 0;
+}
+
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -379,6 +393,25 @@ static void module_list_active(struct module_list *list)
 	*list = active_modules;
 }
 
+static char *get_up_path(const char *path)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	for (i = count_slashes(path); i; i--)
+		strbuf_addstr(&sb, "../");
+
+	/*
+	 * Check if 'path' ends with slash or not
+	 * for having the same output for dir/sub_dir
+	 * and dir/sub_dir/
+	 */
+	if (!is_dir_sep(path[strlen(path) - 1]))
+		strbuf_addstr(&sb, "../");
+
+	return strbuf_detach(&sb, NULL);
+}
+
 static int module_list(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -729,6 +762,154 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct sync_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define SYNC_CB_INIT { NULL, 0, 0 }
+
+static void sync_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct sync_cb *info = cb_data;
+	const struct submodule *sub;
+	char *sub_key, *remote_key;
+	char *sub_origin_url, *super_config_url, *displaypath;
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf sub_repo_path = STRBUF_INIT;
+	char *sub_config_path = NULL;
+
+	if (!is_submodule_active(the_repository, list_item->name))
+		return;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (sub && sub->url) {
+		if (starts_with_dot_dot_slash(sub->url) || starts_with_dot_slash(sub->url)) {
+			char *remote_url, *up_path;
+			char *remote = get_default_remote();
+			char *remote_key = xstrfmt("remote.%s.url", remote);
+
+			if (git_config_get_string(remote_key, &remote_url))
+				remote_url = xgetcwd();
+
+			up_path = get_up_path(list_item->name);
+			sub_origin_url = relative_url(remote_url, sub->url, up_path);
+			super_config_url = relative_url(remote_url, sub->url, NULL);
+
+			free(remote);
+			free(remote_key);
+			free(up_path);
+			free(remote_url);
+		} else {
+			sub_origin_url = xstrdup(sub->url);
+			super_config_url = xstrdup(sub->url);
+		}
+	} else {
+		sub_origin_url = "";
+		super_config_url = "";
+	}
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (!info->quiet)
+		printf(_("Synchronizing submodule url for '%s'\n"),
+			 displaypath);
+
+	sub_key = xstrfmt("submodule.%s.url", sub->name);
+	if (git_config_set_gently(sub_key, super_config_url))
+		die(_("failed to register url for submodule path '%s'"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.dir = list_item->name;
+	argv_array_pushl(&cp.args, "submodule--helper",
+			 "print-default-remote", NULL);
+
+	if (capture_command(&cp, &sb, 0))
+		die(_("failed to get the default remote for submodule '%s'"),
+		      list_item->name);
+
+	strbuf_strip_suffix(&sb, "\n");
+	remote_key = xstrfmt("remote.%s.url", sb.buf);
+	strbuf_release(&sb);
+
+	submodule_to_gitdir(&sub_repo_path, list_item->name);
+	sub_config_path = xstrfmt("%s/config", sub_repo_path.buf);
+	strbuf_release(&sub_repo_path);
+
+	if (git_config_set_in_file_gently(sub_config_path, remote_key, sub_origin_url))
+		die(_("failed to update remote for submodule '%s'"),
+		      list_item->name);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "sync", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	free(sub_key);
+	free(super_config_url);
+	free(displaypath);
+	free(sub_config_path);
+	free(sub_origin_url);
+}
+
+static int module_sync(int argc, const char **argv, const char *prefix)
+{
+	struct sync_cb info = SYNC_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_sync_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of synchronizing submodule url")),
+		OPT_BOOL(0, "recursive", &recursive,
+			N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper sync [--quiet] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_sync_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, sync_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1457,6 +1638,8 @@ static struct cmd_struct commands[] = {
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
+	{"print-default-remote", print_default_remote, 0},
+	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 51b057d82..6bfc5e17d 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1037,63 +1037,9 @@ cmd_sync()
 			;;
 		esac
 	done
-	cd_to_toplevel
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-
-		# skip inactive submodules
-		if ! git submodule--helper is-active "$sm_path"
-		then
-			continue
-		fi
-
-		name=$(git submodule--helper name "$sm_path")
-		url=$(git config -f .gitmodules --get submodule."$name".url)
-
-		# Possibly a url relative to parent
-		case "$url" in
-		./*|../*)
-			# rewrite foo/bar as ../.. to find path from
-			# submodule work tree to superproject work tree
-			up_path="$(printf '%s\n' "$sm_path" | sed "s/[^/][^/]*/../g")" &&
-			# guarantee a trailing /
-			up_path=${up_path%/}/ &&
-			# path from submodule work tree to submodule origin repo
-			sub_origin_url=$(git submodule--helper resolve-relative-url "$url" "$up_path") &&
-			# path from superproject work tree to submodule origin repo
-			super_config_url=$(git submodule--helper resolve-relative-url "$url") || exit
-			;;
-		*)
-			sub_origin_url="$url"
-			super_config_url="$url"
-			;;
-		esac
 
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		say "$(eval_gettext "Synchronizing submodule url for '\$displaypath'")"
-		git config submodule."$name".url "$super_config_url"
-
-		if test -e "$sm_path"/.git
-		then
-		(
-			sanitize_submodule_env
-			cd "$sm_path"
-			remote=$(get_default_remote)
-			git config remote."$remote".url "$sub_origin_url"
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 
-			if test -n "$recursive"
-			then
-				prefix="$prefix$sm_path/"
-				eval cmd_sync
-			fi
-		)
-		fi
-	done
 }
 
 cmd_absorbgitdirs()
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' from shell to C
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (4 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-07-24 20:34 ` " Prathamesh Chavan
  2017-07-24 23:03   ` Brandon Williams
  2017-07-24 20:34 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
                   ` (6 subsequent siblings)
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The same mechanism is used even for porting this submodule
subcommand, as used in the ported subcommands till now.
The function cmd_deinit in split up after porting into three
functions: module_deinit(), for_each_submodule_list() and
deinit_submodule().

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 141 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  55 +----------------
 2 files changed, 142 insertions(+), 54 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2d1d3984d..5e84fc42d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -910,6 +910,146 @@ static int module_sync(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct deinit_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int force: 1;
+	unsigned int all: 1;
+};
+#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
+
+static void deinit_submodule(const struct cache_entry *list_item,
+			     void *cb_data)
+{
+	struct deinit_cb *info = cb_data;
+	const struct submodule *sub;
+	char *displaypath = NULL;
+	struct child_process cp_config = CHILD_PROCESS_INIT;
+	struct strbuf sb_config = STRBUF_INIT;
+	char *sub_git_dir = xstrfmt("%s/.git", list_item->name);
+	struct stat st;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub || !sub->name)
+		goto cleanup;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	/* remove the submodule work tree (unless the user already did it) */
+	if (is_directory(list_item->name)) {
+		/* protect submodules containing a .git directory */
+		if (is_git_directory(sub_git_dir))
+			die(_("Submodule work tree '%s' contains a .git "
+			      "directory use 'rm -rf' if you really want "
+			      "to remove it including all of its history"),
+			      displaypath);
+
+		if (!info->force) {
+			struct child_process cp_rm = CHILD_PROCESS_INIT;
+			cp_rm.git_cmd = 1;
+			argv_array_pushl(&cp_rm.args, "rm", "-qn",
+					 list_item->name, NULL);
+
+			if (run_command(&cp_rm))
+				die(_("Submodule work tree '%s' contains local "
+				      "modifications; use '-f' to discard them"),
+				      displaypath);
+		}
+
+		if (!lstat(list_item->name, &st)) {
+			struct strbuf sb_rm = STRBUF_INIT;
+			const char *format;
+
+			strbuf_addstr(&sb_rm, list_item->name);
+
+			if (!remove_dir_recursively(&sb_rm, 0))
+				format = _("Cleared directory '%s'\n");
+			else
+				format = _("Could not remove submodule work tree '%s'\n");
+
+			if (!info->quiet)
+				printf(format, displaypath);
+
+			strbuf_release(&sb_rm);
+		}
+	}
+
+	if (mkdir(list_item->name, st.st_mode))
+		die(_("could not create empty submodule directory %s"),
+		      displaypath);
+
+	cp_config.git_cmd = 1;
+	argv_array_pushl(&cp_config.args, "config", "--get-regexp", NULL);
+	argv_array_pushf(&cp_config.args, "submodule.%s\\.", sub->name);
+
+	/* remove the .git/config entries (unless the user already did it) */
+	if (!capture_command(&cp_config, &sb_config, 0) && sb_config.len) {
+		char *sub_key = xstrfmt("submodule.%s", sub->name);
+		/*
+		 * remove the whole section so we have a clean state when
+		 * the user later decides to init this submodule again
+		 */
+		git_config_rename_section_in_file(NULL, sub_key, NULL);
+		if (!info->quiet)
+			printf(_("Submodule '%s' (%s) unregistered for path '%s'\n"),
+				 sub->name, sub->url, displaypath);
+		free(sub_key);
+	}
+
+cleanup:
+	free(displaypath);
+	free(sub_git_dir);
+	strbuf_release(&sb_config);
+}
+
+static int module_deinit(int argc, const char **argv, const char *prefix)
+{
+	struct deinit_cb info = DEINIT_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int force = 0;
+	int all = 0;
+
+	struct option module_deinit_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT__FORCE(&force, N_("Remove submodule working trees even if they contain local changes")),
+		OPT_BOOL(0, "all", &all, N_("Unregister all submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule deinit [--quiet] [-f | --force] [--all | [--] [<path>...]]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_deinit_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.all = !!all;
+	info.force = !!force;
+
+	if (all && argc) {
+		error("pathspec and --all are incompatible");
+		usage_with_options(git_submodule_helper_usage,
+				   module_deinit_options);
+	}
+
+	if (!argc && !all)
+		die(_("Use '--all' if you really want to deinitialize all submodules"));
+
+	gitmodules_config();
+	for_each_submodule_list(list, deinit_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1640,6 +1780,7 @@ static struct cmd_struct commands[] = {
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
+	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 6bfc5e17d..73e6f093f 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -428,60 +428,7 @@ cmd_deinit()
 		shift
 	done
 
-	if test -n "$deinit_all" && test "$#" -ne 0
-	then
-		echo >&2 "$(eval_gettext "pathspec and --all are incompatible")"
-		usage
-	fi
-	if test $# = 0 && test -z "$deinit_all"
-	then
-		die "$(eval_gettext "Use '--all' if you really want to deinitialize all submodules")"
-	fi
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-
-		displaypath=$(git submodule--helper relative-path "$sm_path" "$wt_prefix")
-
-		# Remove the submodule work tree (unless the user already did it)
-		if test -d "$sm_path"
-		then
-			# Protect submodules containing a .git directory
-			if test -d "$sm_path/.git"
-			then
-				die "$(eval_gettext "\
-Submodule work tree '\$displaypath' contains a .git directory
-(use 'rm -rf' if you really want to remove it including all of its history)")"
-			fi
-
-			if test -z "$force"
-			then
-				git rm -qn "$sm_path" ||
-				die "$(eval_gettext "Submodule work tree '\$displaypath' contains local modifications; use '-f' to discard them")"
-			fi
-			rm -rf "$sm_path" &&
-			say "$(eval_gettext "Cleared directory '\$displaypath'")" ||
-			say "$(eval_gettext "Could not remove submodule work tree '\$displaypath'")"
-		fi
-
-		mkdir "$sm_path" || say "$(eval_gettext "Could not create empty submodule directory '\$displaypath'")"
-
-		# Remove the .git/config entries (unless the user already did it)
-		if test -n "$(git config --get-regexp submodule."$name\.")"
-		then
-			# Remove the whole section so we have a clean state when
-			# the user later decides to init this submodule again
-			url=$(git config submodule."$name".url)
-			git config --remove-section submodule."$name" 2>/dev/null &&
-			say "$(eval_gettext "Submodule '\$name' (\$url) unregistered for path '\$displaypath'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${force:+--force} ${deinit_all:+--all} "$@"
 }
 
 is_tip_reachable () (
-- 
2.13.0


^ permalink raw reply	[relevance 20%]

* [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' from shell to C
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (5 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
@ 2017-07-24 20:34 ` " Prathamesh Chavan
  2017-07-25  0:09   ` Brandon Williams
  2017-07-24 20:34 ` [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
                   ` (5 subsequent siblings)
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The submodule subcommand 'summary' is ported in the process of
making git-submodule a builtin. The function cmd_summary() from
git-submodule.sh is ported to functions module_summary(),
compute_summary_module_list(), prepare_submodule_summary() and
print_submodule_summary().

The first function module_summary() parses the options of submodule
subcommand and also acts as the front-end of this subcommand.
After parsing them, it calls the compute_summary_module_list()

The functions compute_summary_module_list() runs the diff_cmd,
and generates the modules list, as required by the subcommand.
The generation of this module list is done by the using the
callback function submodule_summary_callback(), and stored in the
structure module_cb.

Once the module list is generated, prepare_submodule_summary()
further goes through the list and filters the list, for
eventually calling the print_submodule_summary() function.

Finally, the print_submodule_summary() takes care of generating
and printing the summary for each submodule.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version of patch, following changes were made:
* the way of generating sub_sha1_src and sub_sha1_dst (abbrev of sha1_src
  and sha1_dst resp.) were changed. Since there was no direct way of
  abbrevating a string(sha1_dst), in this patch sha1_dst was converted first
  to an object id (converting to sha1 was avoided) and then abbrevated using
  find_unique_abbrev().
* A few big if() statements were reduced.
* for reducing the two big if() statements, a new function
  verify_submodule_object_name() was introduced.
* this new version also corrects a few other nits.

 builtin/submodule--helper.c | 428 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 182 +------------------
 2 files changed, 429 insertions(+), 181 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 5e84fc42d..94d6254f0 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -13,6 +13,9 @@
 #include "remote.h"
 #include "refs.h"
 #include "connect.h"
+#include "revision.h"
+#include "diffcore.h"
+#include "diff.h"
 
 typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
 				      void *cb_data);
@@ -762,6 +765,430 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct module_cb {
+	unsigned int mod_src;
+	unsigned int mod_dst;
+	struct object_id oid_src;
+	struct object_id oid_dst;
+	char status;
+	const char *sm_path;
+};
+#define MODULE_CB_INIT { 0, 0, NULL, NULL, '\0', NULL }
+
+struct module_cb_list {
+	struct module_cb **entries;
+	int alloc, nr;
+};
+#define MODULE_CB_LIST_INIT { NULL, 0, 0 }
+
+struct summary_cb {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	char *diff_cmd;
+	unsigned int cached: 1;
+	unsigned int for_status: 1;
+	unsigned int quiet: 1;
+	unsigned int files: 1;
+	int summary_limits;
+};
+#define SUMMARY_CB_INIT { 0, NULL, NULL, NULL, 0, 0, 0, 0, 0 }
+
+static int verify_submodule_object_name(const char *sm_path, const char *sha1)
+{
+	struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+	cp_rev_parse.git_cmd = 1;
+	cp_rev_parse.no_stdout = 1;
+	cp_rev_parse.dir = sm_path;
+	prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+	argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
+			 "--verify", NULL);
+	argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1);
+
+	if (run_command(&cp_rev_parse))
+		return 1;
+
+	return 0;
+}
+
+static void print_submodule_summary(struct summary_cb *info,
+				    struct module_cb *p)
+{
+	int missing_src = 0;
+	int missing_dst = 0;
+	char *displaypath;
+	const char *sha1_abbr_src;
+	const char *sha1_abbr_dst;
+	struct object_id oid_dst;
+	int errmsg = 0;
+	int total_commits = -1;
+	const char *sha1_dst = oid_to_hex(&p->oid_dst);
+	const char *sha1_src = oid_to_hex(&p->oid_src);
+	char *sm_git_dir = xstrfmt("%s/.git", p->sm_path);
+	int is_sm_git_dir = 0;
+
+	if (!info->cached && !strcmp(sha1_dst, sha1_to_hex(null_sha1))) {
+		if (S_ISGITLINK(p->mod_dst)) {
+			struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_parse = STRBUF_INIT;
+
+			cp_rev_parse.git_cmd = 1;
+			cp_rev_parse.no_stderr = 1;
+			cp_rev_parse.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+			argv_array_pushl(&cp_rev_parse.args,
+					 "rev-parse", "HEAD", NULL);
+			if (!capture_command(&cp_rev_parse, &sb_rev_parse, 0)) {
+				strbuf_strip_suffix(&sb_rev_parse, "\n");
+				sha1_dst = xstrdup(sb_rev_parse.buf);
+			}
+			strbuf_release(&sb_rev_parse);
+		} else if (S_ISLNK(p->mod_dst) || S_ISREG(p->mod_dst)) {
+			struct child_process cp_hash_object = CHILD_PROCESS_INIT;
+			struct strbuf sb_hash_object = STRBUF_INIT;
+
+			cp_hash_object.git_cmd = 1;
+			argv_array_pushl(&cp_hash_object.args,
+					 "hash-object", p->sm_path,
+					 NULL);
+			if (!capture_command(&cp_hash_object,
+					     &sb_hash_object, 0)) {
+				strbuf_strip_suffix(&sb_hash_object, "\n");
+				sha1_dst = xstrdup(sb_hash_object.buf);
+			}
+			strbuf_release(&sb_hash_object);
+		} else {
+			if (p->mod_dst)
+				die(_("unexpected mode %d\n"), p->mod_dst);
+		}
+	}
+
+	if (is_git_directory(sm_git_dir))
+		is_sm_git_dir = 1;
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_src))
+		missing_src = verify_submodule_object_name(p->sm_path,
+							   sha1_src);
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_dst))
+		missing_dst = verify_submodule_object_name(p->sm_path,
+							   sha1_dst);
+
+	displaypath = get_submodule_displaypath(p->sm_path, info->prefix);
+
+	if (!missing_dst && !missing_src) {
+		if (is_sm_git_dir) {
+			struct child_process cp_rev_list = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_list = STRBUF_INIT;
+			char *range;
+
+			if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst))
+				range = xstrfmt("%s...%s", sha1_src, sha1_dst);
+			else if (S_ISGITLINK(p->mod_src))
+				range = xstrdup(sha1_src);
+			else
+				range = xstrdup(sha1_dst);
+
+			cp_rev_list.git_cmd = 1;
+			cp_rev_list.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_list.env_array);
+
+			argv_array_pushl(&cp_rev_list.args, "rev-list",
+					 "--first-parent", range, "--", NULL);
+			if (!capture_command(&cp_rev_list, &sb_rev_list, 0)) {
+				if (sb_rev_list.len)
+					total_commits = count_lines(sb_rev_list.buf,
+								    sb_rev_list.len);
+				else
+					total_commits = 0;
+			}
+
+			free(range);
+			strbuf_release(&sb_rev_list);
+		}
+	} else {
+		errmsg = 1;
+	}
+
+	get_oid_hex(sha1_dst, &oid_dst);
+
+	sha1_abbr_src = find_unique_abbrev(p->oid_src.hash, 7);
+	sha1_abbr_dst = find_unique_abbrev(oid_dst.hash, 7);
+
+	if (p->status == 'T') {
+		if (S_ISGITLINK(p->mod_dst))
+			printf(_("* %s %s(blob)->%s(submodule)"),
+				 displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+		else
+			printf(_("* %s %s(submodule)->%s(blob)"),
+				 displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+	} else {
+			printf("* %s %s...%s", displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+	}
+
+	if (total_commits < 0)
+		printf(":\n");
+	else
+		printf(" (%d):\n", total_commits);
+
+	if (errmsg) {
+		/*
+		 * Don't give error msg for modification whose dst is not
+		 * submodule, i.e. deleted or changed to blob
+		 */
+		if (S_ISGITLINK(p->mod_src)) {
+			if (missing_src && missing_dst) {
+				printf(_("  Warn: %s doesn't contain commits %s and %s\n"),
+				 displaypath, sha1_src, sha1_dst);
+			} else if (missing_src) {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, sha1_src);
+			} else {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, sha1_dst);
+			}
+		}
+	} else if (is_sm_git_dir) {
+		struct child_process cp_log = CHILD_PROCESS_INIT;
+
+		cp_log.git_cmd = 1;
+		cp_log.dir = p->sm_path;
+		prepare_submodule_repo_env(&cp_log.env_array);
+		argv_array_pushl(&cp_log.args, "log", NULL);
+
+		if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst)) {
+			if (info->summary_limits > 0)
+				argv_array_pushf(&cp_log.args, "-%d", info->summary_limits);
+
+			argv_array_pushl(&cp_log.args, "--pretty=  %m %s",
+					 "--first-parent", NULL);
+			argv_array_pushf(&cp_log.args, "%s...%s", sha1_src,
+					 sha1_dst);
+		} else if (S_ISGITLINK(p->mod_dst)) {
+			argv_array_pushl(&cp_log.args, "--pretty=  > %s",
+					 "-1", sha1_dst, NULL);
+		} else {
+			argv_array_pushl(&cp_log.args, "--pretty=  < %s",
+					 "-1", sha1_src, NULL);
+		}
+
+		run_command(&cp_log);
+	}
+	printf("\n");
+
+	free(displaypath);
+}
+
+static void prepare_submodule_summary(struct summary_cb *info,
+				      struct module_cb_list *list)
+{
+	int i;
+	for (i = 0; i < list->nr; i++) {
+		struct module_cb *p = list->entries[i];
+		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+		if (p->status == 'D' || p->status == 'T') {
+			print_submodule_summary(info, p);
+			continue;
+		}
+
+		if (info->for_status) {
+			char *config_key;
+			const char *ignore_config = "none";
+			const char *value;
+			const struct submodule *sub = submodule_from_path(null_sha1, p->sm_path);
+
+			if (sub && p->status != 'A') {
+				config_key = xstrfmt("submodule.%s.ignore",
+						     sub->name);
+				if (!git_config_get_value(config_key, &value))
+					ignore_config = value;
+				else if (sub->ignore)
+					ignore_config = sub->ignore;
+
+				free(config_key);
+
+				if (!strcmp(ignore_config, "all"))
+					continue;
+			}
+		}
+
+		/* Also show added or modified modules which are checked out */
+		cp_rev_parse.dir = p->sm_path;
+		cp_rev_parse.git_cmd = 1;
+		cp_rev_parse.no_stderr = 1;
+		cp_rev_parse.no_stdout = 1;
+
+		argv_array_pushl(&cp_rev_parse.args, "rev-parse",
+				 "--git-dir", NULL);
+
+		if (!run_command(&cp_rev_parse))
+			print_submodule_summary(info, p);
+	}
+}
+
+static void submodule_summary_callback(struct diff_queue_struct *q,
+				       struct diff_options *options,
+				       void *data)
+{
+	int i;
+	struct module_cb_list *list = data;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		struct module_cb *temp;
+
+		if (!S_ISGITLINK(p->one->mode) && !S_ISGITLINK(p->two->mode))
+			continue;
+		temp = (struct module_cb*)malloc(sizeof(struct module_cb));
+		temp->mod_src = p->one->mode;
+		temp->mod_dst = p->two->mode;
+		temp->oid_src = p->one->oid;
+		temp->oid_dst = p->two->oid;
+		temp->status = p->status;
+		temp->sm_path = xstrdup(p->one->path);
+
+		ALLOC_GROW(list->entries, list->nr + 1, list->alloc);
+		list->entries[list->nr++] = temp;
+	}
+}
+
+static int compute_summary_module_list(char *head, struct summary_cb *info)
+{
+	struct argv_array diff_args = ARGV_ARRAY_INIT;
+	struct rev_info rev;
+	struct module_cb_list list = MODULE_CB_LIST_INIT;
+
+	argv_array_push(&diff_args, info->diff_cmd);
+	if (info->cached)
+		argv_array_push(&diff_args, "--cached");
+	argv_array_pushl(&diff_args, "--ignore-submodules=dirty", "--raw",
+			 NULL);
+	if (head)
+		argv_array_push(&diff_args, head);
+	argv_array_push(&diff_args, "--");
+	if (info->argc)
+		argv_array_pushv(&diff_args, info->argv);
+
+	git_config(git_diff_basic_config, NULL);
+	init_revisions(&rev, info->prefix);
+	gitmodules_config();
+	rev.abbrev = 0;
+	precompose_argv(diff_args.argc, diff_args.argv);
+
+	diff_args.argc = setup_revisions(diff_args.argc, diff_args.argv,
+					 &rev, NULL);
+	rev.diffopt.output_format = DIFF_FORMAT_NO_OUTPUT | DIFF_FORMAT_CALLBACK;
+	rev.diffopt.format_callback = submodule_summary_callback;
+	rev.diffopt.format_callback_data = &list;
+
+	if (!info->cached) {
+		if (!strcmp(info->diff_cmd, "diff-index"))
+			setup_work_tree();
+		if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
+			perror("read_cache_preload");
+			return -1;
+		}
+	} else if (read_cache() < 0) {
+		perror("read_cache");
+		return -1;
+	}
+
+	if (!strcmp(info->diff_cmd, "diff-index"))
+		run_diff_index(&rev, info->cached);
+	else
+		run_diff_files(&rev, 0);
+	prepare_submodule_summary(info, &list);
+
+	return 0;
+
+}
+
+static int module_summary(int argc, const char **argv, const char *prefix)
+{
+	struct summary_cb info = SUMMARY_CB_INIT;
+	int cached = 0;
+	char *diff_cmd = "diff-index";
+	int for_status = 0;
+	int quiet = 0;
+	int files = 0;
+	int summary_limits = -1;
+	struct child_process cp_rev = CHILD_PROCESS_INIT;
+	char *head;
+	struct strbuf sb = STRBUF_INIT;
+
+	struct option module_summary_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
+		OPT_BOOL(0, "cached", &cached, N_("Use the commit stored in the index instead of the submodule HEAD")),
+		OPT_BOOL(0, "files", &files, N_("To compares the commit in the index with that in the submodule HEAD")),
+		OPT_BOOL(0, "for-status", &for_status, N_("Skip submodules with 'all' ignore_config value")),
+		OPT_INTEGER('n', "summary-limits", &summary_limits, N_("Limit the summary size")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper summary [<options>] [--] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_summary_options,
+			     git_submodule_helper_usage, 0);
+
+	if (!summary_limits)
+		return 0;
+
+	cp_rev.git_cmd = 1;
+	argv_array_pushl(&cp_rev.args, "rev-parse", "-q", "--verify",
+			 argc ? argv[0] : "HEAD", NULL);
+
+	if (!capture_command(&cp_rev, &sb, 0)) {
+		strbuf_strip_suffix(&sb, "\n");
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else if (!argc || !strcmp(argv[0], "HEAD")) {
+		/* before the first commit: compare with an empty tree */
+		struct stat st;
+		unsigned char sha1[20];
+		if (fstat(0, &st) < 0 || index_fd(sha1, 0, &st, 2, prefix, 3))
+			die("Unable to add %s to database", sha1);
+		strbuf_addstr(&sb, sha1_to_hex(sha1));
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else {
+		strbuf_addstr(&sb, "HEAD");
+	}
+
+	head = strbuf_detach(&sb, NULL);
+
+	if (files) {
+		if (cached)
+			die(_("The --cached option cannot be used with the --files option"));
+		diff_cmd = "diff-files";
+		head = NULL;
+	}
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.cached = cached;
+	info.for_status = for_status;
+	info.quiet = quiet;
+	info.files = files;
+	info.summary_limits = summary_limits;
+	info.diff_cmd = diff_cmd;
+
+	return compute_summary_module_list(head, &info);
+}
+
 struct sync_cb {
 	const char *prefix;
 	unsigned int quiet: 1;
@@ -1781,6 +2208,7 @@ static struct cmd_struct commands[] = {
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
+	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 73e6f093f..a427ddafd 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -51,31 +51,6 @@ die_if_unmatched ()
 	fi
 }
 
-#
-# Print a submodule configuration setting
-#
-# $1 = submodule name
-# $2 = option name
-# $3 = default value
-#
-# Checks in the usual git-config places first (for overrides),
-# otherwise it falls back on .gitmodules.  This allows you to
-# distribute project-wide defaults in .gitmodules, while still
-# customizing individual repositories if necessary.  If the option is
-# not in .gitmodules either, print a default value.
-#
-get_submodule_config () {
-	name="$1"
-	option="$2"
-	default="$3"
-	value=$(git config submodule."$name"."$option")
-	if test -z "$value"
-	then
-		value=$(git config -f .gitmodules submodule."$name"."$option")
-	fi
-	printf '%s' "${value:-$default}"
-}
-
 isnumber()
 {
 	n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
@@ -755,163 +730,8 @@ cmd_summary() {
 		shift
 	done
 
-	test $summary_limit = 0 && return
-
-	if rev=$(git rev-parse -q --verify --default HEAD ${1+"$1"})
-	then
-		head=$rev
-		test $# = 0 || shift
-	elif test -z "$1" || test "$1" = "HEAD"
-	then
-		# before the first commit: compare with an empty tree
-		head=$(git hash-object -w -t tree --stdin </dev/null)
-		test -z "$1" || shift
-	else
-		head="HEAD"
-	fi
-
-	if [ -n "$files" ]
-	then
-		test -n "$cached" &&
-		die "$(gettext "The --cached option cannot be used with the --files option")"
-		diff_cmd=diff-files
-		head=
-	fi
-
-	cd_to_toplevel
-	eval "set $(git rev-parse --sq --prefix "$wt_prefix" -- "$@")"
-	# Get modified modules cared by user
-	modules=$(git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- "$@" |
-		sane_egrep '^:([0-7]* )?160000' |
-		while read -r mod_src mod_dst sha1_src sha1_dst status sm_path
-		do
-			# Always show modules deleted or type-changed (blob<->module)
-			if test "$status" = D || test "$status" = T
-			then
-				printf '%s\n' "$sm_path"
-				continue
-			fi
-			# Respect the ignore setting for --for-status.
-			if test -n "$for_status"
-			then
-				name=$(git submodule--helper name "$sm_path")
-				ignore_config=$(get_submodule_config "$name" ignore none)
-				test $status != A && test $ignore_config = all && continue
-			fi
-			# Also show added or modified modules which are checked out
-			GIT_DIR="$sm_path/.git" git-rev-parse --git-dir >/dev/null 2>&1 &&
-			printf '%s\n' "$sm_path"
-		done
-	)
-
-	test -z "$modules" && return
-
-	git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- $modules |
-	sane_egrep '^:([0-7]* )?160000' |
-	cut -c2- |
-	while read -r mod_src mod_dst sha1_src sha1_dst status name
-	do
-		if test -z "$cached" &&
-			test $sha1_dst = 0000000000000000000000000000000000000000
-		then
-			case "$mod_dst" in
-			160000)
-				sha1_dst=$(GIT_DIR="$name/.git" git rev-parse HEAD)
-				;;
-			100644 | 100755 | 120000)
-				sha1_dst=$(git hash-object $name)
-				;;
-			000000)
-				;; # removed
-			*)
-				# unexpected type
-				eval_gettextln "unexpected mode \$mod_dst" >&2
-				continue ;;
-			esac
-		fi
-		missing_src=
-		missing_dst=
-
-		test $mod_src = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_src^0 >/dev/null &&
-		missing_src=t
-
-		test $mod_dst = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_dst^0 >/dev/null &&
-		missing_dst=t
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${for_status:+--for-status} ${files:+--files} ${cached:+--cached} ${summary_limit:+-n $summary_limit} "$@"
 
-		display_name=$(git submodule--helper relative-path "$name" "$wt_prefix")
-
-		total_commits=
-		case "$missing_src,$missing_dst" in
-		t,)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_src")"
-			;;
-		,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_dst")"
-			;;
-		t,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commits \$sha1_src and \$sha1_dst")"
-			;;
-		*)
-			errmsg=
-			total_commits=$(
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				range="$sha1_src...$sha1_dst"
-			elif test $mod_src = 160000
-			then
-				range=$sha1_src
-			else
-				range=$sha1_dst
-			fi
-			GIT_DIR="$name/.git" \
-			git rev-list --first-parent $range -- | wc -l
-			)
-			total_commits=" ($(($total_commits + 0)))"
-			;;
-		esac
-
-		sha1_abbr_src=$(echo $sha1_src | cut -c1-7)
-		sha1_abbr_dst=$(echo $sha1_dst | cut -c1-7)
-		if test $status = T
-		then
-			blob="$(gettext "blob")"
-			submodule="$(gettext "submodule")"
-			if test $mod_dst = 160000
-			then
-				echo "* $display_name $sha1_abbr_src($blob)->$sha1_abbr_dst($submodule)$total_commits:"
-			else
-				echo "* $display_name $sha1_abbr_src($submodule)->$sha1_abbr_dst($blob)$total_commits:"
-			fi
-		else
-			echo "* $display_name $sha1_abbr_src...$sha1_abbr_dst$total_commits:"
-		fi
-		if test -n "$errmsg"
-		then
-			# Don't give error msg for modification whose dst is not submodule
-			# i.e. deleted or changed to blob
-			test $mod_dst = 160000 && echo "$errmsg"
-		else
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				limit=
-				test $summary_limit -gt 0 && limit="-$summary_limit"
-				GIT_DIR="$name/.git" \
-				git log $limit --pretty='format:  %m %s' \
-				--first-parent $sha1_src...$sha1_dst
-			elif test $mod_dst = 160000
-			then
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  > %s' -1 $sha1_dst
-			else
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  < %s' -1 $sha1_src
-			fi
-			echo
-		fi
-		echo
-	done
 }
 #
 # List all submodules, prefixed with:
-- 
2.13.0


^ permalink raw reply	[relevance 15%]

* [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (6 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-25  0:13   ` Brandon Williams
  2017-07-24 20:34 ` [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
                   ` (4 subsequent siblings)
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

When running 'git submodule foreach' from a subdirectory of your
repository, nested submodules get a bogus value for $sm_path:
For a submodule 'sub' that contains a nested submodule 'nested',
running 'git -C dir submodule foreach echo $path' would report
path='../nested' for the nested submodule. The first part '../' is
derived from the logic computing the relative path from $pwd to the
root of the superproject. The second part is the submodule path inside
the submodule. This value is of little use and is hard to document.

There are two different possible solutions that have more value:
(a) The path value is documented as the path from the toplevel of the
    superproject to the mount point of the submodule.
    In this case we would want to have path='sub/nested'.

(b) As Ramsay noticed the documented value is wrong. For the non-nested
    case the path is equal to the relative path from $pwd to the
    submodules working directory. When following this model,
    the expected value would be path='../sub/nested'.

The behavior for (b) was introduced in 091a6eb0fe (submodule: drop the
top-level requirement, 2013-06-16) the intent for $path seemed to be
relative to $cwd to the submodule worktree, but that did not work for
nested submodules, as the intermittent submodules were not included in
the path.

If we were to fix the meaning of the $path using (a) such that "path"
is "the path from the toplevel of the superproject to the mount point
of the submodule", we would break any existing submodule user that runs
foreach from non-root of the superproject as the non-nested submodule
'../sub' would change its path to 'sub'.

If we would fix the meaning of the $path using (b), such that "path"
is "the relative path from $pwd to the submodule", then we would break
any user that uses nested submodules (even from the root directory) as
the 'nested' would become 'sub/nested'.

Both groups can be found in the wild.  The author has no data if one group
outweighs the other by large margin, and offending each one seems equally
bad at first.  However in the authors imagination it is better to go with
(a) as running from a sub directory sounds like it is carried out
by a human rather than by some automation task.  With a human on
the keyboard the feedback loop is short and the changed behavior can be
adapted to quickly unlike some automation that can break silently.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh             |  1 -
 t/t7407-submodule-foreach.sh | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index a427ddafd..493a64372 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -320,7 +320,6 @@ cmd_foreach()
 				prefix="$prefix$sm_path/"
 				sanitize_submodule_env
 				cd "$sm_path" &&
-				sm_path=$(git submodule--helper relative-path "$sm_path" "$wt_prefix") &&
 				# we make $path available to scripts ...
 				path=$sm_path &&
 				if test $# -eq 1
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 6ba5daf42..0663622a4 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,9 +82,9 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-../sub1-$sub1sha1
+$pwd/clone-foo1-sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-../sub3-$sub3sha1
+$pwd/clone-foo3-sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
@@ -196,6 +196,38 @@ test_expect_success 'test messages from "foreach --recursive" from subdirectory'
 	) &&
 	test_i18ncmp expect actual
 '
+sub1sha1=$(cd clone2/sub1 && git rev-parse HEAD)
+sub2sha1=$(cd clone2/sub2 && git rev-parse HEAD)
+sub3sha1=$(cd clone2/sub3 && git rev-parse HEAD)
+nested1sha1=$(cd clone2/nested1 && git rev-parse HEAD)
+nested2sha1=$(cd clone2/nested1/nested2 && git rev-parse HEAD)
+nested3sha1=$(cd clone2/nested1/nested2/nested3 && git rev-parse HEAD)
+submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEAD)
+
+cat >expect <<EOF
+Entering '../nested1'
+$pwd/clone2-nested1-nested1-$nested1sha1
+Entering '../nested1/nested2'
+$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+Entering '../nested1/nested2/nested3'
+$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+Entering '../nested1/nested2/nested3/submodule'
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+Entering '../sub1'
+$pwd/clone2-foo1-sub1-$sub1sha1
+Entering '../sub2'
+$pwd/clone2-foo2-sub2-$sub2sha1
+Entering '../sub3'
+$pwd/clone2-foo3-sub3-$sub3sha1
+EOF
+
+test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
+	(
+		cd clone2/untracked &&
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+	) &&
+	test_i18ncmp expect actual
+'
 
 cat > expect <<EOF
 nested1-nested1
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path'
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (7 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-25  0:15   ` Brandon Williams
  2017-07-24 20:34 ` [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
                   ` (3 subsequent siblings)
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

As using a variable '$path' may be harmful to users due to
capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
use $path variable in eval_gettext string, 2012-04-17). Adjust
the documentation to advocate for using $sm_path,  which contains
the same value. We still make the 'path' variable available and
document it as a deprecated synonym of 'sm_path'.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index ff612001d..a23baef62 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,12 +183,14 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $path, $sha1 and
+	The command has access to the variables $name, $sm_path, $sha1 and
 	$toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
-	$path is the name of the submodule directory relative to the
-	superproject, $sha1 is the commit as recorded in the superproject,
-	and $toplevel is the absolute path to the top-level of the superproject.
+	$sm_path is the path of the submodule as recorded in the superproject,
+	$sha1 is the commit as recorded in the superproject, and
+	$toplevel is the absolute path to the top-level of the superproject.
+	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
+	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given `--quiet`, foreach prints the name
 	of each submodule before evaluating the command.
-- 
2.13.0


^ permalink raw reply	[relevance 23%]

* [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (8 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-24 20:34 ` [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
                   ` (2 subsequent siblings)
  12 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

It does not contain the topmost superproject as the author assumed,
but the direct superproject, such that $toplevel/$sm_path is the
actual absolute path of the submodule.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index a23baef62..8e7930ebc 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -188,7 +188,8 @@ foreach [--recursive] <command>::
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
 	$sha1 is the commit as recorded in the superproject, and
-	$toplevel is the absolute path to the top-level of the superproject.
+	$toplevel is the absolute path to its superproject, such that
+	$toplevel/$sm_path is the absolute path of the submodule.
 	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
 	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
-- 
2.13.0


^ permalink raw reply	[relevance 24%]

* [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath'
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (9 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-25  0:16   ` Brandon Williams
  2017-07-24 20:34 ` [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

It was observer that the variable '$displaypath' was accessible but
undocumented. Hence, document it.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt |  6 ++++--
 t/t7407-submodule-foreach.sh    | 22 +++++++++++-----------
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 8e7930ebc..0cca702cb 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,10 +183,12 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $sm_path, $sha1 and
-	$toplevel:
+	The command has access to the variables $name, $sm_path, $displaypath,
+	$sha1 and $toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
+	$displaypath contains the relative path from the current working
+	directory to the submodules root directory,
 	$sha1 is the commit as recorded in the superproject, and
 	$toplevel is the absolute path to its superproject, such that
 	$toplevel/$sm_path is the absolute path of the submodule.
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 0663622a4..6ad57e061 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,16 +82,16 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-sub1-$sub1sha1
+$pwd/clone-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-sub3-$sub3sha1
+$pwd/clone-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
 	mkdir clone/sub &&
 	(
 		cd clone/sub &&
-		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
@@ -206,25 +206,25 @@ submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEA
 
 cat >expect <<EOF
 Entering '../nested1'
-$pwd/clone2-nested1-nested1-$nested1sha1
+$pwd/clone2-nested1-nested1-../nested1-$nested1sha1
 Entering '../nested1/nested2'
-$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+$pwd/clone2/nested1-nested2-nested2-../nested1/nested2-$nested2sha1
 Entering '../nested1/nested2/nested3'
-$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+$pwd/clone2/nested1/nested2-nested3-nested3-../nested1/nested2/nested3-$nested3sha1
 Entering '../nested1/nested2/nested3/submodule'
-$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-../nested1/nested2/nested3/submodule-$submodulesha1
 Entering '../sub1'
-$pwd/clone2-foo1-sub1-$sub1sha1
+$pwd/clone2-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub2'
-$pwd/clone2-foo2-sub2-$sub2sha1
+$pwd/clone2-foo2-sub2-../sub2-$sub2sha1
 Entering '../sub3'
-$pwd/clone2-foo3-sub3-$sub3sha1
+$pwd/clone2-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
 	(
 		cd clone2/untracked &&
-		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C
  2017-07-24 20:34 [GSoC][PATCH 00/13] Update: Week 10 Prathamesh Chavan
                   ` (10 preceding siblings ...)
  2017-07-24 20:34 ` [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
@ 2017-07-24 20:34 ` Prathamesh Chavan
  2017-07-25  0:29   ` Brandon Williams
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
  12 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-24 20:34 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

This aims to make git-submodule foreach a builtin. This is the very
first step taken in this direction. Hence, 'foreach' is ported to
submodule--helper, and submodule--helper is called from git-submodule.sh.
The code is split up to have one function to obtain all the list of
submodules. This function acts as the front-end of git-submodule foreach
subcommand. It calls the function for_each_submodule_list, which basically
loops through the list and calls function fn, which in this case is
runcommand_in_submodule. This third function is a calling function that
takes care of running the command in that submodule, and recursively
perform the same when --recursive is flagged.

The first function module_foreach first parses the options present in
argv, and then with the help of module_list_compute, generates the list of
submodules present in the current working tree.

The second function for_each_submodule_list traverses through the
list, and calls function fn (which in case of submodule subcommand
foreach is runcommand_in_submodule) is called for each entry.

The third function runcommand_in_submodule, generates a submodule struct sub
for $name, value and then later prepends name=sub->name; and other
value assignment to the env argv_array structure of a child_process.
Also the <command> of submodule-foreach is push to args argv_array
structure and finally, using run_command the commands are executed
using a shell.

The third function also takes care of the recursive flag, by creating
a separate child_process structure and prepending "--super-prefix displaypath",
to the args argv_array structure. Other required arguments and the
input <command> of submodule-foreach is also appended to this argv_array.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 129 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  39 +-------------
 2 files changed, 130 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 94d6254f0..be278bf8d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -765,6 +765,134 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct cb_foreach {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define CB_FOREACH_INIT { 0, NULL, NULL, 0, 0 }
+
+static void runcommand_in_submodule(const struct cache_entry *list_item,
+				    void *cb_data)
+{
+	struct cb_foreach *info = cb_data;
+	const struct submodule *sub;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *displaypath;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub)
+		die(_("No url found for submodule path '%s' in .gitmodules"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+	/* For the purpose of executing <command> in the submodule,
+	 * separate shell is used for the purpose of running the
+	 * child process.
+	 */
+	cp.use_shell = 1;
+	cp.dir = list_item->name;
+
+	if (info->argc == 1) {
+		char *toplevel = xgetcwd();
+
+		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
+		argv_array_pushf(&cp.env_array, "sm_path=%s", list_item->name);
+		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
+		argv_array_pushf(&cp.env_array, "sha1=%s",
+				 oid_to_hex(&list_item->oid));
+		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
+
+		/*
+		 * Since still the path variable was accessible from the
+		 * script before porting, it is also made available.
+		 */
+		argv_array_pushf(&cp.args, "path=%s; %s",
+				 list_item->name, info->argv[0]);
+		free(toplevel);
+	} else {
+		argv_array_pushv(&cp.args, info->argv);
+	}
+
+	if (!info->quiet)
+		printf(_("Entering '%s'\n"), displaypath);
+
+	if (info->argv[0] && run_command(&cp))
+		die(_("run_command returned non-zero status for %s\n."),
+		      displaypath);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "foreach", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		argv_array_pushv(&cpr.args, info->argv);
+
+		if (run_command(&cpr))
+			die(_("run_command returned non-zero status while"
+			      "recursing in the nested submodules of %s\n."),
+			      displaypath);
+	}
+
+cleanup:
+	free(displaypath);
+}
+
+static int module_foreach(int argc, const char **argv, const char *prefix)
+{
+	struct cb_foreach info;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_foreach_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of entering each submodule command")),
+		OPT_BOOL(0, "recursive", &recursive,
+			 N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_foreach_options,
+			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, runcommand_in_submodule, &info);
+
+	return 0;
+}
+
 struct module_cb {
 	unsigned int mod_src;
 	unsigned int mod_dst;
@@ -2203,6 +2331,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
+	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 493a64372..e25b2c613 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -298,44 +298,7 @@ cmd_foreach()
 		shift
 	done
 
-	toplevel=$(pwd)
-
-	# dup stdin so that it can be restored when running the external
-	# command in the subshell (and a recursive call to this function)
-	exec 3<&0
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		if test -e "$sm_path"/.git
-		then
-			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-			say "$(eval_gettext "Entering '\$displaypath'")"
-			name=$(git submodule--helper name "$sm_path")
-			(
-				prefix="$prefix$sm_path/"
-				sanitize_submodule_env
-				cd "$sm_path" &&
-				# we make $path available to scripts ...
-				path=$sm_path &&
-				if test $# -eq 1
-				then
-					eval "$1"
-				else
-					"$@"
-				fi &&
-				if test -n "$recursive"
-				then
-					cmd_foreach "--recursive" "$@"
-				fi
-			) <&3 3<&- ||
-			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 #
-- 
2.13.0


^ permalink raw reply	[relevance 20%]

* Re: [PATCH] recursive submodules: detach HEAD from new state
  2017-07-24 19:07   ` Stefan Beller
@ 2017-07-24 20:57     ` Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2017-07-24 20:57 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Jonathan Nieder, git\

Stefan Beller <sbeller@google.com> writes:

>>  Are you saying
>> the current behavior of updating whatever branch HEAD is on (which,
>> don't get me wrong, is a wrong behavior that needs fixing) bypassed
>> the reflog?
>
> No, I am not saying that.
> I am saying that updating an unrelated branch (which is dragged into
> the affair just because HEAD points at it) is very subtle thing, as any
> commits on that branch can be considered safe (it is on a branch, right?)
> but the detached HEAD is the default unsafe mode we currently have.

Then it is not a data loss as you claimed in the proposed log
message, but is something else, no?

^ permalink raw reply	[relevance 16%]

* Re: [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' from shell to C
  2017-07-24 20:34 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-07-24 21:30   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-24 21:30 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> This aims to make git-submodule 'status' a built-in. Hence, the function
> cmd_status() is ported from shell to C. This is done by introducing
> three functions: module_status(), submodule_status() and print_status().
> 
> The function module_status() acts as the front-end of the subcommand.
> It parses subcommand's options and then calls the function
> module_list_compute() for computing the list of submodules. Then
> this functions calls for_each_submodule_list() looping through the
> list obtained.
> 
> Then for_each_submodule_list() calls submodule_status() for each of the
> submodule in its list. The function submodule_status() is responsible
> for generating the status each submodule it is called for, and
> then calls print_status().
> 
> Finally, the function print_status() handles the printing of submodule's
> status.
> 
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
> In this new version of patch, following changes were made:
> * instead of using the ce_match_stat(), cmd_diff_files is used.
> * currently, no comment about future scope of optimization wrt the
>   cmd_diff_files() usage was added as currently, I'm not fully sure of
>   the way to optimize the function.
> 
>  builtin/submodule--helper.c | 151 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            |  49 +-------------
>  2 files changed, 152 insertions(+), 48 deletions(-)
> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 80f744407..b39828174 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -560,6 +560,156 @@ static int module_init(int argc, const char **argv, const char *prefix)
>  	return 0;
>  }
>  
> +struct status_cb {
> +	const char *prefix;
> +	unsigned int quiet: 1;
> +	unsigned int recursive: 1;
> +	unsigned int cached: 1;
> +};
> +#define STATUS_CB_INIT { NULL, 0, 0, 0 }
> +
> +static void print_status(struct status_cb *info, char state, const char *path,
> +			 char *sub_sha1, char *displaypath)

Lets mark these strings as 'const char *' since you aren't modifying
them in the function, only using them.  Also it may make more sense to
pass the 'struct object_id' instead of a hex string of the object id.
Then you can call the necessary 'oid_to_hex' function when you are
adding the object id to the argv array.  That would also eliminate an
allocation in 'status_submodule'.

Also eliminates the need to use the word 'sha1' as we want to eliminate
its usage in the codebase.

> +{
> +	if (info->quiet)
> +		return;
> +
> +	printf("%c%s %s", state, sub_sha1, displaypath);
> +
> +	if (state == ' ' || state == '+') {
> +		struct argv_array name_rev_args = ARGV_ARRAY_INIT;
> +
> +		argv_array_pushl(&name_rev_args, "print-name-rev",
> +				 path, sub_sha1, NULL);
> +		print_name_rev(name_rev_args.argc, name_rev_args.argv,
> +			       info->prefix);
> +	} else {
> +		printf("\n");
> +	}
> +}
> +
> +static int handle_submodule_head_ref(const char *refname,
> +				     const struct object_id *oid, int flags,
> +				     void *cb_data)
> +{
> +	struct strbuf *output = cb_data;
> +	if (oid)
> +		strbuf_addstr(output, oid_to_hex(oid));

Since we're going to be working with 'struct object_id' instead of
strings this would need to change slightly.  Instead of copying into a
strbuf we could just pass a pointer to a 'struct object_id' and use
'oidcpy' to copy the contents of oid to output.

  struct object_id *output = cb_data;
  if (oid)
    oidcpy(output, oid);

> +	return 0;
> +}
> +
> +static void status_submodule(const struct cache_entry *list_item, void *cb_data)
> +{
> +	struct status_cb *info = cb_data;
> +	char *sub_sha1 = xstrdup(oid_to_hex(&list_item->oid));
> +	char *displaypath;
> +	struct argv_array diff_files_args = ARGV_ARRAY_INIT;

'diff_files_args' needs to be cleared at the end of the function when
doing cleanup to prevent a memory leak.

> +
> +	if (!submodule_from_path(null_sha1, list_item->name))
> +		die(_("no submodule mapping found in .gitmodules for path '%s'"),
> +		      list_item->name);
> +
> +	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
> +
> +	if (list_item->ce_flags) {

Is there a particular flag we are interested in here or only that a flag
is set?

> +		print_status(info, 'U', list_item->name,
> +			     sha1_to_hex(null_sha1), displaypath);

Since we are already using OID's in other parts of this code lets be
consistant and use null_oid instead like 'oid_to_hex(&null_oid)'.

> +		goto cleanup;
> +	}
> +
> +	if (!is_submodule_active(the_repository, list_item->name)) {
> +		print_status(info, '-', list_item->name, sub_sha1, displaypath);
> +		goto cleanup;
> +	}
> +
> +	argv_array_pushl(&diff_files_args, "diff-files",
> +			 "--ignore-submodules=dirty", "--quiet", "--",
> +			 list_item->name, NULL);
> +
> +	if (!cmd_diff_files(diff_files_args.argc, diff_files_args.argv,
> +			    info->prefix)) {
> +		print_status(info, ' ', list_item->name, sub_sha1, displaypath);
> +	} else {
> +		if (!info->cached) {
> +			struct strbuf sb = STRBUF_INIT;
> +			if (head_ref_submodule(list_item->name,
> +					       handle_submodule_head_ref, &sb))
> +				die(_("could not resolve HEAD ref inside the"
> +				      "submodule '%s'"), list_item->name);
> +			print_status(info, '+', list_item->name, sb.buf,
> +				     displaypath);
> +			strbuf_release(&sb);

Like i mentioned above this would change into using a 'struct object_id'
which can be allocated on the stack, eliminating the need for the
allocation and releasing of the strbuf too.

> +		} else {
> +			print_status(info, '+', list_item->name, sub_sha1,
> +				     displaypath);
> +		}
> +	}
> +
> +	if (info->recursive) {
> +		struct child_process cpr = CHILD_PROCESS_INIT;
> +
> +		cpr.git_cmd = 1;
> +		cpr.dir = list_item->name;
> +		prepare_submodule_repo_env(&cpr.env_array);
> +
> +		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,

This bit here doesn't seem right.  'displaypath' can include relative
'..'s if you are not at the root of the project but rather in a
subdirectory.  'super_prefix' has traditionally been defined as the path
from the root of the superproject down to the root of the submodule.
This means that there should not ever be any relative '..'s.

> +				 "submodule--helper", "status", "--recursive",
> +				 NULL);
> +
> +		if (info->cached)
> +			argv_array_push(&cpr.args, "--cached");
> +
> +		if (info->quiet)
> +			argv_array_push(&cpr.args, "--quiet");
> +
> +		if (run_command(&cpr))
> +			die(_("failed to recurse into submodule '%s'"),
> +			      list_item->name);
> +	}
> +
> +cleanup:
> +	free(displaypath);
> +	free(sub_sha1);
> +}
> +
> +static int module_status(int argc, const char **argv, const char *prefix)
> +{
> +	struct status_cb info = STATUS_CB_INIT;
> +	struct pathspec pathspec;
> +	struct module_list list = MODULE_LIST_INIT;
> +	int quiet = 0;
> +	int cached = 0;
> +	int recursive = 0;
> +
> +	struct option module_status_options[] = {
> +		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
> +		OPT_BOOL(0, "cached", &cached, N_("Use commit stored in the index instead of the one stored in the submodule HEAD")),
> +		OPT_BOOL(0, "recursive", &recursive, N_("Recurse into nested submodules")),
> +		OPT_END()
> +	};
> +
> +	const char *const git_submodule_helper_usage[] = {
> +		N_("git submodule status [--quiet] [--cached] [--recursive] [<path>]"),
> +		NULL
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, module_status_options,
> +			     git_submodule_helper_usage, 0);
> +
> +	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
> +		return 1;
> +
> +	info.prefix = prefix;
> +	info.quiet = !!quiet;
> +	info.recursive = !!recursive;
> +	info.cached = !!cached;
> +
> +	gitmodules_config();
> +	for_each_submodule_list(list, status_submodule, &info);
> +
> +	return 0;
> +}
> +
>  static int module_name(int argc, const char **argv, const char *prefix)
>  {
>  	const struct submodule *sub;
> @@ -1306,6 +1456,7 @@ static struct cmd_struct commands[] = {
>  	{"resolve-relative-url-test", resolve_relative_url_test, 0},
>  	{"print-name-rev", print_name_rev, 0},
>  	{"init", module_init, SUPPORT_SUPER_PREFIX},
> +	{"status", module_status, SUPPORT_SUPER_PREFIX},
>  	{"remote-branch", resolve_remote_submodule_branch, 0},
>  	{"push-check", push_check, 0},
>  	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
> diff --git a/git-submodule.sh b/git-submodule.sh
> index e988167e0..51b057d82 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -1005,54 +1005,7 @@ cmd_status()
>  		shift
>  	done
>  
> -	{
> -		git submodule--helper list --prefix "$wt_prefix" "$@" ||
> -		echo "#unmatched" $?
> -	} |
> -	while read -r mode sha1 stage sm_path
> -	do
> -		die_if_unmatched "$mode" "$sha1"
> -		name=$(git submodule--helper name "$sm_path") || exit
> -		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
> -		if test "$stage" = U
> -		then
> -			say "U$sha1 $displaypath"
> -			continue
> -		fi
> -		if ! git submodule--helper is-active "$sm_path" ||
> -		{
> -			! test -d "$sm_path"/.git &&
> -			! test -f "$sm_path"/.git
> -		}
> -		then
> -			say "-$sha1 $displaypath"
> -			continue;
> -		fi
> -		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
> -		then
> -			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
> -			say " $sha1 $displaypath$revname"
> -		else
> -			if test -z "$cached"
> -			then
> -				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
> -			fi
> -			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
> -			say "+$sha1 $displaypath$revname"
> -		fi
> -
> -		if test -n "$recursive"
> -		then
> -			(
> -				prefix="$displaypath/"
> -				sanitize_submodule_env
> -				wt_prefix=
> -				cd "$sm_path" &&
> -				eval cmd_status
> -			) ||
> -			die "$(eval_gettext "Failed to recurse into submodule path '\$sm_path'")"
> -		fi
> -	done
> +	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} "$@"
>  }
>  #
>  # Sync remote urls for submodules
> -- 
> 2.13.0
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 14%]

* Re: [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' from shell to C
  2017-07-24 20:34 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-07-24 21:52   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-24 21:52 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> Port the submodule subcommand 'sync' from shell to C using the same
> mechanism as that used for porting submodule subcommand 'status'.
> Hence, here the function cmd_sync() is ported from shell to C.
> This is done by introducing three functions: module_sync(),
> sync_submodule() and print_default_remote().
> 
> The function print_default_remote() is introduced for getting
> the default remote as stdout.
> 
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
> In this new version of patch, following changes were made:
> * the code use to die when sub->url was found to be NULL. This was not a
>   correct translation of code. It was corrected by using an empty string
>   instead of sub->url.
> * a process was used in the previous patch for registering the submodule
>   url. This was avoided by the suggested changes on the previous patch.
> * some nits were also corrected.
>  
>  builtin/submodule--helper.c | 183 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            |  56 +-------------
>  2 files changed, 184 insertions(+), 55 deletions(-)
> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index b39828174..2d1d3984d 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -44,6 +44,20 @@ static char *get_default_remote(void)
>  	return ret;
>  }
>  
> +static int print_default_remote(int argc, const char **argv, const char *prefix)
> +{
> +	const char *remote;
> +
> +	if (argc != 1)
> +		die(_("submodule--helper print-default-remote takes no arguments"));
> +
> +	remote = get_default_remote();
> +	if (remote)
> +		puts(remote);

Any reason why puts is used instead of a printf function?

> +
> +	return 0;
> +}
> +
>  static int starts_with_dot_slash(const char *str)
>  {
>  	return str[0] == '.' && is_dir_sep(str[1]);
> @@ -379,6 +393,25 @@ static void module_list_active(struct module_list *list)
>  	*list = active_modules;
>  }
>  
> +static char *get_up_path(const char *path)
> +{
> +	int i;
> +	struct strbuf sb = STRBUF_INIT;
> +
> +	for (i = count_slashes(path); i; i--)
> +		strbuf_addstr(&sb, "../");
> +
> +	/*
> +	 * Check if 'path' ends with slash or not
> +	 * for having the same output for dir/sub_dir
> +	 * and dir/sub_dir/
> +	 */
> +	if (!is_dir_sep(path[strlen(path) - 1]))
> +		strbuf_addstr(&sb, "../");
> +
> +	return strbuf_detach(&sb, NULL);
> +}
> +
>  static int module_list(int argc, const char **argv, const char *prefix)
>  {
>  	int i;
> @@ -729,6 +762,154 @@ static int module_name(int argc, const char **argv, const char *prefix)
>  	return 0;
>  }
>  
> +struct sync_cb {
> +	const char *prefix;
> +	unsigned int quiet: 1;
> +	unsigned int recursive: 1;
> +};
> +#define SYNC_CB_INIT { NULL, 0, 0 }
> +
> +static void sync_submodule(const struct cache_entry *list_item, void *cb_data)
> +{
> +	struct sync_cb *info = cb_data;
> +	const struct submodule *sub;
> +	char *sub_key, *remote_key;
> +	char *sub_origin_url, *super_config_url, *displaypath;
> +	struct strbuf sb = STRBUF_INIT;
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +	struct strbuf sub_repo_path = STRBUF_INIT;

Not the most important but you could be a lot more efficient here with
your variables.  You could allocate a single strbuf and reuse it for
'sub_key'. 'remote_key', 'sb', and 'sub_repo_path' as it doesn't look
like you need any of those two at the same time.

> +	char *sub_config_path = NULL;
> +
> +	if (!is_submodule_active(the_repository, list_item->name))
> +		return;
> +
> +	sub = submodule_from_path(null_sha1, list_item->name);

Since is_submodule_active also calls into the submodule-config subsystem
to retrieve a submodule this call should never return a NULL ptr...so it
may be safer to return instead of proceeding if 'sub' ends up being null
here.

> +
> +	if (sub && sub->url) {
> +		if (starts_with_dot_dot_slash(sub->url) || starts_with_dot_slash(sub->url)) {
> +			char *remote_url, *up_path;
> +			char *remote = get_default_remote();
> +			char *remote_key = xstrfmt("remote.%s.url", remote);
> +
> +			if (git_config_get_string(remote_key, &remote_url))
> +				remote_url = xgetcwd();
> +
> +			up_path = get_up_path(list_item->name);
> +			sub_origin_url = relative_url(remote_url, sub->url, up_path);
> +			super_config_url = relative_url(remote_url, sub->url, NULL);
> +
> +			free(remote);
> +			free(remote_key);
> +			free(up_path);
> +			free(remote_url);
> +		} else {
> +			sub_origin_url = xstrdup(sub->url);
> +			super_config_url = xstrdup(sub->url);
> +		}
> +	} else {
> +		sub_origin_url = "";
> +		super_config_url = "";
> +	}
> +
> +	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
> +
> +	if (!info->quiet)
> +		printf(_("Synchronizing submodule url for '%s'\n"),
> +			 displaypath);
> +
> +	sub_key = xstrfmt("submodule.%s.url", sub->name);
> +	if (git_config_set_gently(sub_key, super_config_url))
> +		die(_("failed to register url for submodule path '%s'"),
> +		      displaypath);
> +
> +	if (!is_submodule_populated_gently(list_item->name, NULL))
> +		goto cleanup;
> +
> +	prepare_submodule_repo_env(&cp.env_array);
> +	cp.git_cmd = 1;
> +	cp.dir = list_item->name;
> +	argv_array_pushl(&cp.args, "submodule--helper",
> +			 "print-default-remote", NULL);
> +
> +	if (capture_command(&cp, &sb, 0))
> +		die(_("failed to get the default remote for submodule '%s'"),
> +		      list_item->name);
> +
> +	strbuf_strip_suffix(&sb, "\n");
> +	remote_key = xstrfmt("remote.%s.url", sb.buf);
> +	strbuf_release(&sb);
> +
> +	submodule_to_gitdir(&sub_repo_path, list_item->name);

This function (submodule_to_gitdir) has some poor characteristics in
that it will succeed even if there isn't a configured submodule but
there just happens to be a submodule at the provided path...but it looks
like none of them will affect this as the fist thing this function does
is check if the submodule is active before preceding.

> +	sub_config_path = xstrfmt("%s/config", sub_repo_path.buf);
> +	strbuf_release(&sub_repo_path);
> +
> +	if (git_config_set_in_file_gently(sub_config_path, remote_key, sub_origin_url))
> +		die(_("failed to update remote for submodule '%s'"),
> +		      list_item->name);
> +
> +	if (info->recursive) {
> +		struct child_process cpr = CHILD_PROCESS_INIT;
> +
> +		cpr.git_cmd = 1;
> +		cpr.dir = list_item->name;
> +		prepare_submodule_repo_env(&cpr.env_array);
> +
> +		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,

Same comment as the previous patch here.

> +				 "submodule--helper", "sync", "--recursive",
> +				 NULL);
> +
> +		if (info->quiet)
> +			argv_array_push(&cpr.args, "--quiet");
> +
> +		if (run_command(&cpr))
> +			die(_("failed to recurse into submodule '%s'"),
> +			      list_item->name);
> +	}
> +
> +cleanup:
> +	free(sub_key);
> +	free(super_config_url);
> +	free(displaypath);
> +	free(sub_config_path);
> +	free(sub_origin_url);
> +}
> +
> +static int module_sync(int argc, const char **argv, const char *prefix)
> +{
> +	struct sync_cb info = SYNC_CB_INIT;
> +	struct pathspec pathspec;
> +	struct module_list list = MODULE_LIST_INIT;
> +	int quiet = 0;
> +	int recursive = 0;
> +
> +	struct option module_sync_options[] = {
> +		OPT__QUIET(&quiet, N_("Suppress output of synchronizing submodule url")),
> +		OPT_BOOL(0, "recursive", &recursive,
> +			N_("Recurse into nested submodules")),
> +		OPT_END()
> +	};
> +
> +	const char *const git_submodule_helper_usage[] = {
> +		N_("git submodule--helper sync [--quiet] [--recursive] [<path>]"),
> +		NULL
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, module_sync_options,
> +			     git_submodule_helper_usage, 0);
> +
> +	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
> +		return 1;
> +
> +	info.prefix = prefix;
> +	info.quiet = !!quiet;
> +	info.recursive = !!recursive;
> +
> +	gitmodules_config();
> +	for_each_submodule_list(list, sync_submodule, &info);
> +
> +	return 0;
> +}
> +
>  static int clone_submodule(const char *path, const char *gitdir, const char *url,
>  			   const char *depth, struct string_list *reference,
>  			   int quiet, int progress)
> @@ -1457,6 +1638,8 @@ static struct cmd_struct commands[] = {
>  	{"print-name-rev", print_name_rev, 0},
>  	{"init", module_init, SUPPORT_SUPER_PREFIX},
>  	{"status", module_status, SUPPORT_SUPER_PREFIX},
> +	{"print-default-remote", print_default_remote, 0},
> +	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
>  	{"remote-branch", resolve_remote_submodule_branch, 0},
>  	{"push-check", push_check, 0},
>  	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 51b057d82..6bfc5e17d 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -1037,63 +1037,9 @@ cmd_sync()
>  			;;
>  		esac
>  	done
> -	cd_to_toplevel
> -	{
> -		git submodule--helper list --prefix "$wt_prefix" "$@" ||
> -		echo "#unmatched" $?
> -	} |
> -	while read -r mode sha1 stage sm_path
> -	do
> -		die_if_unmatched "$mode" "$sha1"
> -
> -		# skip inactive submodules
> -		if ! git submodule--helper is-active "$sm_path"
> -		then
> -			continue
> -		fi
> -
> -		name=$(git submodule--helper name "$sm_path")
> -		url=$(git config -f .gitmodules --get submodule."$name".url)
> -
> -		# Possibly a url relative to parent
> -		case "$url" in
> -		./*|../*)
> -			# rewrite foo/bar as ../.. to find path from
> -			# submodule work tree to superproject work tree
> -			up_path="$(printf '%s\n' "$sm_path" | sed "s/[^/][^/]*/../g")" &&
> -			# guarantee a trailing /
> -			up_path=${up_path%/}/ &&
> -			# path from submodule work tree to submodule origin repo
> -			sub_origin_url=$(git submodule--helper resolve-relative-url "$url" "$up_path") &&
> -			# path from superproject work tree to submodule origin repo
> -			super_config_url=$(git submodule--helper resolve-relative-url "$url") || exit
> -			;;
> -		*)
> -			sub_origin_url="$url"
> -			super_config_url="$url"
> -			;;
> -		esac
>  
> -		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
> -		say "$(eval_gettext "Synchronizing submodule url for '\$displaypath'")"
> -		git config submodule."$name".url "$super_config_url"
> -
> -		if test -e "$sm_path"/.git
> -		then
> -		(
> -			sanitize_submodule_env
> -			cd "$sm_path"
> -			remote=$(get_default_remote)
> -			git config remote."$remote".url "$sub_origin_url"
> +	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
>  
> -			if test -n "$recursive"
> -			then
> -				prefix="$prefix$sm_path/"
> -				eval cmd_sync
> -			fi
> -		)
> -		fi
> -	done
>  }
>  
>  cmd_absorbgitdirs()
> -- 
> 2.13.0
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 19%]

* [ANNOUNCE] Git v2.14.0-rc1
@ 2017-07-24 22:44 Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2017-07-24 22:44 UTC (permalink / raw)
  To: git; +Cc: Linux Kernel

A release candidate Git v2.14.0-rc1 is now available for testing
at the usual places.  It is comprised of 708 non-merge commits
since v2.13.0, contributed by 61 people, 14 of which are new faces.

The tarballs are found at:

    https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the
'v2.14.0-rc1' tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.13.0 are as follows.
Welcome to the Git development community!

  A. Wilcox, Ben Peart, Brian Malehorn, James Clarke, Jeff Smith,
  Kaartic Sivaraam, Liam Beguin, Phillip Wood, Rikard Falkeborn,
  Sahil Dua, Samuel Lijin, Stephen Kent, Tyler Brazier, and
  xiaoqiang zhao.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Adam Dinwoodie, Ævar Arnfjörð Bjarmason, Alejandro R. Sedeño,
  Alexander Shopov, Andreas Heiduk, Beat Bolli, Brandon Williams,
  brian m. carlson, Changwoo Ryu, Christian Couder, David Aguilar,
  David Turner, Dennis Kaarsemaker, Dimitriy Ryazantcev, Eric Wong,
  Jean-Noel Avila, Jeff Hostetler, Jeff King, Jiang Xin, Johannes
  Schindelin, Johannes Sixt, Jonathan Nieder, Jonathan Tan, Jordi
  Mas, Junio C Hamano, Kyle J. McKay, Kyle Meyer, Lars Schneider,
  Marc Branchaud, Michael Haggerty, Miguel Torroja, Mike Hommey,
  Nguyễn Thái Ngọc Duy, Patrick Steinhardt, Peter Krefting,
  Prathamesh Chavan, Ralf Thielow, Ramsay Jones, René Scharfe,
  Stefan Beller, Štěpán Němec, Sven Strickroth, SZEDER Gábor,
  Thomas Gummerer, Torsten Bögershausen, Trần Ngọc Quân,
  and Ville Skyttä.

----------------------------------------------------------------

Git 2.14 Release Notes (draft)
==============================

Backward compatibility notes and other notable changes.

 * Use of an empty string as a pathspec element that is used for
   'everything matches' is still warned and Git asks users to use a
   more explicit '.' for that instead.  The hope is that existing
   users will not mind this change, and eventually the warning can be
   turned into a hard error, upgrading the deprecation into removal of
   this (mis)feature.  That is not scheduled to happen in the upcoming
   release (yet).

 * Git now avoids blindly falling back to ".git" when the setup
   sequence said we are _not_ in Git repository.  A corner case that
   happens to work right now may be broken by a call to die("BUG").
   We've tried hard to locate such cases and fixed them, but there
   might still be cases that need to be addressed--bug reports are
   greatly appreciated.

 * The experiment to improve the hunk-boundary selection of textual
   diff output has finished, and the "indent heuristics" has now
   become the default.

 * Git can now be built with PCRE v2 instead of v1 of the PCRE
   library. Replace USE_LIBPCRE=YesPlease with USE_LIBPCRE2=YesPlease
   in existing build scripts to build against the new version.  As the
   upstream PCRE maintainer has abandoned v1 maintenance for all but
   the most critical bug fixes, use of v2 is recommended.


Updates since v2.13
-------------------

UI, Workflows & Features

 * The colors in which "git status --short --branch" showed the names
   of the current branch and its remote-tracking branch are now
   configurable.

 * "git clone" learned the "--no-tags" option not to fetch all tags
   initially, and also set up the tagopt not to follow any tags in
   subsequent fetches.

 * "git archive --format=zip" learned to use zip64 extension when
   necessary to go beyond the 4GB limit.

 * "git reset" learned "--recurse-submodules" option.

 * "git diff --submodule=diff" now recurses into nested submodules.

 * "git repack" learned to accept the --threads=<n> option and pass it
   to pack-objects.

 * "git send-email" learned to run sendemail-validate hook to inspect
   and reject a message before sending it out.

 * There is no good reason why "git fetch $there $sha1" should fail
   when the $sha1 names an object at the tip of an advertised ref,
   even when the other side hasn't enabled allowTipSHA1InWant.

 * The "[includeIf "gitdir:$dir"] path=..." mechanism introduced in
   2.13.0 would canonicalize the path of the gitdir being matched,
   and did not match e.g. "gitdir:~/work/*" against a repo in
   "~/work/main" if "~/work" was a symlink to "/mnt/storage/work".
   Now we match both the resolved canonical path and what "pwd" would
   show. The include will happen if either one matches.

 * The "indent" heuristics is now the default in "diff". The
   diff.indentHeuristic configuration variable can be set to "false"
   for those who do not want it.

 * Many commands learned to pay attention to submodule.recurse
   configuration.

 * The convention for a command line is to follow "git cmdname
   --options" with revisions followed by an optional "--"
   disambiguator and then finally pathspecs.  When "--" is not there,
   we make sure early ones are all interpretable as revs (and do not
   look like paths) and later ones are the other way around.  A
   pathspec with "magic" (e.g. ":/p/a/t/h" that matches p/a/t/h from
   the top-level of the working tree, no matter what subdirectory you
   are working from) are conservatively judged as "not a path", which
   required disambiguation more often.  The command line parser
   learned to say "it's a pathspec" a bit more often when the syntax
   looks like so.

 * Update "perl-compatible regular expression" support to enable JIT
   and also allow linking with the newer PCRE v2 library.

 * "filter-branch" learned a pseudo filter "--setup" that can be used
   to define common functions/variables that can be used by other
   filters.

 * Using "git add d/i/r" when d/i/r is the top of the working tree of
   a separate repository would create a gitlink in the index, which
   would appear as a not-quite-initialized submodule to others.  We
   learned to give warnings when this happens.

 * "git status" learned to optionally give how many stash entries there
   are in its output.

 * "git status" has long shown essentially the same message as "git
   commit"; the message it gives while preparing for the root commit,
   i.e. "Initial commit", was hard to understand for some new users.
   Now it says "No commits yet" to stress more on the current status
   (rather than the commit the user is preparing for, which is more in
   line with the focus of "git commit").

 * "git send-email" now has --batch-size and --relogin-delay options
    which can be used to overcome limitations on SMTP servers that
    restrict on how many of e-mails can be sent in a single session.

 * An old message shown in the commit log template was removed, as it
   has outlived its usefulness.

 * "git pull --rebase --recurse-submodules" learns to rebase the
   branch in the submodules to an updated base.

 * "git log" learned -P as a synonym for --perl-regexp, "git grep"
   already had such a synonym.

 * "git log" didn't understand --regexp-ignore-case when combined with
   --perl-regexp. This has been fixed.

Performance, Internal Implementation, Development Support etc.

 * The default packed-git limit value has been raised on larger
   platforms to save "git fetch" from a (recoverable) failure while
   "gc" is running in parallel.

 * Code to update the cache-tree has been tightened so that we won't
   accidentally write out any 0{40} entry in the tree object.

 * Attempt to allow us notice "fishy" situation where we fail to
   remove the temporary directory used during the test.

 * Travis CI gained a task to format the documentation with both
   AsciiDoc and AsciiDoctor.

 * Some platforms have ulong that is smaller than time_t, and our
   historical use of ulong for timestamp would mean they cannot
   represent some timestamp that the platform allows.  Invent a
   separate and dedicated timestamp_t (so that we can distingiuish
   timestamps and a vanilla ulongs, which along is already a good
   move), and then declare uintmax_t is the type to be used as the
   timestamp_t.

 * We can trigger Windows auto-build tester (credits: Dscho &
   Microsoft) from our existing Travis CI tester now.

 * Conversion from uchar[20] to struct object_id continues.

 * Simplify parse_pathspec() codepath and stop it from looking at the
   default in-core index.

 * Add perf-test for wildmatch.

 * Code from "conversion using external process" codepath has been
   extracted to a separate sub-process.[ch] module.

 * When "git checkout", "git merge", etc. manipulates the in-core
   index, various pieces of information in the index extensions are
   discarded from the original state, as it is usually not the case
   that they are kept up-to-date and in-sync with the operation on the
   main index.  The untracked cache extension is copied across these
   operations now, which would speed up "git status" (as long as the
   cache is properly invalidated).

 * The internal implementation of "git grep" has seen some clean-up.

 * Update the C style recommendation for notes for translators, as
   recent versions of gettext tools can work with our style of
   multi-line comments.

 * The implementation of "ref" API around the "packed refs" have been
   cleaned up, in preparation for further changes.

 * The internal logic used in "git blame" has been libified to make it
   easier to use by cgit.

 * Our code often opens a path to an optional file, to work on its
   contents when we can successfully open it.  We can ignore a failure
   to open if such an optional file does not exist, but we do want to
   report a failure in opening for other reasons (e.g. we got an I/O
   error, or the file is there, but we lack the permission to open).

   The exact errors we need to ignore are ENOENT (obviously) and
   ENOTDIR (less obvious).  Instead of repeating comparison of errno
   with these two constants, introduce a helper function to do so.

 * We often try to open a file for reading whose existence is
   optional, and silently ignore errors from open/fopen; report such
   errors if they are not due to missing files.

 * When an existing repository is used for t/perf testing, we first
   create bit-for-bit copy of it, which may grab a transient state of
   the repository and freeze it into the repository used for testing,
   which then may cause Git operations to fail.  Single out "the index
   being locked" case and forcibly drop the lock from the copy.

 * Three instances of the same helper function have been consolidated
   to one.

 * "fast-import" uses a default pack chain depth that is consistent
   with other parts of the system.

 * A new test to show the interaction between the pattern [^a-z]
   (which matches '/') and a slash in a path has been added.  The
   pattern should not match the slash with "pathmatch", but should
   with "wildmatch".

 * The 'diff-highlight' program (in contrib/) has been restructured
   for easier reuse by an external project 'diff-so-fancy'.

 * A common pattern to free a piece of memory and assign NULL to the
   pointer that used to point at it has been replaced with a new
   FREE_AND_NULL() macro.

 * Traditionally, the default die() routine had a code to prevent it
   from getting called multiple times, which interacted badly when a
   threaded program used it (one downside is that the real error may
   be hidden and instead the only error message given to the user may
   end up being "die recursion detected", which is not very useful).

 * Introduce a "repository" object to eventually make it easier to
   work in multiple repositories (the primary focus is to work with
   the superproject and its submodules) in a single process.

 * Optimize "what are the object names already taken in an alternate
   object database?" query that is used to derive the length of prefix
   an object name is uniquely abbreviated to.

 * The hashmap API has been updated so that data to customize the
   behaviour of the comparison function can be specified at the time a
   hashmap is initialized.

 * The "collision detecting" SHA-1 implementation shipped with 2.13 is
   now integrated into git.git as a submodule (the first submodule to
   ship with git.git). Clone git.git with --recurse-submodules to get
   it. For now a non-submodule copy of the same code is also shipped
   as part of the tree.

 * A recent update made it easier to use "-fsanitize=" option while
   compiling but supported only one sanitize option.  Allow more than
   one to be combined, joined with a comma, like "make SANITIZE=foo,bar".

 * Use "p4 -G" to make "p4 changes" output more Python-friendly
   to parse.

 * We started using "%" PRItime, imitating "%" PRIuMAX and friends, as
   a way to format the internal timestamp value, but this does not
   play well with gettext(1) i18n framework, and causes "make pot"
   that is run by the l10n coordinator to create a broken po/git.pot
   file.  This is a possible workaround for that problem.

 * It turns out that Cygwin also needs the fopen() wrapper that
   returns failure when a directory is opened for reading.

Also contains various documentation updates and code clean-ups.


Fixes since v2.13
-----------------

Unless otherwise noted, all the fixes since v2.13 in the maintenance
track are contained in this release (see the maintenance releases'
notes for details).

 * "git gc" did not interact well with "git worktree"-managed
   per-worktree refs.

 * "git cherry-pick" and other uses of the sequencer machinery
   mishandled a trailer block whose last line is an incomplete line.
   This has been fixed so that an additional sign-off etc. are added
   after completing the existing incomplete line.

 * The codepath in "git am" that is used when running "git rebase"
   leaked memory held for the log message of the commits being rebased.

 * "git clone --config var=val" is a way to populate the
   per-repository configuration file of the new repository, but it did
   not work well when val is an empty string.  This has been fixed.

 * Setting "log.decorate=false" in the configuration file did not take
   effect in v2.13, which has been corrected.

 * A few codepaths in "checkout" and "am" working on an unborn branch
   tried to access an uninitialized piece of memory.

 * The Web interface to gmane news archive is long gone, even though
   the articles are still accessible via NTTP.  Replace the links with
   ones to public-inbox.org.  Because their message identification is
   based on the actual message-id, it is likely that it will be easier
   to migrate away from it if/when necessary.

 * The receive-pack program now makes sure that the push certificate
   records the same set of push options used for pushing.

 * Tests have been updated to pass under GETTEXT_POISON (a mechanism
   to ensure that output strings that should not be translated are
   not translated by mistake), and TravisCI is told to run them.

 * "git checkout --recurse-submodules" did not quite work with a
   submodule that itself has submodules.

 * "pack-objects" can stream a slice of an existing packfile out when
   the pack bitmap can tell that the reachable objects are all needed
   in the output, without inspecting individual objects.  This
   strategy however would not work well when "--local" and other
   options are in use, and need to be disabled.

 * Fix memory leaks pointed out by Coverity (and people).

 * "git read-tree -m" (no tree-ish) gave a nonsense suggestion "use
   --empty if you want to clear the index".  With "-m", such a request
   will still fail anyway, as you'd need to name at least one tree-ish
   to be merged.

 * Make sure our tests would pass when the sources are checked out
   with "platform native" line ending convention by default on
   Windows.  Some "text" files out tests use and the test scripts
   themselves that are meant to be run with /bin/sh, ought to be
   checked out with eol=LF even on Windows.

 * Introduce the BUG() macro to improve die("BUG: ...").

 * Clarify documentation for include.path and includeIf.<condition>.path
   configuration variables.

 * Git sometimes gives an advice in a rhetorical question that does
   not require an answer, which can confuse new users and non native
   speakers.  Attempt to rephrase them.

 * A few http:// links that are redirected to https:// in the
   documentation have been updated to https:// links.

 * "git for-each-ref --format=..." with %(HEAD) in the format used to
   resolve the HEAD symref as many times as it had processed refs,
   which was wasteful, and "git branch" shared the same problem.

 * Regression fix to topic recently merged to 'master'.

 * The shell completion script (in contrib/) learned "git stash" has
   a new "push" subcommand.

 * "git interpret-trailers", when used as GIT_EDITOR for "git commit
   -v", looked for and appended to a trailer block at the very end,
   i.e. at the end of the "diff" output.  The command has been
   corrected to pay attention to the cut-mark line "commit -v" adds to
   the buffer---the real trailer block should appear just before it.

 * A test allowed both "git push" and "git receive-pack" on the other
   end write their traces into the same file.  This is OK on platforms
   that allows atomically appending to a file opened with O_APPEND,
   but on other platforms led to a mangled output, causing
   intermittent test failures.  This has been fixed by disabling
   traces from "receive-pack" in the test.

 * Tag objects, which are not reachable from any ref, that point at
   missing objects were mishandled by "git gc" and friends (they
   should silently be ignored instead)

 * "git describe --contains" penalized light-weight tags so much that
   they were almost never considered.  Instead, give them about the
   same chance to be considered as an annotated tag that is the same
   age as the underlying commit would.

 * The "run-command" API implementation has been made more robust
   against dead-locking in a threaded environment.

 * A recent update to t5545-push-options.sh started skipping all the
   tests in the script when a web server testing is disabled or
   unavailable, not just the ones that require a web server.  Non HTTP
   tests have been salvaged to always run in this script.

 * "git send-email" now uses Net::SMTP::SSL, which is obsolete, only
   when needed.  Recent versions of Net::SMTP can do TLS natively.

 * "foo\bar\baz" in "git fetch foo\bar\baz", even though there is no
   slashes in it, cannot be a nickname for a remote on Windows, as
   that is likely to be a pathname on a local filesystem.

 * "git clean -d" used to clean directories that has ignored files,
   even though the command should not lose ignored ones without "-x".
   "git status --ignored"  did not list ignored and untracked files
   without "-uall".  These have been corrected.

 * The result from "git diff" that compares two blobs, e.g. "git diff
   $commit1:$path $commit2:$path", used to be shown with the full
   object name as given on the command line, but it is more natural to
   use the $path in the output and use it to look up .gitattributes.

 * The "collision detecting" SHA-1 implementation shipped with 2.13
   was quite broken on some big-endian platforms and/or platforms that
   do not like unaligned fetches.  Update to the upstream code which
   has already fixed these issues.

 * "git am -h" triggered a BUG().

 * The interaction of "url.*.insteadOf" and custom URL scheme's
   whitelisting is now documented better.

 * The timestamp of the index file is now taken after the file is
   closed, to help Windows, on which a stale timestamp is reported by
   fstat() on a file that is opened for writing and data was written
   but not yet closed.

 * "git pull --rebase --autostash" didn't auto-stash when the local history
   fast-forwards to the upstream.

 * A flaky test has been corrected.

 * "git $cmd -h" for builtin commands calls the implementation of the
   command (i.e. cmd_$cmd() function) without doing any repository
   set-up, and the commands that expect RUN_SETUP is done by the Git
   potty needs to be prepared to show the help text without barfing.
   (merge d691551192 jk/consistent-h later to maint).

 * Help contributors that visit us at GitHub.

 * "git stash push <pathspec>" did not work from a subdirectory at all.
   Bugfix for a topic in v2.13

 * As there is no portable way to pass timezone information to
   strftime, some output format from "git log" and friends are
   impossible to produce.  Teach our own strbuf_addftime to replace %z
   and %Z with caller-supplied values to help working around this.
   (merge 6eced3ec5e rs/strbuf-addftime-zZ later to maint).

 * "git mergetool" learned to work around a wrapper MacOS X adds
   around underlying meld.

 * An example in documentation that does not work in multi worktree
   configuration has been corrected.

 * The pretty-format specifiers like '%h', '%t', etc. had an
   optimization that no longer works correctly.  In preparation/hope
   of getting it correctly implemented, first discard the optimization
   that is broken.

 * The code to pick up and execute command alias definition from the
   configuration used to switch to the top of the working tree and
   then come back when the expanded alias was executed, which was
   unnecessarilyl complex.  Attempt to simplify the logic by using the
   early-config mechanism that does not chdir around.

 * Fix configuration codepath to pay proper attention to commondir
   that is used in multi-worktree situation, and isolate config API
   into its own header file.
   (merge dc8441fdb4 bw/config-h later to maint).

 * "git add -p" were updated in 2.12 timeframe to cope with custom
   core.commentchar but the implementation was buggy and a
   metacharacter like $ and * did not work.

 * A recent regression in "git rebase -i" has been fixed and tests
   that would have caught it and others have been added.

 * An unaligned 32-bit access in pack-bitmap code has been corrected.

 * Tighten error checks for invalid "git apply" input.

 * The split index code did not honor core.sharedRepository setting
   correctly.

 * The Makefile rule in contrib/subtree for building documentation
   learned to honour USE_ASCIIDOCTOR just like the main documentation
   set does.

 * Code clean-up to fix possible buffer over-reading.
   (merge 2d105451c0 rs/apply-avoid-over-reading later to maint).

 * A few tests that tried to verify the contents of push certificates
   did not use 'git rev-parse' to formulate the line to look for in
   the certificate correctly.

 * Update the character width tables.
   (merge 7560aacd7c bb/unicode-10.0 later to maint).

 * After "git branch --move" of the currently checked out branch, the
   code to walk the reflog of HEAD via "log -g" and friends
   incorrectly stopped at the reflog entry that records the renaming
   of the branch.

 * The rewrite of "git branch --list" using for-each-ref's internals
   that happened in v2.13 regressed its handling of color.branch.local;
   this has been fixed.

 * The build procedure has been improved to allow building and testing
   Git with address sanitizer more easily.
   (merge 425ca6710b jk/build-with-asan later to maint).

 * On Cygwin, similar to Windows, "git push //server/share/repository"
   ought to mean a repository on a network share that can be accessed
   locally, but this did not work correctly due to stripping the double
   slashes at the beginning.
   (merge 496f256989 tb/push-to-cygwin-unc-path later to maint).

 * The progress meter did not give a useful output when we haven't had
   0.5 seconds to measure the throughput during the interval.  Instead
   show the overall throughput rate at the end, which is a much more
   useful number.
   (merge 0fae1e072a rs/progress-overall-throughput-at-the-end later to maint).

 * Code clean-up, that makes us in sync with Debian by one patch.
   (merge 8db1ae5740 jn/hooks-pre-rebase-sample-fix later to maint).

 * We run an early part of "git gc" that deals with refs before
   daemonising (and not under lock) even when running a background
   auto-gc, which caused multiple gc processes attempting to run the
   early part at the same time.  This is now prevented by running the
   early part also under the GC lock.
   (merge c45af94dbc jk/gc-pre-detach-under-hook later to maint).

 * A recent update broke an alias that contained an uppercase letter.
   (merge 643df7e234 js/alias-case-sensitivity later to maint).

 * Other minor doc, test and build updates and code cleanups.
   (merge 3f9c637ec7 pw/unquote-path-in-git-pm later to maint).
   (merge 5053313562 rs/urlmatch-cleanup later to maint).
   (merge 42c78a216e rs/use-div-round-up later to maint).
   (merge 5e8d2729ae rs/wt-status-cleanup later to maint).
   (merge 01826066b0 ks/fix-rebase-doc-picture later to maint).
   (merge f7f6dc340e jk/test-copy-bytes-fix later to maint).
   (merge 9fb9495dae ew/fd-cloexec-fix later to maint).
   (merge 3a33fe5c97 ks/doc-fixes later to maint).

^ permalink raw reply	[relevance 10%]

* Re: [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' from shell to C
  2017-07-24 20:34 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
@ 2017-07-24 23:03   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-24 23:03 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> The same mechanism is used even for porting this submodule
> subcommand, as used in the ported subcommands till now.
> The function cmd_deinit in split up after porting into three
> functions: module_deinit(), for_each_submodule_list() and
> deinit_submodule().
> 
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
>  builtin/submodule--helper.c | 141 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            |  55 +----------------
>  2 files changed, 142 insertions(+), 54 deletions(-)
> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 2d1d3984d..5e84fc42d 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -910,6 +910,146 @@ static int module_sync(int argc, const char **argv, const char *prefix)
>  	return 0;
>  }
>  
> +struct deinit_cb {
> +	const char *prefix;
> +	unsigned int quiet: 1;
> +	unsigned int force: 1;
> +	unsigned int all: 1;
> +};
> +#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
> +
> +static void deinit_submodule(const struct cache_entry *list_item,
> +			     void *cb_data)
> +{
> +	struct deinit_cb *info = cb_data;
> +	const struct submodule *sub;
> +	char *displaypath = NULL;
> +	struct child_process cp_config = CHILD_PROCESS_INIT;
> +	struct strbuf sb_config = STRBUF_INIT;
> +	char *sub_git_dir = xstrfmt("%s/.git", list_item->name);
> +	struct stat st;
> +
> +	sub = submodule_from_path(null_sha1, list_item->name);
> +
> +	if (!sub || !sub->name)
> +		goto cleanup;
> +
> +	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
> +
> +	/* remove the submodule work tree (unless the user already did it) */
> +	if (is_directory(list_item->name)) {
> +		/* protect submodules containing a .git directory */
> +		if (is_git_directory(sub_git_dir))

This may be too strict of a test.  The original code simply checks if
'submodule/.git' is a directory and dies if it is.  This adds additional
checks ensuring it is a gitdir.  If we want to have a straight
conversion from the shell code we should have this only check if it is a
directory.

> +			die(_("Submodule work tree '%s' contains a .git "
> +			      "directory use 'rm -rf' if you really want "
> +			      "to remove it including all of its history"),
> +			      displaypath);
> +
> +		if (!info->force) {
> +			struct child_process cp_rm = CHILD_PROCESS_INIT;
> +			cp_rm.git_cmd = 1;
> +			argv_array_pushl(&cp_rm.args, "rm", "-qn",
> +					 list_item->name, NULL);
> +
> +			if (run_command(&cp_rm))
> +				die(_("Submodule work tree '%s' contains local "
> +				      "modifications; use '-f' to discard them"),
> +				      displaypath);
> +		}
> +
> +		if (!lstat(list_item->name, &st)) {

What's the purpose of the lstat call here?

> +			struct strbuf sb_rm = STRBUF_INIT;
> +			const char *format;
> +
> +			strbuf_addstr(&sb_rm, list_item->name);
> +
> +			if (!remove_dir_recursively(&sb_rm, 0))
> +				format = _("Cleared directory '%s'\n");
> +			else
> +				format = _("Could not remove submodule work tree '%s'\n");
> +
> +			if (!info->quiet)
> +				printf(format, displaypath);
> +
> +			strbuf_release(&sb_rm);
> +		}
> +	}
> +
> +	if (mkdir(list_item->name, st.st_mode))

What should the mode be when making the empty directory? Right now you
have the potential for it to be garbage as 'st' has the potential of not
being set by the lstat call above (since it happens inside the above if
statement).  Also we may not want it to depend on what the permissions
were set as on the filesystem assuming the user didn't already remove
the submodule themselves.

> +		die(_("could not create empty submodule directory %s"),
> +		      displaypath);
> +
> +	cp_config.git_cmd = 1;
> +	argv_array_pushl(&cp_config.args, "config", "--get-regexp", NULL);
> +	argv_array_pushf(&cp_config.args, "submodule.%s\\.", sub->name);
> +
> +	/* remove the .git/config entries (unless the user already did it) */
> +	if (!capture_command(&cp_config, &sb_config, 0) && sb_config.len) {
> +		char *sub_key = xstrfmt("submodule.%s", sub->name);
> +		/*
> +		 * remove the whole section so we have a clean state when
> +		 * the user later decides to init this submodule again
> +		 */
> +		git_config_rename_section_in_file(NULL, sub_key, NULL);
> +		if (!info->quiet)
> +			printf(_("Submodule '%s' (%s) unregistered for path '%s'\n"),
> +				 sub->name, sub->url, displaypath);
> +		free(sub_key);
> +	}
> +
> +cleanup:
> +	free(displaypath);
> +	free(sub_git_dir);
> +	strbuf_release(&sb_config);
> +}
> +
> +static int module_deinit(int argc, const char **argv, const char *prefix)
> +{
> +	struct deinit_cb info = DEINIT_CB_INIT;
> +	struct pathspec pathspec;
> +	struct module_list list = MODULE_LIST_INIT;
> +	int quiet = 0;
> +	int force = 0;
> +	int all = 0;
> +
> +	struct option module_deinit_options[] = {
> +		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
> +		OPT__FORCE(&force, N_("Remove submodule working trees even if they contain local changes")),
> +		OPT_BOOL(0, "all", &all, N_("Unregister all submodules")),
> +		OPT_END()
> +	};
> +
> +	const char *const git_submodule_helper_usage[] = {
> +		N_("git submodule deinit [--quiet] [-f | --force] [--all | [--] [<path>...]]"),
> +		NULL
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, module_deinit_options,
> +			     git_submodule_helper_usage, 0);
> +
> +	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
> +		BUG("module_list_compute should not choke on empty pathspec");
> +
> +	info.prefix = prefix;
> +	info.quiet = !!quiet;
> +	info.all = !!all;
> +	info.force = !!force;
> +
> +	if (all && argc) {
> +		error("pathspec and --all are incompatible");
> +		usage_with_options(git_submodule_helper_usage,
> +				   module_deinit_options);
> +	}
> +
> +	if (!argc && !all)
> +		die(_("Use '--all' if you really want to deinitialize all submodules"));
> +
> +	gitmodules_config();
> +	for_each_submodule_list(list, deinit_submodule, &info);
> +
> +	return 0;
> +}
> +
>  static int clone_submodule(const char *path, const char *gitdir, const char *url,
>  			   const char *depth, struct string_list *reference,
>  			   int quiet, int progress)
> @@ -1640,6 +1780,7 @@ static struct cmd_struct commands[] = {
>  	{"status", module_status, SUPPORT_SUPER_PREFIX},
>  	{"print-default-remote", print_default_remote, 0},
>  	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
> +	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
>  	{"remote-branch", resolve_remote_submodule_branch, 0},
>  	{"push-check", push_check, 0},
>  	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 6bfc5e17d..73e6f093f 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -428,60 +428,7 @@ cmd_deinit()
>  		shift
>  	done
>  
> -	if test -n "$deinit_all" && test "$#" -ne 0
> -	then
> -		echo >&2 "$(eval_gettext "pathspec and --all are incompatible")"
> -		usage
> -	fi
> -	if test $# = 0 && test -z "$deinit_all"
> -	then
> -		die "$(eval_gettext "Use '--all' if you really want to deinitialize all submodules")"
> -	fi
> -
> -	{
> -		git submodule--helper list --prefix "$wt_prefix" "$@" ||
> -		echo "#unmatched" $?
> -	} |
> -	while read -r mode sha1 stage sm_path
> -	do
> -		die_if_unmatched "$mode" "$sha1"
> -		name=$(git submodule--helper name "$sm_path") || exit
> -
> -		displaypath=$(git submodule--helper relative-path "$sm_path" "$wt_prefix")
> -
> -		# Remove the submodule work tree (unless the user already did it)
> -		if test -d "$sm_path"
> -		then
> -			# Protect submodules containing a .git directory
> -			if test -d "$sm_path/.git"
> -			then
> -				die "$(eval_gettext "\
> -Submodule work tree '\$displaypath' contains a .git directory
> -(use 'rm -rf' if you really want to remove it including all of its history)")"
> -			fi
> -
> -			if test -z "$force"
> -			then
> -				git rm -qn "$sm_path" ||
> -				die "$(eval_gettext "Submodule work tree '\$displaypath' contains local modifications; use '-f' to discard them")"
> -			fi
> -			rm -rf "$sm_path" &&
> -			say "$(eval_gettext "Cleared directory '\$displaypath'")" ||
> -			say "$(eval_gettext "Could not remove submodule work tree '\$displaypath'")"
> -		fi
> -
> -		mkdir "$sm_path" || say "$(eval_gettext "Could not create empty submodule directory '\$displaypath'")"
> -
> -		# Remove the .git/config entries (unless the user already did it)
> -		if test -n "$(git config --get-regexp submodule."$name\.")"
> -		then
> -			# Remove the whole section so we have a clean state when
> -			# the user later decides to init this submodule again
> -			url=$(git config submodule."$name".url)
> -			git config --remove-section submodule."$name" 2>/dev/null &&
> -			say "$(eval_gettext "Submodule '\$name' (\$url) unregistered for path '\$displaypath'")"
> -		fi
> -	done
> +	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${force:+--force} ${deinit_all:+--all} "$@"
>  }
>  
>  is_tip_reachable () (
> -- 
> 2.13.0
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 17%]

* Re: [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' from shell to C
  2017-07-24 20:34 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
@ 2017-07-25  0:09   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25  0:09 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> The submodule subcommand 'summary' is ported in the process of
> making git-submodule a builtin. The function cmd_summary() from
> git-submodule.sh is ported to functions module_summary(),
> compute_summary_module_list(), prepare_submodule_summary() and
> print_submodule_summary().
> 
> The first function module_summary() parses the options of submodule
> subcommand and also acts as the front-end of this subcommand.
> After parsing them, it calls the compute_summary_module_list()
> 
> The functions compute_summary_module_list() runs the diff_cmd,
> and generates the modules list, as required by the subcommand.
> The generation of this module list is done by the using the
> callback function submodule_summary_callback(), and stored in the
> structure module_cb.
> 
> Once the module list is generated, prepare_submodule_summary()
> further goes through the list and filters the list, for
> eventually calling the print_submodule_summary() function.
> 
> Finally, the print_submodule_summary() takes care of generating
> and printing the summary for each submodule.
> 
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>

This is a big one, hopefully I can try to understand everything that is
happening.

> ---
> In this new version of patch, following changes were made:
> * the way of generating sub_sha1_src and sub_sha1_dst (abbrev of sha1_src
>   and sha1_dst resp.) were changed. Since there was no direct way of
>   abbrevating a string(sha1_dst), in this patch sha1_dst was converted first
>   to an object id (converting to sha1 was avoided) and then abbrevated using
>   find_unique_abbrev().
> * A few big if() statements were reduced.
> * for reducing the two big if() statements, a new function
>   verify_submodule_object_name() was introduced.
> * this new version also corrects a few other nits.
> 
>  builtin/submodule--helper.c | 428 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            | 182 +------------------
>  2 files changed, 429 insertions(+), 181 deletions(-)
> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 5e84fc42d..94d6254f0 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -13,6 +13,9 @@
>  #include "remote.h"
>  #include "refs.h"
>  #include "connect.h"
> +#include "revision.h"
> +#include "diffcore.h"
> +#include "diff.h"
>  
>  typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
>  				      void *cb_data);
> @@ -762,6 +765,430 @@ static int module_name(int argc, const char **argv, const char *prefix)
>  	return 0;
>  }
>  
> +struct module_cb {
> +	unsigned int mod_src;
> +	unsigned int mod_dst;
> +	struct object_id oid_src;
> +	struct object_id oid_dst;
> +	char status;
> +	const char *sm_path;
> +};
> +#define MODULE_CB_INIT { 0, 0, NULL, NULL, '\0', NULL }
> +
> +struct module_cb_list {
> +	struct module_cb **entries;
> +	int alloc, nr;
> +};
> +#define MODULE_CB_LIST_INIT { NULL, 0, 0 }
> +
> +struct summary_cb {
> +	int argc;
> +	const char **argv;
> +	const char *prefix;
> +	char *diff_cmd;
> +	unsigned int cached: 1;
> +	unsigned int for_status: 1;
> +	unsigned int quiet: 1;
> +	unsigned int files: 1;
> +	int summary_limits;
> +};
> +#define SUMMARY_CB_INIT { 0, NULL, NULL, NULL, 0, 0, 0, 0, 0 }
> +
> +static int verify_submodule_object_name(const char *sm_path, const char *sha1)
> +{
> +	struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
> +
> +	cp_rev_parse.git_cmd = 1;
> +	cp_rev_parse.no_stdout = 1;
> +	cp_rev_parse.dir = sm_path;
> +	prepare_submodule_repo_env(&cp_rev_parse.env_array);
> +
> +	argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
> +			 "--verify", NULL);
> +	argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1);
> +
> +	if (run_command(&cp_rev_parse))
> +		return 1;
> +
> +	return 0;
> +}
> +
> +static void print_submodule_summary(struct summary_cb *info,
> +				    struct module_cb *p)
> +{

This is one large function so it may be difficult to keep track of
everything and I don't know how easy it would be to split.

> +	int missing_src = 0;
> +	int missing_dst = 0;
> +	char *displaypath;
> +	const char *sha1_abbr_src;
> +	const char *sha1_abbr_dst;
> +	struct object_id oid_dst;
> +	int errmsg = 0;
> +	int total_commits = -1;
> +	const char *sha1_dst = oid_to_hex(&p->oid_dst);
> +	const char *sha1_src = oid_to_hex(&p->oid_src);

You have these two variables which are defined as 'const char *' yet you
allocate memory for them at different points via 'xstrdup' and then
never free it.  Really what you should be trying to do is use 'struct
object_id' as much as you can instead of storing the hash in hex form.
that way you can use 'oidcmp' instead of 'strcmp' at some points in the
code.  At one point you then convert back to an oid from hex.  When you
get output from a capture_command call and then dup the hex string, what
you really should do is convert the hex  SHA1 directly to an OID and use
it in OID form as much as possible.  Then you can convert it back to a
hex string at the point of use.  It just seems clunky to pass around hex
strings instead of the OID itself.

> +	char *sm_git_dir = xstrfmt("%s/.git", p->sm_path);

This variable is never freed.

> +	int is_sm_git_dir = 0;
> +
> +	if (!info->cached && !strcmp(sha1_dst, sha1_to_hex(null_sha1))) {
> +		if (S_ISGITLINK(p->mod_dst)) {
> +			struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
> +			struct strbuf sb_rev_parse = STRBUF_INIT;
> +
> +			cp_rev_parse.git_cmd = 1;
> +			cp_rev_parse.no_stderr = 1;
> +			cp_rev_parse.dir = p->sm_path;
> +			prepare_submodule_repo_env(&cp_rev_parse.env_array);
> +
> +			argv_array_pushl(&cp_rev_parse.args,
> +					 "rev-parse", "HEAD", NULL);
> +			if (!capture_command(&cp_rev_parse, &sb_rev_parse, 0)) {
> +				strbuf_strip_suffix(&sb_rev_parse, "\n");
> +				sha1_dst = xstrdup(sb_rev_parse.buf);
> +			}
> +			strbuf_release(&sb_rev_parse);
> +		} else if (S_ISLNK(p->mod_dst) || S_ISREG(p->mod_dst)) {
> +			struct child_process cp_hash_object = CHILD_PROCESS_INIT;
> +			struct strbuf sb_hash_object = STRBUF_INIT;
> +
> +			cp_hash_object.git_cmd = 1;
> +			argv_array_pushl(&cp_hash_object.args,
> +					 "hash-object", p->sm_path,
> +					 NULL);
> +			if (!capture_command(&cp_hash_object,
> +					     &sb_hash_object, 0)) {
> +				strbuf_strip_suffix(&sb_hash_object, "\n");
> +				sha1_dst = xstrdup(sb_hash_object.buf);
> +			}
> +			strbuf_release(&sb_hash_object);
> +		} else {
> +			if (p->mod_dst)
> +				die(_("unexpected mode %d\n"), p->mod_dst);
> +		}
> +	}
> +
> +	if (is_git_directory(sm_git_dir))
> +		is_sm_git_dir = 1;
> +
> +	if (is_sm_git_dir && S_ISGITLINK(p->mod_src))
> +		missing_src = verify_submodule_object_name(p->sm_path,
> +							   sha1_src);
> +
> +	if (is_sm_git_dir && S_ISGITLINK(p->mod_dst))
> +		missing_dst = verify_submodule_object_name(p->sm_path,
> +							   sha1_dst);
> +
> +	displaypath = get_submodule_displaypath(p->sm_path, info->prefix);
> +
> +	if (!missing_dst && !missing_src) {
> +		if (is_sm_git_dir) {
> +			struct child_process cp_rev_list = CHILD_PROCESS_INIT;
> +			struct strbuf sb_rev_list = STRBUF_INIT;
> +			char *range;
> +
> +			if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst))
> +				range = xstrfmt("%s...%s", sha1_src, sha1_dst);
> +			else if (S_ISGITLINK(p->mod_src))
> +				range = xstrdup(sha1_src);
> +			else
> +				range = xstrdup(sha1_dst);
> +
> +			cp_rev_list.git_cmd = 1;
> +			cp_rev_list.dir = p->sm_path;
> +			prepare_submodule_repo_env(&cp_rev_list.env_array);
> +
> +			argv_array_pushl(&cp_rev_list.args, "rev-list",
> +					 "--first-parent", range, "--", NULL);
> +			if (!capture_command(&cp_rev_list, &sb_rev_list, 0)) {
> +				if (sb_rev_list.len)
> +					total_commits = count_lines(sb_rev_list.buf,
> +								    sb_rev_list.len);
> +				else
> +					total_commits = 0;
> +			}
> +
> +			free(range);
> +			strbuf_release(&sb_rev_list);
> +		}
> +	} else {
> +		errmsg = 1;
> +	}
> +
> +	get_oid_hex(sha1_dst, &oid_dst);
> +
> +	sha1_abbr_src = find_unique_abbrev(p->oid_src.hash, 7);
> +	sha1_abbr_dst = find_unique_abbrev(oid_dst.hash, 7);
> +
> +	if (p->status == 'T') {
> +		if (S_ISGITLINK(p->mod_dst))
> +			printf(_("* %s %s(blob)->%s(submodule)"),
> +				 displaypath, sha1_abbr_src,
> +				 sha1_abbr_dst);
> +		else
> +			printf(_("* %s %s(submodule)->%s(blob)"),
> +				 displaypath, sha1_abbr_src,
> +				 sha1_abbr_dst);
> +	} else {
> +			printf("* %s %s...%s", displaypath, sha1_abbr_src,
> +				 sha1_abbr_dst);
> +	}
> +
> +	if (total_commits < 0)
> +		printf(":\n");
> +	else
> +		printf(" (%d):\n", total_commits);
> +
> +	if (errmsg) {
> +		/*
> +		 * Don't give error msg for modification whose dst is not
> +		 * submodule, i.e. deleted or changed to blob
> +		 */
> +		if (S_ISGITLINK(p->mod_src)) {
> +			if (missing_src && missing_dst) {
> +				printf(_("  Warn: %s doesn't contain commits %s and %s\n"),
> +				 displaypath, sha1_src, sha1_dst);
> +			} else if (missing_src) {
> +				printf(_("  Warn: %s doesn't contain commit %s\n"),
> +				 displaypath, sha1_src);
> +			} else {
> +				printf(_("  Warn: %s doesn't contain commit %s\n"),
> +				 displaypath, sha1_dst);
> +			}
> +		}
> +	} else if (is_sm_git_dir) {
> +		struct child_process cp_log = CHILD_PROCESS_INIT;
> +
> +		cp_log.git_cmd = 1;
> +		cp_log.dir = p->sm_path;
> +		prepare_submodule_repo_env(&cp_log.env_array);
> +		argv_array_pushl(&cp_log.args, "log", NULL);
> +
> +		if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst)) {
> +			if (info->summary_limits > 0)
> +				argv_array_pushf(&cp_log.args, "-%d", info->summary_limits);
> +
> +			argv_array_pushl(&cp_log.args, "--pretty=  %m %s",
> +					 "--first-parent", NULL);
> +			argv_array_pushf(&cp_log.args, "%s...%s", sha1_src,
> +					 sha1_dst);
> +		} else if (S_ISGITLINK(p->mod_dst)) {
> +			argv_array_pushl(&cp_log.args, "--pretty=  > %s",
> +					 "-1", sha1_dst, NULL);
> +		} else {
> +			argv_array_pushl(&cp_log.args, "--pretty=  < %s",
> +					 "-1", sha1_src, NULL);
> +		}
> +
> +		run_command(&cp_log);
> +	}
> +	printf("\n");
> +
> +	free(displaypath);
> +}
> +
> +static void prepare_submodule_summary(struct summary_cb *info,
> +				      struct module_cb_list *list)
> +{
> +	int i;
> +	for (i = 0; i < list->nr; i++) {
> +		struct module_cb *p = list->entries[i];
> +		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
> +
> +		if (p->status == 'D' || p->status == 'T') {
> +			print_submodule_summary(info, p);
> +			continue;
> +		}
> +
> +		if (info->for_status) {
> +			char *config_key;
> +			const char *ignore_config = "none";
> +			const char *value;
> +			const struct submodule *sub = submodule_from_path(null_sha1, p->sm_path);
> +
> +			if (sub && p->status != 'A') {
> +				config_key = xstrfmt("submodule.%s.ignore",
> +						     sub->name);
> +				if (!git_config_get_value(config_key, &value))
> +					ignore_config = value;
> +				else if (sub->ignore)
> +					ignore_config = sub->ignore;

Since this is a string it may make sense to use
'git_config_get_string_const' instead of '_value', unless you had a
reason for using value the over string variant.

> +
> +				free(config_key);
> +
> +				if (!strcmp(ignore_config, "all"))
> +					continue;
> +			}
> +		}
> +
> +		/* Also show added or modified modules which are checked out */
> +		cp_rev_parse.dir = p->sm_path;
> +		cp_rev_parse.git_cmd = 1;
> +		cp_rev_parse.no_stderr = 1;
> +		cp_rev_parse.no_stdout = 1;
> +
> +		argv_array_pushl(&cp_rev_parse.args, "rev-parse",
> +				 "--git-dir", NULL);
> +
> +		if (!run_command(&cp_rev_parse))
> +			print_submodule_summary(info, p);
> +	}
> +}
> +
> +static void submodule_summary_callback(struct diff_queue_struct *q,
> +				       struct diff_options *options,
> +				       void *data)
> +{
> +	int i;
> +	struct module_cb_list *list = data;
> +	for (i = 0; i < q->nr; i++) {
> +		struct diff_filepair *p = q->queue[i];
> +		struct module_cb *temp;
> +
> +		if (!S_ISGITLINK(p->one->mode) && !S_ISGITLINK(p->two->mode))
> +			continue;
> +		temp = (struct module_cb*)malloc(sizeof(struct module_cb));
> +		temp->mod_src = p->one->mode;
> +		temp->mod_dst = p->two->mode;
> +		temp->oid_src = p->one->oid;
> +		temp->oid_dst = p->two->oid;
> +		temp->status = p->status;
> +		temp->sm_path = xstrdup(p->one->path);
> +
> +		ALLOC_GROW(list->entries, list->nr + 1, list->alloc);
> +		list->entries[list->nr++] = temp;
> +	}
> +}
> +
> +static int compute_summary_module_list(char *head, struct summary_cb *info)

Make 'head' a 'const char *' since you aren't modifying it in the
function.

> +{
> +	struct argv_array diff_args = ARGV_ARRAY_INIT;
> +	struct rev_info rev;
> +	struct module_cb_list list = MODULE_CB_LIST_INIT;
> +
> +	argv_array_push(&diff_args, info->diff_cmd);
> +	if (info->cached)
> +		argv_array_push(&diff_args, "--cached");
> +	argv_array_pushl(&diff_args, "--ignore-submodules=dirty", "--raw",
> +			 NULL);
> +	if (head)
> +		argv_array_push(&diff_args, head);
> +	argv_array_push(&diff_args, "--");
> +	if (info->argc)
> +		argv_array_pushv(&diff_args, info->argv);
> +
> +	git_config(git_diff_basic_config, NULL);
> +	init_revisions(&rev, info->prefix);
> +	gitmodules_config();
> +	rev.abbrev = 0;
> +	precompose_argv(diff_args.argc, diff_args.argv);

What's the purpose of the precompose call?

> +
> +	diff_args.argc = setup_revisions(diff_args.argc, diff_args.argv,
> +					 &rev, NULL);
> +	rev.diffopt.output_format = DIFF_FORMAT_NO_OUTPUT | DIFF_FORMAT_CALLBACK;
> +	rev.diffopt.format_callback = submodule_summary_callback;
> +	rev.diffopt.format_callback_data = &list;
> +
> +	if (!info->cached) {
> +		if (!strcmp(info->diff_cmd, "diff-index"))
> +			setup_work_tree();
> +		if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
> +			perror("read_cache_preload");
> +			return -1;
> +		}
> +	} else if (read_cache() < 0) {
> +		perror("read_cache");
> +		return -1;
> +	}

I'm not understanding this hunk of code.  Why are you using
read_cache_preload in one part and read_cache in another?  also if you
could use read_index instead of read_cache that would be great.  We want
to reduce the number of cache macros that are used and replace them with
the 'index' flavor.

> +
> +	if (!strcmp(info->diff_cmd, "diff-index"))
> +		run_diff_index(&rev, info->cached);
> +	else
> +		run_diff_files(&rev, 0);
> +	prepare_submodule_summary(info, &list);

'list' has a bunch of allocated entries which need to be freed here.

> +
> +	return 0;
> +
> +}
> +
> +static int module_summary(int argc, const char **argv, const char *prefix)
> +{
> +	struct summary_cb info = SUMMARY_CB_INIT;
> +	int cached = 0;
> +	char *diff_cmd = "diff-index";
> +	int for_status = 0;
> +	int quiet = 0;
> +	int files = 0;
> +	int summary_limits = -1;
> +	struct child_process cp_rev = CHILD_PROCESS_INIT;
> +	char *head;
> +	struct strbuf sb = STRBUF_INIT;
> +
> +	struct option module_summary_options[] = {
> +		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
> +		OPT_BOOL(0, "cached", &cached, N_("Use the commit stored in the index instead of the submodule HEAD")),
> +		OPT_BOOL(0, "files", &files, N_("To compares the commit in the index with that in the submodule HEAD")),
> +		OPT_BOOL(0, "for-status", &for_status, N_("Skip submodules with 'all' ignore_config value")),
> +		OPT_INTEGER('n', "summary-limits", &summary_limits, N_("Limit the summary size")),
> +		OPT_END()
> +	};
> +
> +	const char *const git_submodule_helper_usage[] = {
> +		N_("git submodule--helper summary [<options>] [--] [<path>]"),
> +		NULL
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, module_summary_options,
> +			     git_submodule_helper_usage, 0);
> +
> +	if (!summary_limits)
> +		return 0;
> +
> +	cp_rev.git_cmd = 1;
> +	argv_array_pushl(&cp_rev.args, "rev-parse", "-q", "--verify",
> +			 argc ? argv[0] : "HEAD", NULL);
> +
> +	if (!capture_command(&cp_rev, &sb, 0)) {
> +		strbuf_strip_suffix(&sb, "\n");
> +		if (argc) {
> +			argv++;
> +			argc--;
> +		}
> +	} else if (!argc || !strcmp(argv[0], "HEAD")) {
> +		/* before the first commit: compare with an empty tree */
> +		struct stat st;
> +		unsigned char sha1[20];

Use a 'struct object_id' here.  There's no reason to use sha1's 20 char
arrays anymore.  If you need to use a call that takes an unsigned char
array you can always pass oid.hash.  This limits the amount of
conversion we'll have to do later.

> +		if (fstat(0, &st) < 0 || index_fd(sha1, 0, &st, 2, prefix, 3))

What's the reasoning behind the fstat and index_fd call?  Not saying
they aren't needed I just need more context to understand whats going
on.

> +			die("Unable to add %s to database", sha1);
> +		strbuf_addstr(&sb, sha1_to_hex(sha1));
> +		if (argc) {
> +			argv++;
> +			argc--;
> +		}
> +	} else {
> +		strbuf_addstr(&sb, "HEAD");
> +	}
> +
> +	head = strbuf_detach(&sb, NULL);

So head is a 'char *' and it is never freed.  You should free it at the
end of this function to prevent memleaks.

> +
> +	if (files) {
> +		if (cached)
> +			die(_("The --cached option cannot be used with the --files option"));
> +		diff_cmd = "diff-files";
> +		head = NULL;

If you are assigning NULL here make sure that you free head first as you
could have allocated memory which head is already pointing to.

> +	}
> +
> +	info.argc = argc;
> +	info.argv = argv;
> +	info.prefix = prefix;
> +	info.cached = cached;
> +	info.for_status = for_status;
> +	info.quiet = quiet;
> +	info.files = files;
> +	info.summary_limits = summary_limits;
> +	info.diff_cmd = diff_cmd;

You aren't doing the '!!' trick here like you did in the other patches.
Any reason for that?

> +
> +	return compute_summary_module_list(head, &info);
> +}
> +

-- 
Brandon Williams

^ permalink raw reply	[relevance 8%]

* Re: [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory
  2017-07-24 20:34 ` [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
@ 2017-07-25  0:13   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25  0:13 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> When running 'git submodule foreach' from a subdirectory of your
> repository, nested submodules get a bogus value for $sm_path:
> For a submodule 'sub' that contains a nested submodule 'nested',
> running 'git -C dir submodule foreach echo $path' would report
> path='../nested' for the nested submodule. The first part '../' is
> derived from the logic computing the relative path from $pwd to the
> root of the superproject. The second part is the submodule path inside
> the submodule. This value is of little use and is hard to document.
> 
> There are two different possible solutions that have more value:
> (a) The path value is documented as the path from the toplevel of the
>     superproject to the mount point of the submodule.
>     In this case we would want to have path='sub/nested'.
> 
> (b) As Ramsay noticed the documented value is wrong. For the non-nested
>     case the path is equal to the relative path from $pwd to the
>     submodules working directory. When following this model,
>     the expected value would be path='../sub/nested'.
> 
> The behavior for (b) was introduced in 091a6eb0fe (submodule: drop the
> top-level requirement, 2013-06-16) the intent for $path seemed to be
> relative to $cwd to the submodule worktree, but that did not work for
> nested submodules, as the intermittent submodules were not included in
> the path.
> 
> If we were to fix the meaning of the $path using (a) such that "path"
> is "the path from the toplevel of the superproject to the mount point
> of the submodule", we would break any existing submodule user that runs
> foreach from non-root of the superproject as the non-nested submodule
> '../sub' would change its path to 'sub'.
> 
> If we would fix the meaning of the $path using (b), such that "path"
> is "the relative path from $pwd to the submodule", then we would break
> any user that uses nested submodules (even from the root directory) as
> the 'nested' would become 'sub/nested'.
> 
> Both groups can be found in the wild.  The author has no data if one group
> outweighs the other by large margin, and offending each one seems equally
> bad at first.  However in the authors imagination it is better to go with
> (a) as running from a sub directory sounds like it is carried out
> by a human rather than by some automation task.  With a human on
> the keyboard the feedback loop is short and the changed behavior can be
> adapted to quickly unlike some automation that can break silently.

Great explanation, and I agree with going with choice (a).

> 
> Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  git-submodule.sh             |  1 -
>  t/t7407-submodule-foreach.sh | 36 ++++++++++++++++++++++++++++++++++--
>  2 files changed, 34 insertions(+), 3 deletions(-)
> 
> diff --git a/git-submodule.sh b/git-submodule.sh
> index a427ddafd..493a64372 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -320,7 +320,6 @@ cmd_foreach()
>  				prefix="$prefix$sm_path/"
>  				sanitize_submodule_env
>  				cd "$sm_path" &&
> -				sm_path=$(git submodule--helper relative-path "$sm_path" "$wt_prefix") &&
>  				# we make $path available to scripts ...
>  				path=$sm_path &&
>  				if test $# -eq 1
> diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
> index 6ba5daf42..0663622a4 100755
> --- a/t/t7407-submodule-foreach.sh
> +++ b/t/t7407-submodule-foreach.sh
> @@ -82,9 +82,9 @@ test_expect_success 'test basic "submodule foreach" usage' '
>  
>  cat >expect <<EOF
>  Entering '../sub1'
> -$pwd/clone-foo1-../sub1-$sub1sha1
> +$pwd/clone-foo1-sub1-$sub1sha1
>  Entering '../sub3'
> -$pwd/clone-foo3-../sub3-$sub3sha1
> +$pwd/clone-foo3-sub3-$sub3sha1
>  EOF
>  
>  test_expect_success 'test "submodule foreach" from subdirectory' '
> @@ -196,6 +196,38 @@ test_expect_success 'test messages from "foreach --recursive" from subdirectory'
>  	) &&
>  	test_i18ncmp expect actual
>  '
> +sub1sha1=$(cd clone2/sub1 && git rev-parse HEAD)
> +sub2sha1=$(cd clone2/sub2 && git rev-parse HEAD)
> +sub3sha1=$(cd clone2/sub3 && git rev-parse HEAD)
> +nested1sha1=$(cd clone2/nested1 && git rev-parse HEAD)
> +nested2sha1=$(cd clone2/nested1/nested2 && git rev-parse HEAD)
> +nested3sha1=$(cd clone2/nested1/nested2/nested3 && git rev-parse HEAD)
> +submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEAD)
> +
> +cat >expect <<EOF
> +Entering '../nested1'
> +$pwd/clone2-nested1-nested1-$nested1sha1
> +Entering '../nested1/nested2'
> +$pwd/clone2/nested1-nested2-nested2-$nested2sha1
> +Entering '../nested1/nested2/nested3'
> +$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
> +Entering '../nested1/nested2/nested3/submodule'
> +$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
> +Entering '../sub1'
> +$pwd/clone2-foo1-sub1-$sub1sha1
> +Entering '../sub2'
> +$pwd/clone2-foo2-sub2-$sub2sha1
> +Entering '../sub3'
> +$pwd/clone2-foo3-sub3-$sub3sha1
> +EOF
> +
> +test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
> +	(
> +		cd clone2/untracked &&
> +		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
> +	) &&
> +	test_i18ncmp expect actual
> +'
>  
>  cat > expect <<EOF
>  nested1-nested1
> -- 
> 2.13.0
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 9%]

* Re: [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path'
  2017-07-24 20:34 ` [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
@ 2017-07-25  0:15   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25  0:15 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> As using a variable '$path' may be harmful to users due to
> capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
> use $path variable in eval_gettext string, 2012-04-17). Adjust
> the documentation to advocate for using $sm_path,  which contains
> the same value. We still make the 'path' variable available and
> document it as a deprecated synonym of 'sm_path'.

I assume then at some point we would want to drop support for 'path' via
a normal deprecation cycle (whatever that might be, 6 months or a year
or more).

> 
> Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
>  Documentation/git-submodule.txt | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index ff612001d..a23baef62 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -183,12 +183,14 @@ information too.
>  
>  foreach [--recursive] <command>::
>  	Evaluates an arbitrary shell command in each checked out submodule.
> -	The command has access to the variables $name, $path, $sha1 and
> +	The command has access to the variables $name, $sm_path, $sha1 and
>  	$toplevel:
>  	$name is the name of the relevant submodule section in `.gitmodules`,
> -	$path is the name of the submodule directory relative to the
> -	superproject, $sha1 is the commit as recorded in the superproject,
> -	and $toplevel is the absolute path to the top-level of the superproject.
> +	$sm_path is the path of the submodule as recorded in the superproject,
> +	$sha1 is the commit as recorded in the superproject, and
> +	$toplevel is the absolute path to the top-level of the superproject.
> +	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
> +	variable is now a deprecated synonym of '$sm_path' variable.
>  	Any submodules defined in the superproject but not checked out are
>  	ignored by this command. Unless given `--quiet`, foreach prints the name
>  	of each submodule before evaluating the command.
> -- 
> 2.13.0
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 7%]

* Re: [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath'
  2017-07-24 20:34 ` [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
@ 2017-07-25  0:16   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25  0:16 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> It was observer that the variable '$displaypath' was accessible but

s/observer/observed 

> undocumented. Hence, document it.
> 
> Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
>  Documentation/git-submodule.txt |  6 ++++--
>  t/t7407-submodule-foreach.sh    | 22 +++++++++++-----------
>  2 files changed, 15 insertions(+), 13 deletions(-)
> 
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index 8e7930ebc..0cca702cb 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -183,10 +183,12 @@ information too.
>  
>  foreach [--recursive] <command>::
>  	Evaluates an arbitrary shell command in each checked out submodule.
> -	The command has access to the variables $name, $sm_path, $sha1 and
> -	$toplevel:
> +	The command has access to the variables $name, $sm_path, $displaypath,
> +	$sha1 and $toplevel:
>  	$name is the name of the relevant submodule section in `.gitmodules`,
>  	$sm_path is the path of the submodule as recorded in the superproject,
> +	$displaypath contains the relative path from the current working
> +	directory to the submodules root directory,
>  	$sha1 is the commit as recorded in the superproject, and
>  	$toplevel is the absolute path to its superproject, such that
>  	$toplevel/$sm_path is the absolute path of the submodule.
> diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
> index 0663622a4..6ad57e061 100755
> --- a/t/t7407-submodule-foreach.sh
> +++ b/t/t7407-submodule-foreach.sh
> @@ -82,16 +82,16 @@ test_expect_success 'test basic "submodule foreach" usage' '
>  
>  cat >expect <<EOF
>  Entering '../sub1'
> -$pwd/clone-foo1-sub1-$sub1sha1
> +$pwd/clone-foo1-sub1-../sub1-$sub1sha1
>  Entering '../sub3'
> -$pwd/clone-foo3-sub3-$sub3sha1
> +$pwd/clone-foo3-sub3-../sub3-$sub3sha1
>  EOF
>  
>  test_expect_success 'test "submodule foreach" from subdirectory' '
>  	mkdir clone/sub &&
>  	(
>  		cd clone/sub &&
> -		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
> +		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
>  	) &&
>  	test_i18ncmp expect actual
>  '
> @@ -206,25 +206,25 @@ submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEA
>  
>  cat >expect <<EOF
>  Entering '../nested1'
> -$pwd/clone2-nested1-nested1-$nested1sha1
> +$pwd/clone2-nested1-nested1-../nested1-$nested1sha1
>  Entering '../nested1/nested2'
> -$pwd/clone2/nested1-nested2-nested2-$nested2sha1
> +$pwd/clone2/nested1-nested2-nested2-../nested1/nested2-$nested2sha1
>  Entering '../nested1/nested2/nested3'
> -$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
> +$pwd/clone2/nested1/nested2-nested3-nested3-../nested1/nested2/nested3-$nested3sha1
>  Entering '../nested1/nested2/nested3/submodule'
> -$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
> +$pwd/clone2/nested1/nested2/nested3-submodule-submodule-../nested1/nested2/nested3/submodule-$submodulesha1
>  Entering '../sub1'
> -$pwd/clone2-foo1-sub1-$sub1sha1
> +$pwd/clone2-foo1-sub1-../sub1-$sub1sha1
>  Entering '../sub2'
> -$pwd/clone2-foo2-sub2-$sub2sha1
> +$pwd/clone2-foo2-sub2-../sub2-$sub2sha1
>  Entering '../sub3'
> -$pwd/clone2-foo3-sub3-$sub3sha1
> +$pwd/clone2-foo3-sub3-../sub3-$sub3sha1
>  EOF
>  
>  test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
>  	(
>  		cd clone2/untracked &&
> -		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
> +		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
>  	) &&
>  	test_i18ncmp expect actual
>  '
> -- 
> 2.13.0
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 7%]

* Re: [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C
  2017-07-24 20:34 ` [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
@ 2017-07-25  0:29   ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25  0:29 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, sbeller, christian.couder

On 07/25, Prathamesh Chavan wrote:
> This aims to make git-submodule foreach a builtin. This is the very
> first step taken in this direction. Hence, 'foreach' is ported to
> submodule--helper, and submodule--helper is called from git-submodule.sh.
> The code is split up to have one function to obtain all the list of
> submodules. This function acts as the front-end of git-submodule foreach
> subcommand. It calls the function for_each_submodule_list, which basically
> loops through the list and calls function fn, which in this case is
> runcommand_in_submodule. This third function is a calling function that
> takes care of running the command in that submodule, and recursively
> perform the same when --recursive is flagged.
> 
> The first function module_foreach first parses the options present in
> argv, and then with the help of module_list_compute, generates the list of
> submodules present in the current working tree.
> 
> The second function for_each_submodule_list traverses through the
> list, and calls function fn (which in case of submodule subcommand
> foreach is runcommand_in_submodule) is called for each entry.
> 
> The third function runcommand_in_submodule, generates a submodule struct sub
> for $name, value and then later prepends name=sub->name; and other
> value assignment to the env argv_array structure of a child_process.
> Also the <command> of submodule-foreach is push to args argv_array
> structure and finally, using run_command the commands are executed
> using a shell.
> 
> The third function also takes care of the recursive flag, by creating
> a separate child_process structure and prepending "--super-prefix displaypath",
> to the args argv_array structure. Other required arguments and the
> input <command> of submodule-foreach is also appended to this argv_array.
> 
> Helped-by: Brandon Williams <bmwill@google.com>
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
>  builtin/submodule--helper.c | 129 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            |  39 +-------------
>  2 files changed, 130 insertions(+), 38 deletions(-)
> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 94d6254f0..be278bf8d 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -765,6 +765,134 @@ static int module_name(int argc, const char **argv, const char *prefix)
>  	return 0;
>  }
>  
> +struct cb_foreach {
> +	int argc;
> +	const char **argv;
> +	const char *prefix;
> +	unsigned int quiet: 1;
> +	unsigned int recursive: 1;
> +};
> +#define CB_FOREACH_INIT { 0, NULL, NULL, 0, 0 }
> +
> +static void runcommand_in_submodule(const struct cache_entry *list_item,
> +				    void *cb_data)
> +{
> +	struct cb_foreach *info = cb_data;
> +	const struct submodule *sub;
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +	char *displaypath;
> +
> +	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
> +
> +	sub = submodule_from_path(null_sha1, list_item->name);
> +
> +	if (!sub)
> +		die(_("No url found for submodule path '%s' in .gitmodules"),
> +		      displaypath);
> +
> +	if (!is_submodule_populated_gently(list_item->name, NULL))
> +		goto cleanup;
> +
> +	prepare_submodule_repo_env(&cp.env_array);
> +	/* For the purpose of executing <command> in the submodule,
> +	 * separate shell is used for the purpose of running the
> +	 * child process.
> +	 */

comment style

> +	cp.use_shell = 1;
> +	cp.dir = list_item->name;
> +
> +	if (info->argc == 1) {

Why are you only exposing these variables if argc == 1?

> +		char *toplevel = xgetcwd();
> +
> +		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
> +		argv_array_pushf(&cp.env_array, "sm_path=%s", list_item->name);
> +		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
> +		argv_array_pushf(&cp.env_array, "sha1=%s",
> +				 oid_to_hex(&list_item->oid));
> +		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
> +
> +		/*
> +		 * Since still the path variable was accessible from the
> +		 * script before porting, it is also made available.
> +		 */
> +		argv_array_pushf(&cp.args, "path=%s; %s",
> +				 list_item->name, info->argv[0]);

This bit looks odd. Why are you appending argv[0] after a semicolon?
Oh...its to handle the funny path stuff.  I'd add a comment indicating
why you have to expose path  via the args argv_array and not the
env_array.

> +		free(toplevel);
> +	} else {
> +		argv_array_pushv(&cp.args, info->argv);
> +	}
> +
> +	if (!info->quiet)
> +		printf(_("Entering '%s'\n"), displaypath);
> +
> +	if (info->argv[0] && run_command(&cp))
> +		die(_("run_command returned non-zero status for %s\n."),
> +		      displaypath);
> +
> +	if (info->recursive) {
> +		struct child_process cpr = CHILD_PROCESS_INIT;
> +
> +		cpr.git_cmd = 1;
> +		cpr.dir = list_item->name;
> +		prepare_submodule_repo_env(&cpr.env_array);
> +
> +		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,

Same comment as a few of the other commands about super-prefix.

> +				 "submodule--helper", "foreach", "--recursive",
> +				 NULL);
> +
> +		if (info->quiet)
> +			argv_array_push(&cpr.args, "--quiet");
> +
> +		argv_array_pushv(&cpr.args, info->argv);
> +
> +		if (run_command(&cpr))
> +			die(_("run_command returned non-zero status while"
> +			      "recursing in the nested submodules of %s\n."),
> +			      displaypath);
> +	}
> +
> +cleanup:
> +	free(displaypath);
> +}
> +
> +static int module_foreach(int argc, const char **argv, const char *prefix)
> +{
> +	struct cb_foreach info;
> +	struct pathspec pathspec;
> +	struct module_list list = MODULE_LIST_INIT;
> +	int quiet = 0;
> +	int recursive = 0;
> +
> +	struct option module_foreach_options[] = {
> +		OPT__QUIET(&quiet, N_("Suppress output of entering each submodule command")),
> +		OPT_BOOL(0, "recursive", &recursive,
> +			 N_("Recurse into nested submodules")),
> +		OPT_END()
> +	};
> +
> +	const char *const git_submodule_helper_usage[] = {
> +		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
> +		NULL
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, module_foreach_options,
> +			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
> +
> +	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
> +		BUG("module_list_compute should not choke on empty pathspec");
> +
> +	info.argc = argc;
> +	info.argv = argv;
> +	info.prefix = prefix;
> +	info.quiet = !!quiet;
> +	info.recursive = !!recursive;
> +
> +	gitmodules_config();
> +	for_each_submodule_list(list, runcommand_in_submodule, &info);
> +
> +	return 0;
> +}
> +
>  struct module_cb {
>  	unsigned int mod_src;
>  	unsigned int mod_dst;
> @@ -2203,6 +2331,7 @@ static struct cmd_struct commands[] = {
>  	{"resolve-relative-url", resolve_relative_url, 0},
>  	{"resolve-relative-url-test", resolve_relative_url_test, 0},
>  	{"print-name-rev", print_name_rev, 0},
> +	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
>  	{"init", module_init, SUPPORT_SUPER_PREFIX},
>  	{"status", module_status, SUPPORT_SUPER_PREFIX},
>  	{"print-default-remote", print_default_remote, 0},
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 493a64372..e25b2c613 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -298,44 +298,7 @@ cmd_foreach()
>  		shift
>  	done
>  
> -	toplevel=$(pwd)
> -
> -	# dup stdin so that it can be restored when running the external
> -	# command in the subshell (and a recursive call to this function)
> -	exec 3<&0
> -
> -	{
> -		git submodule--helper list --prefix "$wt_prefix" ||
> -		echo "#unmatched" $?
> -	} |
> -	while read -r mode sha1 stage sm_path
> -	do
> -		die_if_unmatched "$mode" "$sha1"
> -		if test -e "$sm_path"/.git
> -		then
> -			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
> -			say "$(eval_gettext "Entering '\$displaypath'")"
> -			name=$(git submodule--helper name "$sm_path")
> -			(
> -				prefix="$prefix$sm_path/"
> -				sanitize_submodule_env
> -				cd "$sm_path" &&
> -				# we make $path available to scripts ...
> -				path=$sm_path &&
> -				if test $# -eq 1
> -				then
> -					eval "$1"
> -				else
> -					"$@"
> -				fi &&
> -				if test -n "$recursive"
> -				then
> -					cmd_foreach "--recursive" "$@"
> -				fi
> -			) <&3 3<&- ||
> -			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
> -		fi
> -	done
> +	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
>  }
>  
>  #
> -- 
> 2.13.0
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 9%]

* Re: [PATCH] recursive submodules: detach HEAD from new state
      [irrelevant]     ` <xmqqr2x5bhk7.fsf@gitster.mtv.corp.google.com>
@ 2017-07-25 22:27       ` Stefan Beller
  2017-07-26 19:36         ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-25 22:27 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jonathan Nieder, git

On Mon, Jul 24, 2017 at 3:23 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Also, while I do agree with you that the problem exists, it is
>> unclear why this patch is a solution and not a hack that sweeps a
>> problem under the rug.
>>
>> It is unclear why this "silently detach HEAD without telling the
>> user" is a better solution than erroring out, for example [*1*].
>
> Just to avoid possible confusion; I am not claiming that it would be
> more (or less for that matter) sensible to error out than silently
> detaching HEAD, because I am not giving the reason to substantiate
> the claim and I do not have a strong opinion to favour which one (or
> another potential solution, if any).
>
> I am just saying that the patch that proposes a solution should be
> backed with an explanation why it is a good idea, especially when
> there are obvious alternatives that are not so clearly inferior.
>
> Thanks.

So I took a step back and wrote about different proposals where
we want to go long term. See below. This will help us
figuring out how to approach this bug correctly.
------



RFC: A new type of symbolic refs

A symbolic ref can currently only point at a ref or another symbolic ref.
This proposal show cases different scenarios on how this could change in
the future.



A: HEAD pointing at the superprojects index
===========================================

Introduce a new symbolic ref that points at the superprojects index of
the gitlink. The format is

  "repo:" <superprojects gitdir> '\0' <gitlink-path> '\0'

Ref read operations
-------------------
  e.g. git log HEAD

Just like existing symrefs, the content of the ref will be read and followed.
On reading "repo:", the sha1 will be obtained equivalent to:

    git -C <superproject> ls-files -s <gitlink-path> | awk '{ print $2}'

In case of error
(superproject not found, gitlink path does not exist), the ref is broken and

Ref write operations driven by the submodule, affecting symrefs
---------------------------------------------------------------
  e.g. git checkout <other branch> (in the submodule)

In this scenario only the HEAD is optionally attached to the superproject,
so we can rewrite the HEAD to be anything else, such as a branch just fine.
Once the HEAD is not pointing at the superproject any more, we'll leave the
submodule alone in operations driven by the superproject.

Ref write operations driven by the submodule, affecting target ref
------------------------------------------------------------------
  e.g. git commit, reset --hard, update-ref (in the submodule)

The HEAD stays the same, pointing at the superproject.
The gitlink is changed to the target sha1, using

  git -C <superproject> update-index --add \
      --cacheinfo 160000,$SHA1,<gitlink-path>

This will affect the superprojects index, such that then a commit in
the superproject is needed.

Ref write operations driven by the superproject, changing the gitlink
---------------------------------------------------------------------
  e.g. git checkout <tree-ish>, git reset --hard (in the superproject)

This will change the gitlink in the superprojects index, such that the HEAD
in the submodule changes, which would trigger an update of the
submodules working tree.

Consistency considerations (gc)
-------------------------------
  e.g. git gc --aggressive --prune=now

The repacking logic is already aware of a detached HEAD, such that
using this new symref mechanism would not generate problems as long as
we keep the HEAD attached to the superproject. However when commits/objects
are created while the HEAD is attached to the superproject and then HEAD
switches to a local branch, there are problems with the created objects
as they seem unreachable now.

This problem is not new as a superproject may record submodule objects
that are not reachable from any of the submodule branches. Such objects
fall prey to overzealous packing in the submodule.

This proposal however exposes this problem a lot more, as the submodule
has fewer needs for branches.




B: HEAD pointing at a superprojects branch
==========================================

Instead of pointing at the index of the superproject, we also
encode a branch name:

    repo:" <superprojects gitdir> '\0' <gitlink-path> '\0' branch '\0'

Ref read operations
-------------------
  e.g. git log HEAD

This is similar to the case of pointing at the index, except that the reading
operation reads from the tip of the branch:

    git -C <superproject> ls-tree <superproject branch> -- \
        <gitlink-path> | awk '{ print $3}'

Ref write operations driven by the submodule, affecting symrefs
---------------------------------------------------------------
  e.g. git checkout <other branch> (in the submodule)

HEAD will be pointed at the local target branch, dropping the affliation to
the superproject.

Ref write operations driven by the submodule, affecting target ref
------------------------------------------------------------------
  e.g. git commit, reset --hard, update-ref (in the submodule)

As we're pointing at the superprojects branch, this would have to create
a dummy(?) commit in the superproject, that just changes the submodule
pointer in the superprojects branch, such that the operation of storing
a new sha1 for the submodule is equivalent to

  git -C <superproject> update-index --add \
      --cacheinfo 160000,$SHA1,<gitlink-path>
  git -C <superproject> commit -m "Update submodule"

This behavior in the superproject is similar to Gerrits subscription model
where superprojects are updated from the submodule.

Each operation in the submodule triggers a local superproject commit.

Ref write operations driven by the superproject, changing the gitlink
---------------------------------------------------------------------
  e.g. git merge, git pull (in the superproject)

This will change the gitlink in the superprojects index, such that the HEAD
in the submodule changes, which would trigger an update of the
submodules working tree.

This would require a good merge strategy for submodules, i.e. on merge
the submodule would create a merge commit that is recorded in the
superprojects merge commit.

Consistency considerations (gc)
-------------------------------
  e.g. git gc --aggressive --prune=now

The repacking problem comes with a solution unlike the previous proposal.
This is because any relevant commit in the submodule is recorded in the
superproject via a commit in a branch. Then even non-fast-forward histories
in the submodule can all be kept by walking the superproject and looking at
all gitlink entries of the submodule.



C: All branches are symbolic references to the superproject
===========================================================

Instead of having just HEAD pointed at a superproject, all(!) branches
in the submodule point at the superprojects branch of the same name.
Symbolic refs that resolve to a local sha1 are not allowed, any symbolic ref
ends up pointing at the superproject eventually.
e.g. HEAD points at a submodule branch, which in turn points at
the superproject branch of the same name.

Ref read operations
-------------------
  e.g. git log

HEAD is read, which may be either (a) locally detached or (b) pointing at a
superproject branch. Resolve as in B.

Ref write operations driven by the submodule, affecting symrefs
---------------------------------------------------------------
  e.g. git checkout <other branch> (in the submodule)

As there is no other local branch, HEAD would point at the other submodule
branch, which then points at another branch in the superproject.

Ref write operations driven by the submodule, affecting target ref
------------------------------------------------------------------
  e.g. git commit, reset --hard, update-ref (in the submodule)

  same as B.

Ref write operations driven by the superproject, changing the gitlink
---------------------------------------------------------------------
  e.g. git merge, git pull (in the superproject)

  same as B.

Consistency considerations (gc)
-------------------------------
  e.g. git gc --aggressive --prune=now

As the superproject contains all knowledge, the gc starts with a
walk of all superproject branches, destilling the recorded gitlink entries
and then starts walking in the submodule from all the recorded gitlinks
to create a pack.

gc and repacking would either be forbidden in the submodule or deflected
to the superproject.

^ permalink raw reply	[relevance 20%]

* Re: [PATCH 02/15] submodule: don't use submodule_from_name
      [irrelevant] ` <20170725213928.125998-3-bmwill@google.com>
@ 2017-07-25 23:17   ` Stefan Beller
  2017-07-26 21:06     ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-25 23:17 UTC (permalink / raw)
  To: Brandon Williams, Jens Lehmann; +Cc: git, Jonathan Nieder

On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> The function 'submodule_from_name()' is being used incorrectly here as a
> submodule path is being used instead of a submodule name.  Since the
> correct function to use with a path to a submodule is already being used
> ('submodule_from_path()') let's remove the call to
> 'submodule_from_name()'.

This blames to 851e18c385 (submodule: use new config API for worktree
configurations, 2015-08-17), but that is a refactoring. The issue of using
the path instead of a name was there before that. The actual issue
was introduced in 7dce19d374 (fetch/pull: Add the
--recurse-submodules option, 2010-11-12).

+     name = ce->name;
+     name_for_path =
unsorted_string_list_lookup(&config_name_for_path, ce->name);
+     if (name_for_path)
+         name = name_for_path->util;

Rereading the archives, there was quite some discussion on the design
of these patches, but these lines of code did not get any attention

    https://public-inbox.org/git/4CDB3063.5010801@web.de/

I cc'd Jens in the hope of him having a good memory why he
wrote the code that way. :)

Note that this is the last caller of submodule_from_name being
removed, so I would expect removal of submodule_from_name from
the t/helper/test-submodule-config.c as well as
Documentation/technical/api-submodule-config.txt
in a later part of this series. (Well technically it could go outside
of the series, but in the mean time we'd document and test
dead code)

> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  submodule.c | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/submodule.c b/submodule.c
> index 7e87e4698..fd391aea6 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1177,8 +1177,6 @@ static int get_next_submodule(struct child_process *cp,
>                         continue;
>
>                 submodule = submodule_from_path(&null_oid, ce->name);
> -               if (!submodule)
> -                       submodule = submodule_from_name(&null_oid, ce->name);
>
>                 default_argv = "yes";
>                 if (spf->command_line_option == RECURSE_SUBMODULES_DEFAULT) {
> --
> 2.14.0.rc0.400.g1c36432dff-goog
>

^ permalink raw reply	[relevance 25%]

* Re: [PATCH 03/15] add, reset: ensure submodules can be added or reset
      [irrelevant] ` <20170725213928.125998-4-bmwill@google.com>
@ 2017-07-25 23:33   ` Stefan Beller
  2017-07-25 23:37     ` Brandon Williams
  2017-07-26 21:25     ` Junio C Hamano
  0 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2017-07-25 23:33 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Jonathan Nieder

On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> Commit aee9c7d65 (Submodules: Add the new "ignore" config option for
> diff and status) ...

introduced in 2010, so quite widely spread.

> ...  introduced the ignore configuration option for
> submodules so that configured submodules could be omitted from the
> status and diff commands.  Because this flag is respected in the diff
> machinery it has the unintended consequence of potentially prohibiting
> users from adding or resetting a submodule, even when a path to the
> submodule is explicitly given.
>
> Ensure that submodules can be added or set, even if they are configured
> to be ignored, by setting the `DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG` diff
> flag.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  builtin/add.c   | 1 +
>  builtin/reset.c | 1 +
>  2 files changed, 2 insertions(+)
>
> diff --git a/builtin/add.c b/builtin/add.c
> index e888fb8c5..6f271512f 100644
> --- a/builtin/add.c
> +++ b/builtin/add.c
> @@ -116,6 +116,7 @@ int add_files_to_cache(const char *prefix,
>         rev.diffopt.output_format = DIFF_FORMAT_CALLBACK;
>         rev.diffopt.format_callback = update_callback;
>         rev.diffopt.format_callback_data = &data;
> +       rev.diffopt.flags |= DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;


This flag occurs once in the code base, with the comment:
    /*
     * Unless the user did explicitly request a submodule
     * ignore mode by passing a command line option we do
     * not ignore any changed submodule SHA-1s when
     * comparing index and parent, no matter what is
     * configured. Otherwise we won't commit any
     * submodules which were manually staged, which would
     * be really confusing.
     */
    int diff_flags = DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;

in prepare_commit, so commit ignores the .gitmodules file.

This allows git-add to add ignored submodules, currently ignored submodules
would have to be added using the plumbing
    git update-index --add --cacheinfo 160000,$SHA1,<gitlink>

This makes sense, though a test demonstrating the change in behavior
would be nice, but git-add doesn't seem to change as it doesn't even load
the git modules config?

>         rev.max_count = 0; /* do not compare unmerged paths with stage #2 */
>         run_diff_files(&rev, DIFF_RACY_IS_MODIFIED);
>         return !!data.add_errors;
> diff --git a/builtin/reset.c b/builtin/reset.c
> index 046403ed6..772d078b8 100644
> --- a/builtin/reset.c
> +++ b/builtin/reset.c
> @@ -156,6 +156,7 @@ static int read_from_tree(const struct pathspec *pathspec,
>         opt.output_format = DIFF_FORMAT_CALLBACK;
>         opt.format_callback = update_index_from_diff;
>         opt.format_callback_data = &intent_to_add;
> +       opt.flags |= DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;

same here? Also as this is not failing any test, it may be worth adding one
to document the behavior of the "submodule.<name>.ignore" flag in tests?

>
>         if (do_diff_cache(tree_oid, &opt))
>                 return 1;
> --
> 2.14.0.rc0.400.g1c36432dff-goog
>

^ permalink raw reply	[relevance 26%]

* Re: [PATCH 04/15] submodule--helper: don't overlay config in remote_submodule_branch
      [irrelevant] ` <20170725213928.125998-5-bmwill@google.com>
@ 2017-07-25 23:35   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-25 23:35 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Jonathan Nieder

On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> Don't rely on overlaying the repository's config on top of the
> submodule-config, instead query the repository's config directly for the
> branch field.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>

Reviewed-by: Stefan Beller <sbeller@google.com>

^ permalink raw reply	[relevance 24%]

* Re: [PATCH 05/15] submodule--helper: don't overlay config in update-clone
      [irrelevant] ` <20170725213928.125998-6-bmwill@google.com>
@ 2017-07-25 23:37   ` Stefan Beller
  2017-07-25 23:39     ` Brandon Williams
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-25 23:37 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Jonathan Nieder

On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> Don't rely on overlaying the repository's config on top of the
> submodule-config, instead query the repository's config directly for the
> url and the update strategy configuration.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
...

> +struct submodule_update_strategy submodule_strategy_with_config_overlayed(struct repository *repo,
> +                                                                         const struct submodule *sub)
> +{
> +       struct submodule_update_strategy strat = sub->update_strategy;
> +       const char *update;
> +       char *key;
> +
> +       key = xstrfmt("submodule.%s.update", sub->name);
> +       if (!repo_config_get_string_const(repo, key, &update)) {
> +               strat.command = NULL;
> +               if (!strcmp(update, "none")) {
> +                       strat.type = SM_UPDATE_NONE;
> +               } else if (!strcmp(update, "checkout")) {
> +                       strat.type = SM_UPDATE_CHECKOUT;
> +               } else if (!strcmp(update, "rebase")) {
> +                       strat.type = SM_UPDATE_REBASE;
> +               } else if (!strcmp(update, "merge")) {
> +                       strat.type = SM_UPDATE_MERGE;
> +               } else if (skip_prefix(update, "!", &update)) {
> +                       strat.type = SM_UPDATE_COMMAND;
> +                       strat.command = update;
> +               } else {
> +                       die("invalid submodule update strategy '%s'", update);
> +               }
> +       }

Can this be simplified by reusing
    parse_submodule_update_strategy(value, dest)
?

^ permalink raw reply	[relevance 16%]

* Re: [PATCH 03/15] add, reset: ensure submodules can be added or reset
  2017-07-25 23:33   ` [PATCH 03/15] add, reset: ensure submodules can be added or reset Stefan Beller
@ 2017-07-25 23:37     ` Brandon Williams
  2017-07-26 21:25     ` Junio C Hamano
  1 sibling, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25 23:37 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Jonathan Nieder

On 07/25, Stefan Beller wrote:
> On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> > Commit aee9c7d65 (Submodules: Add the new "ignore" config option for
> > diff and status) ...
> 
> introduced in 2010, so quite widely spread.
> 
> > ...  introduced the ignore configuration option for
> > submodules so that configured submodules could be omitted from the
> > status and diff commands.  Because this flag is respected in the diff
> > machinery it has the unintended consequence of potentially prohibiting
> > users from adding or resetting a submodule, even when a path to the
> > submodule is explicitly given.
> >
> > Ensure that submodules can be added or set, even if they are configured
> > to be ignored, by setting the `DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG` diff
> > flag.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> > ---
> >  builtin/add.c   | 1 +
> >  builtin/reset.c | 1 +
> >  2 files changed, 2 insertions(+)
> >
> > diff --git a/builtin/add.c b/builtin/add.c
> > index e888fb8c5..6f271512f 100644
> > --- a/builtin/add.c
> > +++ b/builtin/add.c
> > @@ -116,6 +116,7 @@ int add_files_to_cache(const char *prefix,
> >         rev.diffopt.output_format = DIFF_FORMAT_CALLBACK;
> >         rev.diffopt.format_callback = update_callback;
> >         rev.diffopt.format_callback_data = &data;
> > +       rev.diffopt.flags |= DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
> 
> 
> This flag occurs once in the code base, with the comment:
>     /*
>      * Unless the user did explicitly request a submodule
>      * ignore mode by passing a command line option we do
>      * not ignore any changed submodule SHA-1s when
>      * comparing index and parent, no matter what is
>      * configured. Otherwise we won't commit any
>      * submodules which were manually staged, which would
>      * be really confusing.
>      */
>     int diff_flags = DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
> 
> in prepare_commit, so commit ignores the .gitmodules file.
> 
> This allows git-add to add ignored submodules, currently ignored submodules
> would have to be added using the plumbing
>     git update-index --add --cacheinfo 160000,$SHA1,<gitlink>
> 
> This makes sense, though a test demonstrating the change in behavior
> would be nice, but git-add doesn't seem to change as it doesn't even load
> the git modules config?

I can add a comment to the code but its already being tested in the
submodule test suite.  The only reason this doesn't cause any changes
now is that the gitmodules config is never loaded, but that may
change if we decide to allow lazy-loading of the gitmodules file (like
the last couple patches in this series do).

-- 
Brandon Williams

^ permalink raw reply	[relevance 22%]

* Re: [PATCH 05/15] submodule--helper: don't overlay config in update-clone
  2017-07-25 23:37   ` [PATCH 05/15] submodule--helper: don't overlay config in update-clone Stefan Beller
@ 2017-07-25 23:39     ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25 23:39 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Jonathan Nieder

On 07/25, Stefan Beller wrote:
> On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> > Don't rely on overlaying the repository's config on top of the
> > submodule-config, instead query the repository's config directly for the
> > url and the update strategy configuration.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> > ---
> ...
> 
> > +struct submodule_update_strategy submodule_strategy_with_config_overlayed(struct repository *repo,
> > +                                                                         const struct submodule *sub)
> > +{
> > +       struct submodule_update_strategy strat = sub->update_strategy;
> > +       const char *update;
> > +       char *key;
> > +
> > +       key = xstrfmt("submodule.%s.update", sub->name);
> > +       if (!repo_config_get_string_const(repo, key, &update)) {
> > +               strat.command = NULL;
> > +               if (!strcmp(update, "none")) {
> > +                       strat.type = SM_UPDATE_NONE;
> > +               } else if (!strcmp(update, "checkout")) {
> > +                       strat.type = SM_UPDATE_CHECKOUT;
> > +               } else if (!strcmp(update, "rebase")) {
> > +                       strat.type = SM_UPDATE_REBASE;
> > +               } else if (!strcmp(update, "merge")) {
> > +                       strat.type = SM_UPDATE_MERGE;
> > +               } else if (skip_prefix(update, "!", &update)) {
> > +                       strat.type = SM_UPDATE_COMMAND;
> > +                       strat.command = update;
> > +               } else {
> > +                       die("invalid submodule update strategy '%s'", update);
> > +               }
> > +       }
> 
> Can this be simplified by reusing
>     parse_submodule_update_strategy(value, dest)
> ?

It would result in a memory leak if we did.  Really I'd like to just
remove this entirely. The only reason this needs to be done is for
checkout, which if we don't have respect the update config it can be
removed.

-- 
Brandon Williams

^ permalink raw reply	[relevance 16%]

* Re: [PATCH 06/15] fetch: don't overlay config with submodule-config
      [irrelevant] ` <20170725213928.125998-7-bmwill@google.com>
@ 2017-07-25 23:44   ` Stefan Beller
  2017-07-25 23:48     ` Brandon Williams
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-25 23:44 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Jonathan Nieder

On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> Don't rely on overlaying the repository's config on top of the
> submodule-config, instead query the repository's config directly for the
> fetch_recurse field.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>

Reviewed-by: Stefan Beller <sbeller@google.com>

> ---
>  builtin/fetch.c |  1 -
>  submodule.c     | 24 +++++++++++++++++-------
>  2 files changed, 17 insertions(+), 8 deletions(-)
>
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index d84c26391..3fe99073d 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -1362,7 +1362,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
>
>         if (recurse_submodules != RECURSE_SUBMODULES_OFF) {
>                 gitmodules_config();
> -               git_config(submodule_config, NULL);
>         }
>
>         if (all) {
> diff --git a/submodule.c b/submodule.c
> index 8b9e48a61..c5058a4b8 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1210,14 +1210,24 @@ static int get_next_submodule(struct child_process *cp,
>
>                 default_argv = "yes";
>                 if (spf->command_line_option == RECURSE_SUBMODULES_DEFAULT) {
> -                       if (submodule &&
> -                           submodule->fetch_recurse !=
> -                                               RECURSE_SUBMODULES_NONE) {
> -                               if (submodule->fetch_recurse ==
> -                                               RECURSE_SUBMODULES_OFF)
> +                       int fetch_recurse = RECURSE_SUBMODULES_NONE;
> +
> +                       if (submodule) {
> +                               char *key;
> +                               const char *value;
> +
> +                               fetch_recurse = submodule->fetch_recurse;
> +                               key = xstrfmt("submodule.%s.fetchRecurseSubmodules", submodule->name);
> +                               if (!repo_config_get_string_const(the_repository, key, &value)) {
> +                                       fetch_recurse = parse_fetch_recurse_submodules_arg(key, value);
> +                               }
> +                               free(key);
> +                       }

I wonder if it would be better to parse this in builtin/fetch.c#git_fetch_config
and then pass it in here as a parameter, instead of looking it up directly here?
That way it is easier to keep track of what a builtin pays attention to.


> +
> +                       if (fetch_recurse != RECURSE_SUBMODULES_NONE) {
> +                               if (fetch_recurse == RECURSE_SUBMODULES_OFF)
>                                         continue;
> -                               if (submodule->fetch_recurse ==
> -                                               RECURSE_SUBMODULES_ON_DEMAND) {
> +                               if (fetch_recurse == RECURSE_SUBMODULES_ON_DEMAND) {
>                                         if (!unsorted_string_list_lookup(&changed_submodule_paths, ce->name))
>                                                 continue;
>                                         default_argv = "on-demand";
> --
> 2.14.0.rc0.400.g1c36432dff-goog
>

^ permalink raw reply	[relevance 24%]

* Re: [PATCH 07/15] submodule: don't rely on overlayed config when setting diffopts
      [irrelevant] ` <20170725213928.125998-8-bmwill@google.com>
@ 2017-07-25 23:46   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-25 23:46 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Jonathan Nieder

On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> Don't rely on overlaying the repository's config on top of the
> submodule-config, instead query the repository's config directory for
> the ignore field.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  submodule.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/submodule.c b/submodule.c
> index c5058a4b8..f86b82fbb 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -165,8 +165,16 @@ void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,
>  {
>         const struct submodule *submodule = submodule_from_path(&null_oid, path);
>         if (submodule) {
> -               if (submodule->ignore)
> -                       handle_ignore_submodules_arg(diffopt, submodule->ignore);
> +               const char *ignore;
> +               char *key;
> +
> +               key = xstrfmt("submodule.%s.ignore", submodule->name);
> +               if (repo_config_get_string_const(the_repository, key, &ignore))
> +                       ignore = submodule->ignore;

Unlike the last patch, we have to use a direct lookup here as
the alternative is hugely painful.


> +               free(key);
> +
> +               if (ignore)
> +                       handle_ignore_submodules_arg(diffopt, ignore);
>                 else if (is_gitmodules_unmerged(&the_index))
>                         DIFF_OPT_SET(diffopt, IGNORE_SUBMODULES);
>         }
> --
> 2.14.0.rc0.400.g1c36432dff-goog
>

^ permalink raw reply	[relevance 16%]

* Re: [PATCH 06/15] fetch: don't overlay config with submodule-config
  2017-07-25 23:44   ` [PATCH 06/15] fetch: don't overlay config with submodule-config Stefan Beller
@ 2017-07-25 23:48     ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-25 23:48 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Jonathan Nieder

On 07/25, Stefan Beller wrote:
> On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> > Don't rely on overlaying the repository's config on top of the
> > submodule-config, instead query the repository's config directly for the
> > fetch_recurse field.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> 
> Reviewed-by: Stefan Beller <sbeller@google.com>
> 
> > ---
> >  builtin/fetch.c |  1 -
> >  submodule.c     | 24 +++++++++++++++++-------
> >  2 files changed, 17 insertions(+), 8 deletions(-)
> >
> > diff --git a/builtin/fetch.c b/builtin/fetch.c
> > index d84c26391..3fe99073d 100644
> > --- a/builtin/fetch.c
> > +++ b/builtin/fetch.c
> > @@ -1362,7 +1362,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
> >
> >         if (recurse_submodules != RECURSE_SUBMODULES_OFF) {
> >                 gitmodules_config();
> > -               git_config(submodule_config, NULL);
> >         }
> >
> >         if (all) {
> > diff --git a/submodule.c b/submodule.c
> > index 8b9e48a61..c5058a4b8 100644
> > --- a/submodule.c
> > +++ b/submodule.c
> > @@ -1210,14 +1210,24 @@ static int get_next_submodule(struct child_process *cp,
> >
> >                 default_argv = "yes";
> >                 if (spf->command_line_option == RECURSE_SUBMODULES_DEFAULT) {
> > -                       if (submodule &&
> > -                           submodule->fetch_recurse !=
> > -                                               RECURSE_SUBMODULES_NONE) {
> > -                               if (submodule->fetch_recurse ==
> > -                                               RECURSE_SUBMODULES_OFF)
> > +                       int fetch_recurse = RECURSE_SUBMODULES_NONE;
> > +
> > +                       if (submodule) {
> > +                               char *key;
> > +                               const char *value;
> > +
> > +                               fetch_recurse = submodule->fetch_recurse;
> > +                               key = xstrfmt("submodule.%s.fetchRecurseSubmodules", submodule->name);
> > +                               if (!repo_config_get_string_const(the_repository, key, &value)) {
> > +                                       fetch_recurse = parse_fetch_recurse_submodules_arg(key, value);
> > +                               }
> > +                               free(key);
> > +                       }
> 
> I wonder if it would be better to parse this in builtin/fetch.c#git_fetch_config
> and then pass it in here as a parameter, instead of looking it up directly here?
> That way it is easier to keep track of what a builtin pays attention to.

Really the fact that you can configure individual submodules in
.gitmodules to be fetched recursively or not is a terrible design IMO.
Also this is a per-submodule configuration so having it in
builtin/fetch.c would be incredibly annoying to handle.

> 
> 
> > +
> > +                       if (fetch_recurse != RECURSE_SUBMODULES_NONE) {
> > +                               if (fetch_recurse == RECURSE_SUBMODULES_OFF)
> >                                         continue;
> > -                               if (submodule->fetch_recurse ==
> > -                                               RECURSE_SUBMODULES_ON_DEMAND) {
> > +                               if (fetch_recurse == RECURSE_SUBMODULES_ON_DEMAND) {
> >                                         if (!unsorted_string_list_lookup(&changed_submodule_paths, ce->name))
> >                                                 continue;
> >                                         default_argv = "on-demand";
> > --
> > 2.14.0.rc0.400.g1c36432dff-goog
> >

-- 
Brandon Williams

^ permalink raw reply	[relevance 24%]

* Re: [PATCH] recursive submodules: detach HEAD from new state
  2017-07-25 22:27       ` Stefan Beller
@ 2017-07-26 19:36         ` Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2017-07-26 19:36 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Jonathan Nieder, git\

Stefan Beller <sbeller@google.com> writes:

> So I took a step back and wrote about different proposals where
> we want to go long term. See below. This will help us
> figuring out how to approach this bug correctly.

Thanks for writing this.

> RFC: A new type of symbolic refs
>
> A symbolic ref can currently only point at a ref or another symbolic ref.
> This proposal show cases different scenarios on how this could change in
> the future.
>
>
>
> A: HEAD pointing at the superprojects index
> ===========================================
>
> Introduce a new symbolic ref that points at the superprojects index of
> the gitlink. The format is
>
>   "repo:" <superprojects gitdir> '\0' <gitlink-path> '\0'
>
> Ref read operations
> -------------------
>   e.g. git log HEAD
>
> Just like existing symrefs, the content of the ref will be read and followed.
> On reading "repo:", the sha1 will be obtained equivalent to:
>
>     git -C <superproject> ls-files -s <gitlink-path> | awk '{ print $2}'
>
> In case of error
> (superproject not found, gitlink path does not exist), the ref is broken and
>
> Ref write operations driven by the submodule, affecting symrefs
> ---------------------------------------------------------------
>   e.g. git checkout <other branch> (in the submodule)
>
> In this scenario only the HEAD is optionally attached to the superproject,
> so we can rewrite the HEAD to be anything else, such as a branch just fine.
> Once the HEAD is not pointing at the superproject any more, we'll leave the
> submodule alone in operations driven by the superproject.

That explains what the proposed code _does_.  It does not explain
why the chosen behaviour is a sensible one.

This illustrates the point I have trouble with when trying to judge
all of these discrete update proposals to submodules.  They only say
"This feature does this in this case, does that in that case,..."
but lack "this is meant to be used when you want to implement the
workflow that goes like this, and fits as a building block at this
point for that workflow. Other elements needed to support that
workflow well are ...".  No proposal gives a big picture and explain
how these small bits fit together.

For example, I would understand better if this write-up of yours
were not organized with the "proposal X added A that behaves this
way and added B that behaves that way" as its major axis, but
instead was written with the workflow that is meant to be realized
as its major axis, e.g.

    A project may want to use submodules as if it is just part of
    superproject.  In such a project, checking out branch X at the
    superproject level, working on files in both superproject and
    submodules, and then committing recursively and pushing the
    results out recursively at the superproject level, all would
    want to affect the same branch X at all levels in the upstream.

may be one possible workflow you want to support.  As one ingredient
to support such structure, the HEAD in the submodule that points at
an index entry in the superproject may be very useful.  After a
recursive checkout at the superproject level, the HEAD of the
submodule ought to be what came from and recorded in the tree in the
superproject, and after a commit in the submodule, the HEAD moves to
the new commit and the entry in the superproject's index also gets
updated which would have a nice property that "commit" in submodule
acts almost like "add" in superproject.  A recursive "git diff" would
show that submodule is clean after such a commit, recursive "push"
would know which branch to push out, etc.

And when operating in such a mode, it would make most sense if "git
checkout" of a different branch Y in a submodule repository is
either forbidden, or should behave as if the submodule directory
were an ordinary directory of the superproject (i.e. causing
recursive checkout of the branch Y at the superproject level).

BUT.

Because none of the proposals paint a big picture (e.g. the big
picture the above hypothetical example gives is that the core
concept of this particular workflow being supported is that
everything recursively stays on the branch with the same name), we
cannot judge if it is sensible for "a new style symref" to be
updated/demoted to a normal branch pointer when "git checkout"
happens.  It is not sensible in such a hypothetical workflow, but it
may be very sensible in another workflow.  Without stating what
big-picture goal is being achieved, it is impossible to see if a
proposal to add/change an individual component that is to be used as
a building block makes sense.

Historically, we can get away without giving choices of "supported
workflows", allowing the user to pick one, and explaining how things
fit together, primarily because the operations that can recurse were
primarily read-only e.g. status, grep, etc., and the supported model
was "the user can be on whatever branch or detached in each
submodule that may or may not be consistent with what happens in the
superproject; it is up to the user to hang themselves with the long
rope".  When allowing potentially destructive operations like
checkout to go recursive [*1*], depending on how the entire tree of
repositories is meant to be managed, sensible mode of operation
would be different, but without defining what various ways "how the
entire tree of repositories is meant to be managed" are supported,
we cannot teach such operations to go recursive in a sensible way.

[Footnote]

*1* Some readers may wonder "checkout is destructive???", and in the
    context of this discussion, it is.  Recursive checkout done at
    the superproject level that rewinds the branch currently checked
    out in a submodule is destructive by potentially losing history,
    and a recursive checkout that checks out a different branch in a
    submodule can be destructive by changing where the next "git push"
    in a submodule would go, depending on how the entire tree of
    repositories is meant to be managed.  In some workflows, always
    detaching HEAD to the commit that is bound to the superproject
    may be _the_ sensible way to recursively check out a branch.  In
    some other workflows, detecting that the submodule is on a
    branch that is not the branch the superproject is checking out
    and erroring out may be more sensible way.

^ permalink raw reply	[relevance 20%]

* [PATCH] submodule: correct error message for missing commits.
@ 2017-07-26 20:08 Stefan Beller
  2017-07-26 20:30 ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-26 20:08 UTC (permalink / raw)
  To: jacob.keller; +Cc: git, Stefan Beller

When a submodule diff should be displayed we currently just add the
submodule objects to the main object store and then e.g. walk the
revision graph and create a summary for that submodule.

It is possible that we are missing the submodule either completely or
partially, which we currently differentiate with different error messages
depending on whether (1) the whole submodule object store is missing or
(2) just the needed for this particular diff. (1) is reported as
"not initialized", and (2) is reported as "commits not present".

If a submodule is deinit'ed its repository data is still around inside
the superproject, such that the diff can still be produced. In that way
the error message (1) is misleading as we can have a diff despite the
submodule being not initialized.

Downgrade the error message (1) to be the same as (2) and just say
the commits are not present, as that is the true reason why the diff
cannot be shown.

Signed-off-by: Stefan Beller <sbeller@google.com>
---

I came across this error message in the series for the
object store modularisation[1], when I was trying to replace
'add_submodule_odb' by a custom loaded object store from a
submodule repo object, which got me thinking on the error
message and the true cause for it.  

While this could go in separately, I may carry it in that
series, as there we'd come up with more error messages
("could not create submodule object store" as well as the
"commits not present", maybe even "submodule not lookup failed")

Thanks,
Stefan

[1] https://public-inbox.org/git/20170706202739.6056-1-sbeller@google.com/
  

 submodule.c                               | 2 +-
 t/t4059-diff-submodule-not-initialized.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/submodule.c b/submodule.c
index 6531c5d609..280c246477 100644
--- a/submodule.c
+++ b/submodule.c
@@ -567,7 +567,7 @@ static void show_submodule_header(FILE *f, const char *path,
 
 	if (add_submodule_odb(path)) {
 		if (!message)
-			message = "(not initialized)";
+			message = "(commits not present)";
 		goto output_header;
 	}
 
diff --git a/t/t4059-diff-submodule-not-initialized.sh b/t/t4059-diff-submodule-not-initialized.sh
index cd70fd5192..49bca7b48d 100755
--- a/t/t4059-diff-submodule-not-initialized.sh
+++ b/t/t4059-diff-submodule-not-initialized.sh
@@ -95,7 +95,7 @@ test_expect_success 'submodule not initialized in new clone' '
 	git clone . sm3 &&
 	git -C sm3 diff-tree -p --no-commit-id --submodule=log HEAD >actual &&
 	cat >expected <<-EOF &&
-	Submodule sm1 $smhead1...$smhead2 (not initialized)
+	Submodule sm1 $smhead1...$smhead2 (commits not present)
 	EOF
 	test_cmp expected actual
 '
-- 
2.14.0.rc0.3.g6c2e499285


^ permalink raw reply	[relevance 30%]

* Re: [PATCH] submodule: correct error message for missing commits.
  2017-07-26 20:08 [PATCH] submodule: correct error message for missing commits Stefan Beller
@ 2017-07-26 20:30 ` Junio C Hamano
  2017-07-26 20:56   ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-07-26 20:30 UTC (permalink / raw)
  To: Stefan Beller; +Cc: jacob.keller, git

Stefan Beller <sbeller@google.com> writes:

> When a submodule diff should be displayed we currently just add the
> submodule objects to the main object store and then e.g. walk the
> revision graph and create a summary for that submodule.
>
> It is possible that we are missing the submodule either completely or
> partially, which we currently differentiate with different error messages
> depending on whether (1) the whole submodule object store is missing or
> (2) just the needed for this particular diff. (1) is reported as
> "not initialized", and (2) is reported as "commits not present".
>
> If a submodule is deinit'ed its repository data is still around inside
> the superproject, such that the diff can still be produced. In that way
> the error message (1) is misleading as we can have a diff despite the
> submodule being not initialized.

This is confusing...  

So are you saying that if we do "submodule init A && submodule
update A" followed by "submodule deinit A", we _could_ show the
difference for submodule A between two commits in the superproject,
because we already have the necessary data for the submodule, but we
_choose_ not to show it because the user told us explicitly that the
submodule is not interesting?

That sounds like a very sensible and user-centric behaviour to me,
and "not initialized" sounds like the right message to give in such
a case (as opposed to "commits not present"---even the user told us
they are not interesting, we may have them, so "not present" is not
just incorrect but irrelevant because that is not the reason why we
are not showing).

Or are you saying that even the user told us that the submodule is
not interesting, if we had "init" it earlier even once, we show the
difference and with a wrong label?  Showing the difference sounds
like a bug that is more severe than using a wrong label to me.

Puzzled.

>
> Downgrade the error message (1) to be the same as (2) and just say
> the commits are not present, as that is the true reason why the diff
> cannot be shown.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>
> I came across this error message in the series for the
> object store modularisation[1], when I was trying to replace
> 'add_submodule_odb' by a custom loaded object store from a
> submodule repo object, which got me thinking on the error
> message and the true cause for it.  
>
> While this could go in separately, I may carry it in that
> series, as there we'd come up with more error messages
> ("could not create submodule object store" as well as the
> "commits not present", maybe even "submodule not lookup failed")
>
> Thanks,
> Stefan
>
> [1] https://public-inbox.org/git/20170706202739.6056-1-sbeller@google.com/
>   
>
>  submodule.c                               | 2 +-
>  t/t4059-diff-submodule-not-initialized.sh | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/submodule.c b/submodule.c
> index 6531c5d609..280c246477 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -567,7 +567,7 @@ static void show_submodule_header(FILE *f, const char *path,
>  
>  	if (add_submodule_odb(path)) {
>  		if (!message)
> -			message = "(not initialized)";
> +			message = "(commits not present)";
>  		goto output_header;
>  	}
>  
> diff --git a/t/t4059-diff-submodule-not-initialized.sh b/t/t4059-diff-submodule-not-initialized.sh
> index cd70fd5192..49bca7b48d 100755
> --- a/t/t4059-diff-submodule-not-initialized.sh
> +++ b/t/t4059-diff-submodule-not-initialized.sh
> @@ -95,7 +95,7 @@ test_expect_success 'submodule not initialized in new clone' '
>  	git clone . sm3 &&
>  	git -C sm3 diff-tree -p --no-commit-id --submodule=log HEAD >actual &&
>  	cat >expected <<-EOF &&
> -	Submodule sm1 $smhead1...$smhead2 (not initialized)
> +	Submodule sm1 $smhead1...$smhead2 (commits not present)
>  	EOF
>  	test_cmp expected actual
>  '

^ permalink raw reply	[relevance 25%]

* Re: [PATCH] submodule: correct error message for missing commits.
  2017-07-26 20:30 ` Junio C Hamano
@ 2017-07-26 20:56   ` Stefan Beller
  2017-07-26 21:10     ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-26 20:56 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jacob Keller, git

On Wed, Jul 26, 2017 at 1:30 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> When a submodule diff should be displayed we currently just add the
>> submodule objects to the main object store and then e.g. walk the
>> revision graph and create a summary for that submodule.
>>
>> It is possible that we are missing the submodule either completely or
>> partially, which we currently differentiate with different error messages
>> depending on whether (1) the whole submodule object store is missing or
>> (2) just the needed for this particular diff. (1) is reported as
>> "not initialized", and (2) is reported as "commits not present".
>>
>> If a submodule is deinit'ed its repository data is still around inside
>> the superproject, such that the diff can still be produced. In that way
>> the error message (1) is misleading as we can have a diff despite the
>> submodule being not initialized.
>
> This is confusing...
>
> So are you saying that if we do "submodule init A && submodule
> update A" followed by "submodule deinit A",

  $ git clone https://gerrit.googlesource.com/gerrit
  $ git show --submodule=log bb798b00bb
  ...
  Submodule plugins/replication ... (not initialized)

  $ git submodule update --init
  $ # a good example of cross repo changes:
  $ git show --submodule=log bb798b00bb
  ...
  Submodule plugins/replication db4aecb2b8...98b7156cee:
  > Stop using WorkQueue#unregisterWorkQueue.
  < PushOne: Remove redundant string concatenation

  $ git submodule deinit -f --all
  $ git show --submodule=log bb798b00bb
  ...
  Submodule plugins/replication db4aecb2b8...98b7156cee:
  > Stop using WorkQueue#unregisterWorkQueue.
  < PushOne: Remove redundant string concatenation

  $ rm -rf .git/modules/*
  $ git show --submodule=log bb798b00bb
  ...
  Submodule plugins/replication ... (not initialized)

> we _could_ show the
> difference for submodule A between two commits in the superproject,
> because we already have the necessary data for the submodule, but we
> _choose_ not to show it because the user told us explicitly that the
> submodule is not interesting?

We _do_ show the submodule as demonstrated by the code sample above
if we possess the objects.

Hence the "not initialized" is not quite technically correct. (After deinit it
is not initialized, but we show nevertheless, so the user perceived
_reason_ why we do not show the submodule is "commits not present".

> That sounds like a very sensible and user-centric behaviour to me,
> and "not initialized" sounds like the right message to give in such
> a case (as opposed to "commits not present"---even the user told us
> they are not interesting, we may have them, so "not present" is not
> just incorrect but irrelevant because that is not the reason why we
> are not showing).

So you are saying we should instead do:

  if (not active)
    message = "not initialized"
  if (problems with object loading)
    message = "commits not present"
  ...

> Or are you saying that even the user told us that the submodule is
> not interesting, if we had "init" it earlier even once, we show the
> difference and with a wrong label?  Showing the difference sounds
> like a bug that is more severe than using a wrong label to me.

I looked through the man pages and they never specify if submodule
activeness affects the superproject diff, so we'd want to change that
so that only active submodules are diffed.

^ permalink raw reply	[relevance 26%]

* Re: [PATCH 02/15] submodule: don't use submodule_from_name
  2017-07-25 23:17   ` [PATCH 02/15] submodule: don't use submodule_from_name Stefan Beller
@ 2017-07-26 21:06     ` Junio C Hamano
  2017-07-30 13:43       ` Jens Lehmann
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-07-26 21:06 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Brandon Williams, Jens Lehmann, git\, Jonathan Nieder

Stefan Beller <sbeller@google.com> writes:

> Rereading the archives, there was quite some discussion on the design
> of these patches, but these lines of code did not get any attention
>
>     https://public-inbox.org/git/4CDB3063.5010801@web.de/
>
> I cc'd Jens in the hope of him having a good memory why he
> wrote the code that way. :)

Thanks for digging.  I wouldn't be surprised if this were a fallback
to help a broken entry in .gitmodules that lack .path variable, but
we shouldn't be sweeping the problem under the rug like that.  

I wonder if we should barf loudly if there shouldn't be a submodule
at that path, i.e.

	if (!submodule)
		die("there is no submodule defined for path '%s'"...);

though.

> Note that this is the last caller of submodule_from_name being
> removed, so I would expect removal of submodule_from_name from
> the t/helper/test-submodule-config.c as well as
> Documentation/technical/api-submodule-config.txt
> in a later part of this series. (Well technically it could go outside
> of the series, but in the mean time we'd document and test
> dead code)

Good thinking.  As this is "cleanup" series, I think it is within
its scope to remove an API function that becomes unused.

>
>> Signed-off-by: Brandon Williams <bmwill@google.com>
>> ---
>>  submodule.c | 2 --
>>  1 file changed, 2 deletions(-)
>>
>> diff --git a/submodule.c b/submodule.c
>> index 7e87e4698..fd391aea6 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>> @@ -1177,8 +1177,6 @@ static int get_next_submodule(struct child_process *cp,
>>                         continue;
>>
>>                 submodule = submodule_from_path(&null_oid, ce->name);
>> -               if (!submodule)
>> -                       submodule = submodule_from_name(&null_oid, ce->name);
>>
>>                 default_argv = "yes";
>>                 if (spf->command_line_option == RECURSE_SUBMODULES_DEFAULT) {
>> --
>> 2.14.0.rc0.400.g1c36432dff-goog
>>

^ permalink raw reply	[relevance 25%]

* Re: [PATCH] submodule: correct error message for missing commits.
  2017-07-26 20:56   ` Stefan Beller
@ 2017-07-26 21:10     ` Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2017-07-26 21:10 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Jacob Keller, git\

Stefan Beller <sbeller@google.com> writes:

> We _do_ show the submodule as demonstrated by the code sample above
> if we possess the objects.
> ...
>> That sounds like a very sensible and user-centric behaviour to me,
>> and "not initialized" sounds like the right message to give in such
>> a case (as opposed to "commits not present"---even the user told us
>> they are not interesting, we may have them, so "not present" is not
>> just incorrect but irrelevant because that is not the reason why we
>> are not showing).
>
> So you are saying we should instead do:
>
>   if (not active)
>     message = "not initialized"
>   if (problems with object loading)
>     message = "commits not present"
>   ...

I think I am.

>> Or are you saying that even the user told us that the submodule is
>> not interesting, if we had "init" it earlier even once, we show the
>> difference and with a wrong label?  Showing the difference sounds
>> like a bug that is more severe than using a wrong label to me.
>
> I looked through the man pages and they never specify if submodule
> activeness affects the superproject diff, so we'd want to change that
> so that only active submodules are diffed.

I would think that would match my expectation more closely; if I
explicitly told Git to "deinit", and still see the diff to distrat
me (i.e. the current behaviour), I would probably feel that it is a
bug.  I do not know about others who are used to the current
beehaviour, though.  Do people actively "deinit" a submodule that
they once "init"ed, and if so for what purpose?  It's not like they
want to release the disk resource, so I'd imagine the only reason is
to reduce the distracting noise, but I'd prefer to hear from real
users rather than speculating.

Thanks.

^ permalink raw reply	[relevance 22%]

* Re: [PATCH 03/15] add, reset: ensure submodules can be added or reset
  2017-07-25 23:33   ` [PATCH 03/15] add, reset: ensure submodules can be added or reset Stefan Beller
  2017-07-25 23:37     ` Brandon Williams
@ 2017-07-26 21:25     ` Junio C Hamano
  2017-07-31 20:50       ` Brandon Williams
  1 sibling, 1 reply; 200+ results
From: Junio C Hamano @ 2017-07-26 21:25 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Brandon Williams, git\, Jonathan Nieder

Stefan Beller <sbeller@google.com> writes:

> On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
>> Commit aee9c7d65 (Submodules: Add the new "ignore" config option for
>> diff and status) ...
>
> introduced in 2010, so quite widely spread.
>
>> ...  introduced the ignore configuration option for
>> submodules so that configured submodules could be omitted from the
>> status and diff commands.  Because this flag is respected in the diff
>> machinery it has the unintended consequence of potentially prohibiting
>> users from adding or resetting a submodule, even when a path to the
>> submodule is explicitly given.
>>
>> Ensure that submodules can be added or set, even if they are configured
>> to be ignored, by setting the `DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG` diff
>> flag.
>>
>> Signed-off-by: Brandon Williams <bmwill@google.com>
>> ---
>>  builtin/add.c   | 1 +
>>  builtin/reset.c | 1 +
>>  2 files changed, 2 insertions(+)
>>
>> diff --git a/builtin/add.c b/builtin/add.c
>> index e888fb8c5..6f271512f 100644
>> --- a/builtin/add.c
>> +++ b/builtin/add.c
>> @@ -116,6 +116,7 @@ int add_files_to_cache(const char *prefix,
>>         rev.diffopt.output_format = DIFF_FORMAT_CALLBACK;
>>         rev.diffopt.format_callback = update_callback;
>>         rev.diffopt.format_callback_data = &data;
>> +       rev.diffopt.flags |= DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
>
>
> This flag occurs once in the code base, with the comment:
>     /*
>      * Unless the user did explicitly request a submodule
>      * ignore mode by passing a command line option we do
>      * not ignore any changed submodule SHA-1s when
>      * comparing index and parent, no matter what is
>      * configured. Otherwise we won't commit any
>      * submodules which were manually staged, which would
>      * be really confusing.
>      */
>     int diff_flags = DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
>
> in prepare_commit, so commit ignores the .gitmodules file.
>
> This allows git-add to add ignored submodules, currently ignored submodules
> would have to be added using the plumbing
>     git update-index --add --cacheinfo 160000,$SHA1,<gitlink>

Let me play devil's advocate (as I have this suspicion that .ignore
thing specific for submodule is probably misdesigned and certainly
its implementation is backwards).  Is the primary use case for this
.ignore thing to be able to do

	git add .

without having to worry about adding the submodule marked as such?  
And if so, wouldn't it surprise these users who do use .ignore if
"git add" suddenly started adding them?

I think the right tool to use these days for excluding some paths
when adding all others is the negative pathspec; perhaps back when
the .ignore thing was added, it didn't exist or not widely known?  

I suspect that it may result in a better system overall if we can
deprecate and remove the submodule-specific .ignore thing.  At
least, I think the DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG is backwards
in that .ignore causes a submodule to be excluded from the diff by
default and forces paths that care about differences to opt into the
"override" thing, which is wrong---the specific UI thing that wants
not to show them should instead opt into ignoring, while keeping the
default not to special case such a flag that can only be set to a
submodule path.

> This makes sense, though a test demonstrating the change in behavior
> would be nice, but git-add doesn't seem to change as it doesn't even load
> the git modules config?
>
>>         rev.max_count = 0; /* do not compare unmerged paths with stage #2 */
>>         run_diff_files(&rev, DIFF_RACY_IS_MODIFIED);
>>         return !!data.add_errors;
>> diff --git a/builtin/reset.c b/builtin/reset.c
>> index 046403ed6..772d078b8 100644
>> --- a/builtin/reset.c
>> +++ b/builtin/reset.c
>> @@ -156,6 +156,7 @@ static int read_from_tree(const struct pathspec *pathspec,
>>         opt.output_format = DIFF_FORMAT_CALLBACK;
>>         opt.format_callback = update_index_from_diff;
>>         opt.format_callback_data = &intent_to_add;
>> +       opt.flags |= DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
>
> same here? Also as this is not failing any test, it may be worth adding one
> to document the behavior of the "submodule.<name>.ignore" flag in tests?
>
>>
>>         if (do_diff_cache(tree_oid, &opt))
>>                 return 1;
>> --
>> 2.14.0.rc0.400.g1c36432dff-goog
>>

^ permalink raw reply	[relevance 25%]

* [PATCH 1/2] alter hash function: expose test dependencies on sha1
      [irrelevant] <20170728171817.21458-1-sbeller@google.com>
@ 2017-07-28 17:18 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-28 17:18 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

DO NOT APPLY.

Alter the hash function such that with this patch
any dependency on sha1 in tests will make the test
fail. This patch applied on master yields this list:

./t0000-basic.sh
./t0002-gitfile.sh
./t0005-signals.sh
./t0013-sha1dc.sh
./t0021-conversion.sh
./t0090-cache-tree.sh
./t1001-read-tree-m-2way.sh
./t1007-hash-object.sh
./t1011-read-tree-sparse-checkout.sh
./t1013-read-tree-submodule.sh
./t1100-commit-tree-options.sh
./t1200-tutorial.sh
./t1300-repo-config.sh
./t1304-default-acl.sh
./t1400-update-ref.sh
./t1411-reflog-show.sh
./t1450-fsck.sh
./t1507-rev-parse-upstream.sh
./t1512-rev-parse-disambiguation.sh
./t1700-split-index.sh
./t2011-checkout-invalid-head.sh
./t2013-checkout-submodule.sh
./t2015-checkout-unborn.sh
./t2017-checkout-orphan.sh
./t2022-checkout-paths.sh
./t2101-update-index-reupdate.sh
./t2107-update-index-basic.sh
./t2203-add-intent.sh
./t3033-merge-toplevel.sh
./t3102-ls-tree-wildcards.sh
./t3103-ls-tree-misc.sh
./t3201-branch-contains.sh
./t3301-notes.sh
./t3305-notes-fanout.sh
./t3306-notes-prune.sh
./t3308-notes-merge.sh
./t3309-notes-merge-auto-resolve.sh
./t3310-notes-merge-manual-resolve.sh
./t3311-notes-merge-fanout.sh
./t3400-rebase.sh
./t3404-rebase-interactive.sh
./t3405-rebase-malformed.sh
./t3408-rebase-multi-line.sh
./t3415-rebase-autosquash.sh
./t3419-rebase-patch-id.sh
./t3421-rebase-topology-linear.sh
./t3501-revert-cherry-pick.sh
./t3502-cherry-pick-merge.sh
./t3503-cherry-pick-root.sh
./t3506-cherry-pick-ff.sh
./t3509-cherry-pick-merge-df.sh
./t3600-rm.sh
./t3700-add.sh
./t3701-add-interactive.sh
./t3702-add-edit.sh
./t3903-stash.sh
./t3905-stash-include-untracked.sh
./t4002-diff-basic.sh
./t4007-rename-3.sh
./t4008-diff-break-rewrite.sh
./t4010-diff-pathspec.sh
./t4011-diff-symlink.sh
./t4013-diff-various.sh
./t4014-format-patch.sh
./t4015-diff-whitespace.sh
./t4020-diff-external.sh
./t4022-diff-rewrite.sh
./t4029-diff-trailing-space.sh
./t4030-diff-textconv.sh
./t4033-diff-patience.sh
./t4034-diff-words.sh
./t4039-diff-assume-unchanged.sh
./t4042-diff-textconv-caching.sh
./t4044-diff-index-unique-abbrev.sh
./t4045-diff-relative.sh
./t4048-diff-combined-binary.sh
./t4050-diff-histogram.sh
./t4052-stat-output.sh
./t4054-diff-bogus-tree.sh
./t4060-diff-submodule-option-diff-format.sh
./t4126-apply-empty.sh
./t4151-am-abort.sh
./t4202-log.sh
./t4205-log-pretty-formats.sh
./t4208-log-magic-pathspec.sh
./t4211-line-log.sh
./t4300-merge-tree.sh
./t5150-request-pull.sh
./t5300-pack-object.sh
./t5306-pack-nobase.sh
./t5308-pack-detect-duplicates.sh
./t5309-pack-delta-cycles.sh
./t5313-pack-bounds-checks.sh
./t5512-ls-remote.sh
./t5515-fetch-merge-logic.sh
./t5516-fetch-push.sh
./t5520-pull.sh
./t5521-pull-options.sh
./t6000-rev-list-misc.sh
./t6012-rev-list-simplify.sh
./t6020-merge-df.sh
./t6022-merge-rename.sh
./t6024-recursive-merge.sh
./t6030-bisect-porcelain.sh
./t6031-merge-filemode.sh
./t6035-merge-dir-to-symlink.sh
./t6300-for-each-ref.sh
./t6500-gc.sh
./t7003-filter-branch.sh
./t7012-skip-worktree-writing.sh
./t7063-status-untracked-cache.sh
./t7102-reset.sh
./t7106-reset-unborn-branch.sh
./t7112-reset-submodule.sh
./t7201-co.sh
./t7400-submodule-basic.sh
./t7506-status-submodule.sh
./t7507-commit-verbose.sh
./t7508-status.sh
./t7600-merge.sh
./t7607-merge-overwrite.sh
./t7609-merge-co-error-msgs.sh
./t8008-blame-formats.sh

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 sha1dc/sha1.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sha1dc/sha1.c b/sha1dc/sha1.c
index 25eded1399..e18acee9ca 100644
--- a/sha1dc/sha1.c
+++ b/sha1dc/sha1.c
@@ -1756,7 +1756,7 @@ static void sha1_process(SHA1_CTX* ctx, const uint32_t block[16])
 void SHA1DCInit(SHA1_CTX* ctx)
 {
 	ctx->total = 0;
-	ctx->ihv[0] = 0x67452301;
+	ctx->ihv[0] = 0x07452301;
 	ctx->ihv[1] = 0xEFCDAB89;
 	ctx->ihv[2] = 0x98BADCFE;
 	ctx->ihv[3] = 0x10325476;
-- 
2.14.0.rc0.3.g6c2e499285


^ permalink raw reply	[relevance 21%]

* [PATCH] Documentation/checkout: clarify submodule HEADs to be detached
@ 2017-07-28 21:40 Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-28 21:40 UTC (permalink / raw)
  To: gitster; +Cc: git, Stefan Beller

Signed-off-by: Stefan Beller <sbeller@google.com>
---

  On top of sb/submodule-recursive-checkout-detach-head
  
  I also checked other man pages such as read-tree, which
  already mentions the behavior as the fix implements:
  
       --[no-]recurse-submodules
           Using --recurse-submodules will update the content of all
           initialized submodules according to the commit recorded in the
           superproject by calling read-tree recursively, also setting the
           submodules HEAD to be detached at that commit.

 Documentation/git-checkout.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index d6399c0af8..84bd323a00 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -262,6 +262,8 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	local modifications in a submodule would be overwritten the checkout
 	will fail unless `-f` is used. If nothing (or --no-recurse-submodules)
 	is used, the work trees of submodules will not be updated.
+	Just like linkgit:git-submodule[1], this will detach the
+	submodules HEAD.
 
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
-- 
2.14.0.rc0.3.g6c2e499285


^ permalink raw reply	[relevance 33%]

* [GSoC][PATCH v2 01/13] submodule--helper: introduce get_submodule_displaypath()
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
@ 2017-07-29 22:23   ` Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
                     ` (10 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

Introduce function get_submodule_displaypath() to replace the code
occurring in submodule_init() for generating displaypath of the
submodule with a call to it.

This new function will also be used in other parts of the system
in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 6abdad329..7af4de09b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -220,6 +220,27 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+
+	if (prefix && super_prefix) {
+		BUG("cannot have prefix '%s' and superprefix '%s'",
+		    prefix, super_prefix);
+	} else if (prefix) {
+		struct strbuf sb = STRBUF_INIT;
+		char *displaypath = xstrdup(relative_path(path, prefix, &sb));
+		strbuf_release(&sb);
+		return displaypath;
+	} else if (super_prefix) {
+		int len = strlen(super_prefix);
+		const char *format = is_dir_sep(super_prefix[len - 1]) ? "%s%s" : "%s/%s";
+		return xstrfmt(format, super_prefix, path);
+	} else {
+		return xstrdup(path);
+	}
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -339,16 +360,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 	/* Only loads from .gitmodules, no overlay with .git/config */
 	gitmodules_config();
-
-	if (prefix && get_super_prefix())
-		die("BUG: cannot have prefix and superprefix");
-	else if (prefix)
-		displaypath = xstrdup(relative_path(path, prefix, &sb));
-	else if (get_super_prefix()) {
-		strbuf_addf(&sb, "%s%s", get_super_prefix(), path);
-		displaypath = strbuf_detach(&sb, NULL);
-	} else
-		displaypath = xstrdup(path);
+	displaypath = get_submodule_displaypath(path, prefix);
 
 	sub = submodule_from_path(null_sha1, path);
 
@@ -363,7 +375,6 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 * Set active flag for the submodule being initialized
 	 */
 	if (!is_submodule_active(the_repository, path)) {
-		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH v2 02/13] submodule--helper: introduce for_each_submodule_list()
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
  2017-07-29 22:23   ` [GSoC][PATCH v2 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
@ 2017-07-29 22:23   ` Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
                     ` (9 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

Introduce function for_each_submodule_list() and
replace a loop in module_init() with a call to it.

The new function will also be used in other parts of the
system in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7af4de09b..e41572f7a 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -14,6 +14,9 @@
 #include "refs.h"
 #include "connect.h"
 
+typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
+				      void *cb_data);
+
 static char *get_default_remote(void)
 {
 	char *dest = NULL, *ret;
@@ -352,17 +355,30 @@ static int module_list(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static void init_submodule(const char *path, const char *prefix, int quiet)
+static void for_each_submodule_list(const struct module_list list,
+				    submodule_list_func_t fn, void *cb_data)
 {
+	int i;
+	for (i = 0; i < list.nr; i++)
+		fn(list.entries[i], cb_data);
+}
+
+struct init_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+};
+#define INIT_CB_INIT { NULL, 0 }
+
+static void init_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct init_cb *info = cb_data;
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	/* Only loads from .gitmodules, no overlay with .git/config */
-	gitmodules_config();
-	displaypath = get_submodule_displaypath(path, prefix);
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
 
-	sub = submodule_from_path(null_sha1, path);
+	sub = submodule_from_path(null_sha1, list_item->name);
 
 	if (!sub)
 		die(_("No url found for submodule path '%s' in .gitmodules"),
@@ -374,7 +390,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_active(the_repository, path)) {
+	if (!is_submodule_active(the_repository, list_item->name)) {
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
@@ -416,7 +432,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 		if (git_config_set_gently(sb.buf, url))
 			die(_("Failed to register url for submodule path '%s'"),
 			    displaypath);
-		if (!quiet)
+		if (!info->quiet)
 			fprintf(stderr,
 				_("Submodule '%s' (%s) registered for path '%s'\n"),
 				sub->name, url, displaypath);
@@ -445,10 +461,10 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 static int module_init(int argc, const char **argv, const char *prefix)
 {
+	struct init_cb info = INIT_CB_INIT;
 	struct pathspec pathspec;
 	struct module_list list = MODULE_LIST_INIT;
 	int quiet = 0;
-	int i;
 
 	struct option module_init_options[] = {
 		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
@@ -473,8 +489,11 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	if (!argc && git_config_get_value_multi("submodule.active"))
 		module_list_active(&list);
 
-	for (i = 0; i < list.nr; i++)
-		init_submodule(list.entries[i]->name, prefix, quiet);
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+
+	gitmodules_config();
+	for_each_submodule_list(list, init_submodule, &info);
 
 	return 0;
 }
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH v2 03/13] submodule: port set_name_rev() from shell to C
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
  2017-07-29 22:23   ` [GSoC][PATCH v2 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
@ 2017-07-29 22:23   ` Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
                     ` (8 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

Function set_name_rev() is ported from git-submodule to the
submodule--helper builtin. The function get_name_rev() generates the
value of the revision name as required, and the function
print_name_rev() handles the formating and printing of the obtained
revision name.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* The variable namerev from print_name_rev is now freed at the end of the
  function.

 builtin/submodule--helper.c | 64 +++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 16 ++----------
 2 files changed, 66 insertions(+), 14 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index e41572f7a..2cb72d68e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -244,6 +244,69 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+static char *get_name_rev(const char *sub_path, const char* object_id)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char ***d;
+
+	static const char *describe_bare[] = {
+		NULL
+	};
+
+	static const char *describe_tags[] = {
+		"--tags", NULL
+	};
+
+	static const char *describe_contains[] = {
+		"--contains", NULL
+	};
+
+	static const char *describe_all_always[] = {
+		"--all", "--always", NULL
+	};
+
+	static const char **describe_argv[] = {
+		describe_bare, describe_tags, describe_contains,
+		describe_all_always, NULL
+	};
+
+	for (d = describe_argv; *d; d++) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+		prepare_submodule_repo_env(&cp.env_array);
+		cp.dir = sub_path;
+		cp.git_cmd = 1;
+		cp.no_stderr = 1;
+
+		argv_array_push(&cp.args, "describe");
+		argv_array_pushv(&cp.args, *d);
+		argv_array_push(&cp.args, object_id);
+
+		if (!capture_command(&cp, &sb, 0) && sb.len) {
+			strbuf_strip_suffix(&sb, "\n");
+			return strbuf_detach(&sb, NULL);
+		}
+
+	}
+
+	strbuf_release(&sb);
+	return NULL;
+}
+
+static int print_name_rev(int argc, const char **argv, const char *prefix)
+{
+	char *namerev;
+	if (argc != 3)
+		die("print-name-rev only accepts two arguments: <path> <sha1>");
+
+	namerev = get_name_rev(argv[1], argv[2]);
+	if (namerev && namerev[0])
+		printf(" (%s)", namerev);
+	printf("\n");
+
+	free(namerev);
+	return 0;
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -1242,6 +1305,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index e131760ee..e988167e0 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -759,18 +759,6 @@ cmd_update()
 	}
 }
 
-set_name_rev () {
-	revname=$( (
-		sanitize_submodule_env
-		cd "$1" && {
-			git describe "$2" 2>/dev/null ||
-			git describe --tags "$2" 2>/dev/null ||
-			git describe --contains "$2" 2>/dev/null ||
-			git describe --all --always "$2"
-		}
-	) )
-	test -z "$revname" || revname=" ($revname)"
-}
 #
 # Show commit summary for submodules in index or working tree
 #
@@ -1042,14 +1030,14 @@ cmd_status()
 		fi
 		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
 		then
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say " $sha1 $displaypath$revname"
 		else
 			if test -z "$cached"
 			then
 				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
 			fi
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say "+$sha1 $displaypath$revname"
 		fi
 
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH v2 04/13] submodule: port submodule subcommand 'status' from shell to C
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (2 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
@ 2017-07-29 22:23   ` " Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
                     ` (7 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

This aims to make git-submodule 'status' a built-in. Hence, the function
cmd_status() is ported from shell to C. This is done by introducing
three functions: module_status(), submodule_status() and print_status().

The function module_status() acts as the front-end of the subcommand.
It parses subcommand's options and then calls the function
module_list_compute() for computing the list of submodules. Then
this functions calls for_each_submodule_list() looping through the
list obtained.

Then for_each_submodule_list() calls submodule_status() for each of the
submodule in its list. The function submodule_status() is responsible
for generating the status each submodule it is called for, and
then calls print_status().

Finally, the function print_status() handles the printing of submodule's
status.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* parameters passed to the function print_status() have been changed.
  Instead of passing char *sub_sha1, instead the object_id is being passed.

* Also, since the passed parameter displaypath's value isn't changed
  by the function, it is passed to the funcition as const char *displaypath
  instead of char *displaypath.

* the output type of the function handle_submodule_head_ref() is changed
  from strbuf to object_id, as we will use the object_id instead of the
  hex of sha1 being stored in a struct strbuf.

* diff_files_Args is cleared after using it by passing it as args in the
  function cmd_diff_files.

* In the function status_submodule(), for checking if a submodule has merge
  conflicts, the patch currently checks if the value of any of the ce_flags
  is non-zero. Currently, I think the we aren't interested in a partiular flag,
  but I'm not sure on this.

* The confusion with displaypath being passed as te super-prefix in many
  of the ported subcommands may be a result of the fact that the
  function generating the displaypath: get_submodule_displaypath()
  uses the super-prefix as simply a path concatenated with the current
  submodule name to denote our current location.
  Also, for generating any submodule's displaypath, it would be important to
  have ".." passed to the submodule, and currently it is possible only via the
  super-prefix.

 builtin/submodule--helper.c | 157 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  49 +-------------
 2 files changed, 158 insertions(+), 48 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2cb72d68e..0bd969b7c 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -561,6 +561,162 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct status_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+	unsigned int cached: 1;
+};
+#define STATUS_CB_INIT { NULL, 0, 0, 0 }
+
+static void print_status(struct status_cb *info, char state, const char *path,
+			 const struct object_id *oid, const char *displaypath)
+{
+	if (info->quiet)
+		return;
+
+	printf("%c%s %s", state, oid_to_hex(oid), displaypath);
+
+
+	if (state == ' ' || state == '+') {
+		struct argv_array name_rev_args = ARGV_ARRAY_INIT;
+
+		argv_array_pushl(&name_rev_args, "print-name-rev",
+				 path, oid_to_hex(oid), NULL);
+		print_name_rev(name_rev_args.argc, name_rev_args.argv,
+			       info->prefix);
+	} else {
+		printf("\n");
+	}
+}
+
+static int handle_submodule_head_ref(const char *refname,
+				     const struct object_id *oid, int flags,
+				     void *cb_data)
+{
+	struct object_id *output = cb_data;
+	if (oid)
+		oidcpy(output, oid);
+
+	return 0;
+}
+
+static void status_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct status_cb *info = cb_data;
+	char *displaypath;
+	struct argv_array diff_files_args = ARGV_ARRAY_INIT;
+
+	if (!submodule_from_path(null_sha1, list_item->name))
+		die(_("no submodule mapping found in .gitmodules for path '%s'"),
+		      list_item->name);
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	trace_printf("the value of flag is %d\n", list_item->ce_flags);
+	if (list_item->ce_flags) {
+		trace_printf("for U the value of flag is %d\n", list_item->ce_flags);
+		print_status(info, 'U', list_item->name,
+			     &null_oid, displaypath);
+		goto cleanup;
+	}
+
+	if (!is_submodule_active(the_repository, list_item->name)) {
+		print_status(info, '-', list_item->name, &list_item->oid,
+			     displaypath);
+		goto cleanup;
+	}
+
+	argv_array_pushl(&diff_files_args, "diff-files",
+			 "--ignore-submodules=dirty", "--quiet", "--",
+			 list_item->name, NULL);
+
+	if (!cmd_diff_files(diff_files_args.argc, diff_files_args.argv,
+			    info->prefix)) {
+		print_status(info, ' ', list_item->name, &list_item->oid,
+			     displaypath);
+	} else {
+		if (!info->cached) {
+			struct object_id oid;
+
+			if (head_ref_submodule(list_item->name,
+					       handle_submodule_head_ref, &oid))
+				die(_("could not resolve HEAD ref inside the"
+				      "submodule '%s'"), list_item->name);
+
+			print_status(info, '+', list_item->name, &oid,
+				     displaypath);
+		} else {
+			print_status(info, '+', list_item->name,
+				     &list_item->oid, displaypath);
+		}
+	}
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "status", "--recursive",
+				 NULL);
+
+		if (info->cached)
+			argv_array_push(&cpr.args, "--cached");
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	argv_array_clear(&diff_files_args);
+	free(displaypath);
+}
+
+static int module_status(int argc, const char **argv, const char *prefix)
+{
+	struct status_cb info = STATUS_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int cached = 0;
+	int recursive = 0;
+
+	struct option module_status_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT_BOOL(0, "cached", &cached, N_("Use commit stored in the index instead of the one stored in the submodule HEAD")),
+		OPT_BOOL(0, "recursive", &recursive, N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule status [--quiet] [--cached] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_status_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+	info.cached = !!cached;
+
+	gitmodules_config();
+	for_each_submodule_list(list, status_submodule, &info);
+
+	return 0;
+}
+
 static int module_name(int argc, const char **argv, const char *prefix)
 {
 	const struct submodule *sub;
@@ -1307,6 +1463,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
+	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index e988167e0..51b057d82 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1005,54 +1005,7 @@ cmd_status()
 		shift
 	done
 
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		if test "$stage" = U
-		then
-			say "U$sha1 $displaypath"
-			continue
-		fi
-		if ! git submodule--helper is-active "$sm_path" ||
-		{
-			! test -d "$sm_path"/.git &&
-			! test -f "$sm_path"/.git
-		}
-		then
-			say "-$sha1 $displaypath"
-			continue;
-		fi
-		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
-		then
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say " $sha1 $displaypath$revname"
-		else
-			if test -z "$cached"
-			then
-				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
-			fi
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say "+$sha1 $displaypath$revname"
-		fi
-
-		if test -n "$recursive"
-		then
-			(
-				prefix="$displaypath/"
-				sanitize_submodule_env
-				wt_prefix=
-				cd "$sm_path" &&
-				eval cmd_status
-			) ||
-			die "$(eval_gettext "Failed to recurse into submodule path '\$sm_path'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} "$@"
 }
 #
 # Sync remote urls for submodules
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH v2 05/13] submodule: port submodule subcommand 'sync' from shell to C
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (3 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-07-29 22:23   ` " Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
                     ` (6 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

Port the submodule subcommand 'sync' from shell to C using the same
mechanism as that used for porting submodule subcommand 'status'.
Hence, here the function cmd_sync() is ported from shell to C.
This is done by introducing three functions: module_sync(),
sync_submodule() and print_default_remote().

The function print_default_remote() is introduced for getting
the default remote as stdout.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* There was no good reason for using puts in the function print_default_remote()
  Hence, in this patch, we instead use printf to do the same, as it is what
  is generally used throughout the codebase.

* As suggested, this patch ensures a more efficient use of variables, and
  removes most of the variables by reusing 'strbuf sb' at places required.

 builtin/submodule--helper.c | 182 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  56 +-------------
 2 files changed, 183 insertions(+), 55 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 0bd969b7c..877215567 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -44,6 +44,20 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static int print_default_remote(int argc, const char **argv, const char *prefix)
+{
+	const char *remote;
+
+	if (argc != 1)
+		die(_("submodule--helper print-default-remote takes no arguments"));
+
+	remote = get_default_remote();
+	if (remote)
+		printf("%s\n", remote);
+
+	return 0;
+}
+
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -380,6 +394,25 @@ static void module_list_active(struct module_list *list)
 	*list = active_modules;
 }
 
+static char *get_up_path(const char *path)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	for (i = count_slashes(path); i; i--)
+		strbuf_addstr(&sb, "../");
+
+	/*
+	 * Check if 'path' ends with slash or not
+	 * for having the same output for dir/sub_dir
+	 * and dir/sub_dir/
+	 */
+	if (!is_dir_sep(path[strlen(path) - 1]))
+		strbuf_addstr(&sb, "../");
+
+	return strbuf_detach(&sb, NULL);
+}
+
 static int module_list(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -736,6 +769,153 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct sync_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define SYNC_CB_INIT { NULL, 0, 0 }
+
+static void sync_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct sync_cb *info = cb_data;
+	const struct submodule *sub;
+	char *remote_key;
+	char *sub_origin_url, *super_config_url, *displaypath;
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *sub_config_path = NULL;
+
+	if (!is_submodule_active(the_repository, list_item->name))
+		return;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (sub && sub->url) {
+		if (starts_with_dot_dot_slash(sub->url) || starts_with_dot_slash(sub->url)) {
+			char *remote_url, *up_path;
+			char *remote = get_default_remote();
+			strbuf_addf(&sb, "remote.%s.url", remote);
+
+			if (git_config_get_string(sb.buf, &remote_url))
+				remote_url = xgetcwd();
+
+			up_path = get_up_path(list_item->name);
+			sub_origin_url = relative_url(remote_url, sub->url, up_path);
+			super_config_url = relative_url(remote_url, sub->url, NULL);
+
+			free(remote);
+			free(up_path);
+			free(remote_url);
+		} else {
+			sub_origin_url = xstrdup(sub->url);
+			super_config_url = xstrdup(sub->url);
+		}
+	} else {
+		sub_origin_url = "";
+		super_config_url = "";
+	}
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (!info->quiet)
+		printf(_("Synchronizing submodule url for '%s'\n"),
+			 displaypath);
+
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "submodule.%s.url", sub->name);
+	if (git_config_set_gently(sb.buf, super_config_url))
+		die(_("failed to register url for submodule path '%s'"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.dir = list_item->name;
+	argv_array_pushl(&cp.args, "submodule--helper",
+			 "print-default-remote", NULL);
+
+	strbuf_reset(&sb);
+	if (capture_command(&cp, &sb, 0))
+		die(_("failed to get the default remote for submodule '%s'"),
+		      list_item->name);
+
+	strbuf_strip_suffix(&sb, "\n");
+	remote_key = xstrfmt("remote.%s.url", sb.buf);
+
+	strbuf_reset(&sb);
+	submodule_to_gitdir(&sb, list_item->name);
+	strbuf_addstr(&sb, "/config");
+
+	if (git_config_set_in_file_gently(sb.buf, remote_key, sub_origin_url))
+		die(_("failed to update remote for submodule '%s'"),
+		      list_item->name);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "sync", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	strbuf_release(&sb);
+	free(super_config_url);
+	free(displaypath);
+	free(sub_config_path);
+	free(sub_origin_url);
+}
+
+static int module_sync(int argc, const char **argv, const char *prefix)
+{
+	struct sync_cb info = SYNC_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_sync_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of synchronizing submodule url")),
+		OPT_BOOL(0, "recursive", &recursive,
+			N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper sync [--quiet] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_sync_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, sync_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1464,6 +1644,8 @@ static struct cmd_struct commands[] = {
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
+	{"print-default-remote", print_default_remote, 0},
+	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 51b057d82..6bfc5e17d 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1037,63 +1037,9 @@ cmd_sync()
 			;;
 		esac
 	done
-	cd_to_toplevel
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-
-		# skip inactive submodules
-		if ! git submodule--helper is-active "$sm_path"
-		then
-			continue
-		fi
-
-		name=$(git submodule--helper name "$sm_path")
-		url=$(git config -f .gitmodules --get submodule."$name".url)
-
-		# Possibly a url relative to parent
-		case "$url" in
-		./*|../*)
-			# rewrite foo/bar as ../.. to find path from
-			# submodule work tree to superproject work tree
-			up_path="$(printf '%s\n' "$sm_path" | sed "s/[^/][^/]*/../g")" &&
-			# guarantee a trailing /
-			up_path=${up_path%/}/ &&
-			# path from submodule work tree to submodule origin repo
-			sub_origin_url=$(git submodule--helper resolve-relative-url "$url" "$up_path") &&
-			# path from superproject work tree to submodule origin repo
-			super_config_url=$(git submodule--helper resolve-relative-url "$url") || exit
-			;;
-		*)
-			sub_origin_url="$url"
-			super_config_url="$url"
-			;;
-		esac
 
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		say "$(eval_gettext "Synchronizing submodule url for '\$displaypath'")"
-		git config submodule."$name".url "$super_config_url"
-
-		if test -e "$sm_path"/.git
-		then
-		(
-			sanitize_submodule_env
-			cd "$sm_path"
-			remote=$(get_default_remote)
-			git config remote."$remote".url "$sub_origin_url"
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 
-			if test -n "$recursive"
-			then
-				prefix="$prefix$sm_path/"
-				eval cmd_sync
-			fi
-		)
-		fi
-	done
 }
 
 cmd_absorbgitdirs()
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH v2 06/13] submodule: port submodule subcommand 'deinit' from shell to C
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (4 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-07-29 22:23   ` " Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
                     ` (5 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

The same mechanism is used even for porting this submodule
subcommand, as used in the ported subcommands till now.
The function cmd_deinit in split up after porting into three
functions: module_deinit(), for_each_submodule_list() and
deinit_submodule().

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* In the function deinit_submodule, since the test is_git_directory()
  adds an additional condition, instead is_directory() is used to check
  if "sm_path/.git" is a directory.

* since it was possible in the previous path that the value st.st_mode passed
  to the function mkdir contained a garbage value, instead we intrduce a
  mode_t variable mode, initially containing a default mode value '0777'.
  This is what the default of mode is set in case, that the value is
  not set after the lstat call.

 builtin/submodule--helper.c | 144 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  55 +----------------
 2 files changed, 145 insertions(+), 54 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 877215567..038be7ee2 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -916,6 +916,149 @@ static int module_sync(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct deinit_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int force: 1;
+	unsigned int all: 1;
+};
+#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
+
+static void deinit_submodule(const struct cache_entry *list_item,
+			     void *cb_data)
+{
+	struct deinit_cb *info = cb_data;
+	const struct submodule *sub;
+	char *displaypath = NULL;
+	struct child_process cp_config = CHILD_PROCESS_INIT;
+	struct strbuf sb_config = STRBUF_INIT;
+	char *sub_git_dir = xstrfmt("%s/.git", list_item->name);
+	mode_t mode = 0777;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub || !sub->name)
+		goto cleanup;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	/* remove the submodule work tree (unless the user already did it) */
+	if (is_directory(list_item->name)) {
+		struct stat st;
+		/* protect submodules containing a .git directory */
+		if (is_directory(sub_git_dir))
+			die(_("Submodule work tree '%s' contains a .git "
+			      "directory use 'rm -rf' if you really want "
+			      "to remove it including all of its history"),
+			      displaypath);
+
+		if (!info->force) {
+			struct child_process cp_rm = CHILD_PROCESS_INIT;
+			cp_rm.git_cmd = 1;
+			argv_array_pushl(&cp_rm.args, "rm", "-qn",
+					 list_item->name, NULL);
+
+			if (run_command(&cp_rm))
+				die(_("Submodule work tree '%s' contains local "
+				      "modifications; use '-f' to discard them"),
+				      displaypath);
+		}
+
+		if (!lstat(list_item->name, &st)) {
+			struct strbuf sb_rm = STRBUF_INIT;
+			const char *format;
+
+			strbuf_addstr(&sb_rm, list_item->name);
+
+			if (!remove_dir_recursively(&sb_rm, 0))
+				format = _("Cleared directory '%s'\n");
+			else
+				format = _("Could not remove submodule work tree '%s'\n");
+
+			if (!info->quiet)
+				printf(format, displaypath);
+
+			mode = st.st_mode;
+
+			strbuf_release(&sb_rm);
+		}
+	}
+
+	if (mkdir(list_item->name, mode))
+		die(_("could not create empty submodule directory %s"),
+		      displaypath);
+
+	cp_config.git_cmd = 1;
+	argv_array_pushl(&cp_config.args, "config", "--get-regexp", NULL);
+	argv_array_pushf(&cp_config.args, "submodule.%s\\.", sub->name);
+
+	/* remove the .git/config entries (unless the user already did it) */
+	if (!capture_command(&cp_config, &sb_config, 0) && sb_config.len) {
+		char *sub_key = xstrfmt("submodule.%s", sub->name);
+		/*
+		 * remove the whole section so we have a clean state when
+		 * the user later decides to init this submodule again
+		 */
+		git_config_rename_section_in_file(NULL, sub_key, NULL);
+		if (!info->quiet)
+			printf(_("Submodule '%s' (%s) unregistered for path '%s'\n"),
+				 sub->name, sub->url, displaypath);
+		free(sub_key);
+	}
+
+cleanup:
+	free(displaypath);
+	free(sub_git_dir);
+	strbuf_release(&sb_config);
+}
+
+static int module_deinit(int argc, const char **argv, const char *prefix)
+{
+	struct deinit_cb info = DEINIT_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int force = 0;
+	int all = 0;
+
+	struct option module_deinit_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT__FORCE(&force, N_("Remove submodule working trees even if they contain local changes")),
+		OPT_BOOL(0, "all", &all, N_("Unregister all submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule deinit [--quiet] [-f | --force] [--all | [--] [<path>...]]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_deinit_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.all = !!all;
+	info.force = !!force;
+
+	if (all && argc) {
+		error("pathspec and --all are incompatible");
+		usage_with_options(git_submodule_helper_usage,
+				   module_deinit_options);
+	}
+
+	if (!argc && !all)
+		die(_("Use '--all' if you really want to deinitialize all submodules"));
+
+	gitmodules_config();
+	for_each_submodule_list(list, deinit_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1646,6 +1789,7 @@ static struct cmd_struct commands[] = {
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
+	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 6bfc5e17d..73e6f093f 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -428,60 +428,7 @@ cmd_deinit()
 		shift
 	done
 
-	if test -n "$deinit_all" && test "$#" -ne 0
-	then
-		echo >&2 "$(eval_gettext "pathspec and --all are incompatible")"
-		usage
-	fi
-	if test $# = 0 && test -z "$deinit_all"
-	then
-		die "$(eval_gettext "Use '--all' if you really want to deinitialize all submodules")"
-	fi
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-
-		displaypath=$(git submodule--helper relative-path "$sm_path" "$wt_prefix")
-
-		# Remove the submodule work tree (unless the user already did it)
-		if test -d "$sm_path"
-		then
-			# Protect submodules containing a .git directory
-			if test -d "$sm_path/.git"
-			then
-				die "$(eval_gettext "\
-Submodule work tree '\$displaypath' contains a .git directory
-(use 'rm -rf' if you really want to remove it including all of its history)")"
-			fi
-
-			if test -z "$force"
-			then
-				git rm -qn "$sm_path" ||
-				die "$(eval_gettext "Submodule work tree '\$displaypath' contains local modifications; use '-f' to discard them")"
-			fi
-			rm -rf "$sm_path" &&
-			say "$(eval_gettext "Cleared directory '\$displaypath'")" ||
-			say "$(eval_gettext "Could not remove submodule work tree '\$displaypath'")"
-		fi
-
-		mkdir "$sm_path" || say "$(eval_gettext "Could not create empty submodule directory '\$displaypath'")"
-
-		# Remove the .git/config entries (unless the user already did it)
-		if test -n "$(git config --get-regexp submodule."$name\.")"
-		then
-			# Remove the whole section so we have a clean state when
-			# the user later decides to init this submodule again
-			url=$(git config submodule."$name".url)
-			git config --remove-section submodule."$name" 2>/dev/null &&
-			say "$(eval_gettext "Submodule '\$name' (\$url) unregistered for path '\$displaypath'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${force:+--force} ${deinit_all:+--all} "$@"
 }
 
 is_tip_reachable () (
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH v2 08/13] submodule: port submodule subcommand 'summary' from shell to C
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (5 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
@ 2017-07-29 22:23   ` " Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
                     ` (4 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

The submodule subcommand 'summary' is ported in the process of
making git-submodule a builtin. The function cmd_summary() from
git-submodule.sh is ported to functions module_summary(),
compute_summary_module_list(), prepare_submodule_summary() and
print_submodule_summary().

The first function module_summary() parses the options of submodule
subcommand and also acts as the front-end of this subcommand.
After parsing them, it calls the compute_summary_module_list()

The functions compute_summary_module_list() runs the diff_cmd,
and generates the modules list, as required by the subcommand.
The generation of this module list is done by the using the
callback function submodule_summary_callback(), and stored in the
structure module_cb.

Once the module list is generated, prepare_submodule_summary()
further goes through the list and filters the list, for
eventually calling the print_submodule_summary() function.

Finally, the print_submodule_summary() takes care of generating
and printing the summary for each submodule.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:

* Firstly, about the function compute_summary_module_list().
  This function is created to generate the list of modules, for which
  we will generate the summary further. Since the list is actually
  generated using the git-diff-files or git-diff-index command, but for
  porting this, we required to create a function similar to the builtin
  functions of the above commands. But we can't directly call cmd_diff_files()
  and cmd_diff_index() since we don't have to display the output and instead
  need to store it. Hence, this function is introduced.

* Also, the module_cb_list *list is not freed since it is a non-heap object.
  Hence, free() can't be using on the non-heap objects.

* In the function prepare_submodule_summary(), as suggested
  'git_config_get_string_const' was used instead of instead of '_value'

* Some variables which weren't modified throughout the function-call were
  passed as const.

* The '!!' trick, which wasn't used in the last patch, is now used in this
  new version .

* the variables sha1_dst and sha1_src are removed from the function
  print_submodule_summary(), and instead the p->oid_src and p->oid_dst are
  used.

* The variable sm_git_dir is freed at the end of the function.

 builtin/submodule--helper.c | 433 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 182 +------------------
 2 files changed, 434 insertions(+), 181 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 038be7ee2..d8bf16f1d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -13,6 +13,9 @@
 #include "remote.h"
 #include "refs.h"
 #include "connect.h"
+#include "revision.h"
+#include "diffcore.h"
+#include "diff.h"
 
 typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
 				      void *cb_data);
@@ -769,6 +772,435 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct module_cb {
+	unsigned int mod_src;
+	unsigned int mod_dst;
+	struct object_id oid_src;
+	struct object_id oid_dst;
+	char status;
+	const char *sm_path;
+};
+#define MODULE_CB_INIT { 0, 0, NULL, NULL, '\0', NULL }
+
+struct module_cb_list {
+	struct module_cb **entries;
+	int alloc, nr;
+};
+#define MODULE_CB_LIST_INIT { NULL, 0, 0 }
+
+struct summary_cb {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	char *diff_cmd;
+	unsigned int cached: 1;
+	unsigned int for_status: 1;
+	unsigned int quiet: 1;
+	unsigned int files: 1;
+	int summary_limits;
+};
+#define SUMMARY_CB_INIT { 0, NULL, NULL, NULL, 0, 0, 0, 0, 0 }
+
+static int verify_submodule_object_name(const char *sm_path, const char *sha1)
+{
+	struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+	cp_rev_parse.git_cmd = 1;
+	cp_rev_parse.no_stdout = 1;
+	cp_rev_parse.dir = sm_path;
+	prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+	argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
+			 "--verify", NULL);
+	argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1);
+
+	if (run_command(&cp_rev_parse))
+		return 1;
+
+	return 0;
+}
+
+static void print_submodule_summary(struct summary_cb *info,
+				    struct module_cb *p)
+{
+	int missing_src = 0;
+	int missing_dst = 0;
+	char *displaypath;
+	const char *sha1_abbr_src;
+	const char *sha1_abbr_dst;
+	int errmsg = 0;
+	int total_commits = -1;
+	char *sm_git_dir = xstrfmt("%s/.git", p->sm_path);
+	int is_sm_git_dir = 0;
+
+	if (!info->cached && !oidcmp(&p->oid_dst, &null_oid)) {
+		if (S_ISGITLINK(p->mod_dst)) {
+			struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_parse = STRBUF_INIT;
+
+			cp_rev_parse.git_cmd = 1;
+			cp_rev_parse.no_stderr = 1;
+			cp_rev_parse.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+			argv_array_pushl(&cp_rev_parse.args,
+					 "rev-parse", "HEAD", NULL);
+			if (!capture_command(&cp_rev_parse, &sb_rev_parse, 0)) {
+				strbuf_strip_suffix(&sb_rev_parse, "\n");
+
+				get_oid_hex(sb_rev_parse.buf, &p->oid_dst);
+			}
+			strbuf_release(&sb_rev_parse);
+		} else if (S_ISLNK(p->mod_dst) || S_ISREG(p->mod_dst)) {
+			struct child_process cp_hash_object = CHILD_PROCESS_INIT;
+			struct strbuf sb_hash_object = STRBUF_INIT;
+
+			cp_hash_object.git_cmd = 1;
+			argv_array_pushl(&cp_hash_object.args,
+					 "hash-object", p->sm_path,
+					 NULL);
+			if (!capture_command(&cp_hash_object,
+					     &sb_hash_object, 0)) {
+				strbuf_strip_suffix(&sb_hash_object, "\n");
+
+				get_oid_hex(sb_hash_object.buf, &p->oid_dst);
+			}
+			strbuf_release(&sb_hash_object);
+		} else {
+			if (p->mod_dst)
+				die(_("unexpected mode %d\n"), p->mod_dst);
+		}
+	}
+
+	if (is_git_directory(sm_git_dir))
+		is_sm_git_dir = 1;
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_src))
+		missing_src = verify_submodule_object_name(p->sm_path,
+							   oid_to_hex(&p->oid_src));
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_dst))
+		missing_dst = verify_submodule_object_name(p->sm_path,
+							   oid_to_hex(&p->oid_dst));
+
+	displaypath = get_submodule_displaypath(p->sm_path, info->prefix);
+
+	if (!missing_dst && !missing_src) {
+		if (is_sm_git_dir) {
+			struct child_process cp_rev_list = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_list = STRBUF_INIT;
+			char *range;
+
+			if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst))
+				range = xstrfmt("%s...%s", oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+			else if (S_ISGITLINK(p->mod_src))
+				range = xstrdup(oid_to_hex(&p->oid_src));
+			else
+				range = xstrdup(oid_to_hex(&p->oid_dst));
+
+			cp_rev_list.git_cmd = 1;
+			cp_rev_list.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_list.env_array);
+
+			argv_array_pushl(&cp_rev_list.args, "rev-list",
+					 "--first-parent", range, "--", NULL);
+			if (!capture_command(&cp_rev_list, &sb_rev_list, 0)) {
+				if (sb_rev_list.len)
+					total_commits = count_lines(sb_rev_list.buf,
+								    sb_rev_list.len);
+				else
+					total_commits = 0;
+			}
+
+			free(range);
+			strbuf_release(&sb_rev_list);
+		}
+	} else {
+		errmsg = 1;
+	}
+
+	sha1_abbr_src = find_unique_abbrev(p->oid_src.hash, 7);
+	sha1_abbr_dst = find_unique_abbrev(p->oid_dst.hash, 7);
+
+	if (p->status == 'T') {
+		if (S_ISGITLINK(p->mod_dst))
+			printf(_("* %s %s(blob)->%s(submodule)"),
+				 displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+		else
+			printf(_("* %s %s(submodule)->%s(blob)"),
+				 displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+	} else {
+			printf("* %s %s...%s", displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+	}
+
+	if (total_commits < 0)
+		printf(":\n");
+	else
+		printf(" (%d):\n", total_commits);
+
+	if (errmsg) {
+		/*
+		 * Don't give error msg for modification whose dst is not
+		 * submodule, i.e. deleted or changed to blob
+		 */
+		if (S_ISGITLINK(p->mod_src)) {
+			if (missing_src && missing_dst) {
+				printf(_("  Warn: %s doesn't contain commits %s and %s\n"),
+				 displaypath, oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+			} else if (missing_src) {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, oid_to_hex(&p->oid_src));
+			} else {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, oid_to_hex(&p->oid_dst));
+			}
+		}
+	} else if (is_sm_git_dir) {
+		struct child_process cp_log = CHILD_PROCESS_INIT;
+
+		cp_log.git_cmd = 1;
+		cp_log.dir = p->sm_path;
+		prepare_submodule_repo_env(&cp_log.env_array);
+		argv_array_pushl(&cp_log.args, "log", NULL);
+
+		if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst)) {
+			if (info->summary_limits > 0)
+				argv_array_pushf(&cp_log.args, "-%d", info->summary_limits);
+
+			argv_array_pushl(&cp_log.args, "--pretty=  %m %s",
+					 "--first-parent", NULL);
+			argv_array_pushf(&cp_log.args, "%s...%s", oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+		} else if (S_ISGITLINK(p->mod_dst)) {
+			argv_array_pushl(&cp_log.args, "--pretty=  > %s",
+					 "-1", oid_to_hex(&p->oid_dst), NULL);
+		} else {
+			argv_array_pushl(&cp_log.args, "--pretty=  < %s",
+					 "-1", oid_to_hex(&p->oid_src), NULL);
+		}
+
+		run_command(&cp_log);
+	}
+	printf("\n");
+
+	free(displaypath);
+	free(sm_git_dir);
+}
+
+static void prepare_submodule_summary(struct summary_cb *info,
+				      struct module_cb_list *list)
+{
+	int i;
+	for (i = 0; i < list->nr; i++) {
+		struct module_cb *p = list->entries[i];
+		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+		if (p->status == 'D' || p->status == 'T') {
+			print_submodule_summary(info, p);
+			continue;
+		}
+
+		if (info->for_status) {
+			char *config_key;
+			const char *ignore_config = "none";
+			const char *value;
+			const struct submodule *sub = submodule_from_path(null_sha1, p->sm_path);
+
+			if (sub && p->status != 'A') {
+				config_key = xstrfmt("submodule.%s.ignore",
+						     sub->name);
+				if (!git_config_get_string_const(config_key, &value))
+					ignore_config = value;
+				else if (sub->ignore)
+					ignore_config = sub->ignore;
+
+				free(config_key);
+
+				if (!strcmp(ignore_config, "all"))
+					continue;
+			}
+		}
+
+		/* Also show added or modified modules which are checked out */
+		cp_rev_parse.dir = p->sm_path;
+		cp_rev_parse.git_cmd = 1;
+		cp_rev_parse.no_stderr = 1;
+		cp_rev_parse.no_stdout = 1;
+
+		argv_array_pushl(&cp_rev_parse.args, "rev-parse",
+				 "--git-dir", NULL);
+
+		if (!run_command(&cp_rev_parse))
+			print_submodule_summary(info, p);
+	}
+}
+
+static void submodule_summary_callback(struct diff_queue_struct *q,
+				       struct diff_options *options,
+				       void *data)
+{
+	int i;
+	struct module_cb_list *list = data;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		struct module_cb *temp;
+
+		if (!S_ISGITLINK(p->one->mode) && !S_ISGITLINK(p->two->mode))
+			continue;
+		temp = (struct module_cb*)malloc(sizeof(struct module_cb));
+		temp->mod_src = p->one->mode;
+		temp->mod_dst = p->two->mode;
+		temp->oid_src = p->one->oid;
+		temp->oid_dst = p->two->oid;
+		temp->status = p->status;
+		temp->sm_path = xstrdup(p->one->path);
+
+		ALLOC_GROW(list->entries, list->nr + 1, list->alloc);
+		list->entries[list->nr++] = temp;
+	}
+}
+
+static int compute_summary_module_list(const char *head, struct summary_cb *info)
+{
+	struct argv_array diff_args = ARGV_ARRAY_INIT;
+	struct rev_info rev;
+	struct module_cb_list list = MODULE_CB_LIST_INIT;
+
+	argv_array_push(&diff_args, info->diff_cmd);
+	if (info->cached)
+		argv_array_push(&diff_args, "--cached");
+	argv_array_pushl(&diff_args, "--ignore-submodules=dirty", "--raw",
+			 NULL);
+	if (head)
+		argv_array_push(&diff_args, head);
+	argv_array_push(&diff_args, "--");
+	if (info->argc)
+		argv_array_pushv(&diff_args, info->argv);
+
+	git_config(git_diff_basic_config, NULL);
+	init_revisions(&rev, info->prefix);
+	gitmodules_config();
+	rev.abbrev = 0;
+	precompose_argv(diff_args.argc, diff_args.argv);
+
+	diff_args.argc = setup_revisions(diff_args.argc, diff_args.argv,
+					 &rev, NULL);
+	rev.diffopt.output_format = DIFF_FORMAT_NO_OUTPUT | DIFF_FORMAT_CALLBACK;
+	rev.diffopt.format_callback = submodule_summary_callback;
+	rev.diffopt.format_callback_data = &list;
+
+	if (!info->cached) {
+		if (!strcmp(info->diff_cmd, "diff-index"))
+			setup_work_tree();
+		if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
+			perror("read_cache_preload");
+			return -1;
+		}
+	} else if (read_cache() < 0) {
+		perror("read_cache");
+		return -1;
+	}
+
+	if (!strcmp(info->diff_cmd, "diff-index"))
+		run_diff_index(&rev, info->cached);
+	else
+		run_diff_files(&rev, 0);
+	prepare_submodule_summary(info, &list);
+
+	return 0;
+
+}
+
+static int module_summary(int argc, const char **argv, const char *prefix)
+{
+	struct summary_cb info = SUMMARY_CB_INIT;
+	int cached = 0;
+	char *diff_cmd = "diff-index";
+	int for_status = 0;
+	int quiet = 0;
+	int files = 0;
+	int summary_limits = -1;
+	struct child_process cp_rev = CHILD_PROCESS_INIT;
+	char *head;
+	struct strbuf sb = STRBUF_INIT;
+	int ret;
+
+	struct option module_summary_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
+		OPT_BOOL(0, "cached", &cached, N_("Use the commit stored in the index instead of the submodule HEAD")),
+		OPT_BOOL(0, "files", &files, N_("To compares the commit in the index with that in the submodule HEAD")),
+		OPT_BOOL(0, "for-status", &for_status, N_("Skip submodules with 'all' ignore_config value")),
+		OPT_INTEGER('n', "summary-limits", &summary_limits, N_("Limit the summary size")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper summary [<options>] [--] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_summary_options,
+			     git_submodule_helper_usage, 0);
+
+	if (!summary_limits)
+		return 0;
+
+	cp_rev.git_cmd = 1;
+	argv_array_pushl(&cp_rev.args, "rev-parse", "-q", "--verify",
+			 argc ? argv[0] : "HEAD", NULL);
+
+	if (!capture_command(&cp_rev, &sb, 0)) {
+		strbuf_strip_suffix(&sb, "\n");
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else if (!argc || !strcmp(argv[0], "HEAD")) {
+		/* before the first commit: compare with an empty tree */
+		struct stat st;
+		struct object_id oid;
+		if (fstat(0, &st) < 0 || index_fd(oid.hash, 0, &st, 2, prefix, 3))
+			die("Unable to add %s to database", oid.hash);
+		strbuf_addstr(&sb, oid_to_hex(&oid));
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else {
+		strbuf_addstr(&sb, "HEAD");
+	}
+
+	head = strbuf_detach(&sb, NULL);
+
+	if (files) {
+		if (cached)
+			die(_("The --cached option cannot be used with the --files option"));
+		diff_cmd = "diff-files";
+
+		free(head);
+
+		head = NULL;
+	}
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.cached = !!cached;
+	info.for_status = !!for_status;
+	info.quiet = quiet;
+	info.files = files;
+	info.summary_limits = summary_limits;
+	info.diff_cmd = diff_cmd;
+
+	ret = compute_summary_module_list(head, &info);
+	if (head)
+		free(head);
+	return ret;
+
+}
+
 struct sync_cb {
 	const char *prefix;
 	unsigned int quiet: 1;
@@ -1790,6 +2222,7 @@ static struct cmd_struct commands[] = {
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
+	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 73e6f093f..a427ddafd 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -51,31 +51,6 @@ die_if_unmatched ()
 	fi
 }
 
-#
-# Print a submodule configuration setting
-#
-# $1 = submodule name
-# $2 = option name
-# $3 = default value
-#
-# Checks in the usual git-config places first (for overrides),
-# otherwise it falls back on .gitmodules.  This allows you to
-# distribute project-wide defaults in .gitmodules, while still
-# customizing individual repositories if necessary.  If the option is
-# not in .gitmodules either, print a default value.
-#
-get_submodule_config () {
-	name="$1"
-	option="$2"
-	default="$3"
-	value=$(git config submodule."$name"."$option")
-	if test -z "$value"
-	then
-		value=$(git config -f .gitmodules submodule."$name"."$option")
-	fi
-	printf '%s' "${value:-$default}"
-}
-
 isnumber()
 {
 	n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
@@ -755,163 +730,8 @@ cmd_summary() {
 		shift
 	done
 
-	test $summary_limit = 0 && return
-
-	if rev=$(git rev-parse -q --verify --default HEAD ${1+"$1"})
-	then
-		head=$rev
-		test $# = 0 || shift
-	elif test -z "$1" || test "$1" = "HEAD"
-	then
-		# before the first commit: compare with an empty tree
-		head=$(git hash-object -w -t tree --stdin </dev/null)
-		test -z "$1" || shift
-	else
-		head="HEAD"
-	fi
-
-	if [ -n "$files" ]
-	then
-		test -n "$cached" &&
-		die "$(gettext "The --cached option cannot be used with the --files option")"
-		diff_cmd=diff-files
-		head=
-	fi
-
-	cd_to_toplevel
-	eval "set $(git rev-parse --sq --prefix "$wt_prefix" -- "$@")"
-	# Get modified modules cared by user
-	modules=$(git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- "$@" |
-		sane_egrep '^:([0-7]* )?160000' |
-		while read -r mod_src mod_dst sha1_src sha1_dst status sm_path
-		do
-			# Always show modules deleted or type-changed (blob<->module)
-			if test "$status" = D || test "$status" = T
-			then
-				printf '%s\n' "$sm_path"
-				continue
-			fi
-			# Respect the ignore setting for --for-status.
-			if test -n "$for_status"
-			then
-				name=$(git submodule--helper name "$sm_path")
-				ignore_config=$(get_submodule_config "$name" ignore none)
-				test $status != A && test $ignore_config = all && continue
-			fi
-			# Also show added or modified modules which are checked out
-			GIT_DIR="$sm_path/.git" git-rev-parse --git-dir >/dev/null 2>&1 &&
-			printf '%s\n' "$sm_path"
-		done
-	)
-
-	test -z "$modules" && return
-
-	git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- $modules |
-	sane_egrep '^:([0-7]* )?160000' |
-	cut -c2- |
-	while read -r mod_src mod_dst sha1_src sha1_dst status name
-	do
-		if test -z "$cached" &&
-			test $sha1_dst = 0000000000000000000000000000000000000000
-		then
-			case "$mod_dst" in
-			160000)
-				sha1_dst=$(GIT_DIR="$name/.git" git rev-parse HEAD)
-				;;
-			100644 | 100755 | 120000)
-				sha1_dst=$(git hash-object $name)
-				;;
-			000000)
-				;; # removed
-			*)
-				# unexpected type
-				eval_gettextln "unexpected mode \$mod_dst" >&2
-				continue ;;
-			esac
-		fi
-		missing_src=
-		missing_dst=
-
-		test $mod_src = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_src^0 >/dev/null &&
-		missing_src=t
-
-		test $mod_dst = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_dst^0 >/dev/null &&
-		missing_dst=t
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${for_status:+--for-status} ${files:+--files} ${cached:+--cached} ${summary_limit:+-n $summary_limit} "$@"
 
-		display_name=$(git submodule--helper relative-path "$name" "$wt_prefix")
-
-		total_commits=
-		case "$missing_src,$missing_dst" in
-		t,)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_src")"
-			;;
-		,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_dst")"
-			;;
-		t,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commits \$sha1_src and \$sha1_dst")"
-			;;
-		*)
-			errmsg=
-			total_commits=$(
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				range="$sha1_src...$sha1_dst"
-			elif test $mod_src = 160000
-			then
-				range=$sha1_src
-			else
-				range=$sha1_dst
-			fi
-			GIT_DIR="$name/.git" \
-			git rev-list --first-parent $range -- | wc -l
-			)
-			total_commits=" ($(($total_commits + 0)))"
-			;;
-		esac
-
-		sha1_abbr_src=$(echo $sha1_src | cut -c1-7)
-		sha1_abbr_dst=$(echo $sha1_dst | cut -c1-7)
-		if test $status = T
-		then
-			blob="$(gettext "blob")"
-			submodule="$(gettext "submodule")"
-			if test $mod_dst = 160000
-			then
-				echo "* $display_name $sha1_abbr_src($blob)->$sha1_abbr_dst($submodule)$total_commits:"
-			else
-				echo "* $display_name $sha1_abbr_src($submodule)->$sha1_abbr_dst($blob)$total_commits:"
-			fi
-		else
-			echo "* $display_name $sha1_abbr_src...$sha1_abbr_dst$total_commits:"
-		fi
-		if test -n "$errmsg"
-		then
-			# Don't give error msg for modification whose dst is not submodule
-			# i.e. deleted or changed to blob
-			test $mod_dst = 160000 && echo "$errmsg"
-		else
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				limit=
-				test $summary_limit -gt 0 && limit="-$summary_limit"
-				GIT_DIR="$name/.git" \
-				git log $limit --pretty='format:  %m %s' \
-				--first-parent $sha1_src...$sha1_dst
-			elif test $mod_dst = 160000
-			then
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  > %s' -1 $sha1_dst
-			else
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  < %s' -1 $sha1_src
-			fi
-			echo
-		fi
-		echo
-	done
 }
 #
 # List all submodules, prefixed with:
-- 
2.13.0


^ permalink raw reply	[relevance 15%]

* [GSoC][PATCH v2 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (6 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
@ 2017-07-29 22:23   ` Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
                     ` (3 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

When running 'git submodule foreach' from a subdirectory of your
repository, nested submodules get a bogus value for $sm_path:
For a submodule 'sub' that contains a nested submodule 'nested',
running 'git -C dir submodule foreach echo $path' would report
path='../nested' for the nested submodule. The first part '../' is
derived from the logic computing the relative path from $pwd to the
root of the superproject. The second part is the submodule path inside
the submodule. This value is of little use and is hard to document.

There are two different possible solutions that have more value:
(a) The path value is documented as the path from the toplevel of the
    superproject to the mount point of the submodule.
    In this case we would want to have path='sub/nested'.

(b) As Ramsay noticed the documented value is wrong. For the non-nested
    case the path is equal to the relative path from $pwd to the
    submodules working directory. When following this model,
    the expected value would be path='../sub/nested'.

The behavior for (b) was introduced in 091a6eb0fe (submodule: drop the
top-level requirement, 2013-06-16) the intent for $path seemed to be
relative to $cwd to the submodule worktree, but that did not work for
nested submodules, as the intermittent submodules were not included in
the path.

If we were to fix the meaning of the $path using (a) such that "path"
is "the path from the toplevel of the superproject to the mount point
of the submodule", we would break any existing submodule user that runs
foreach from non-root of the superproject as the non-nested submodule
'../sub' would change its path to 'sub'.

If we would fix the meaning of the $path using (b), such that "path"
is "the relative path from $pwd to the submodule", then we would break
any user that uses nested submodules (even from the root directory) as
the 'nested' would become 'sub/nested'.

Both groups can be found in the wild.  The author has no data if one group
outweighs the other by large margin, and offending each one seems equally
bad at first.  However in the authors imagination it is better to go with
(a) as running from a sub directory sounds like it is carried out
by a human rather than by some automation task.  With a human on
the keyboard the feedback loop is short and the changed behavior can be
adapted to quickly unlike some automation that can break silently.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh             |  1 -
 t/t7407-submodule-foreach.sh | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index a427ddafd..493a64372 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -320,7 +320,6 @@ cmd_foreach()
 				prefix="$prefix$sm_path/"
 				sanitize_submodule_env
 				cd "$sm_path" &&
-				sm_path=$(git submodule--helper relative-path "$sm_path" "$wt_prefix") &&
 				# we make $path available to scripts ...
 				path=$sm_path &&
 				if test $# -eq 1
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 6ba5daf42..0663622a4 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,9 +82,9 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-../sub1-$sub1sha1
+$pwd/clone-foo1-sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-../sub3-$sub3sha1
+$pwd/clone-foo3-sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
@@ -196,6 +196,38 @@ test_expect_success 'test messages from "foreach --recursive" from subdirectory'
 	) &&
 	test_i18ncmp expect actual
 '
+sub1sha1=$(cd clone2/sub1 && git rev-parse HEAD)
+sub2sha1=$(cd clone2/sub2 && git rev-parse HEAD)
+sub3sha1=$(cd clone2/sub3 && git rev-parse HEAD)
+nested1sha1=$(cd clone2/nested1 && git rev-parse HEAD)
+nested2sha1=$(cd clone2/nested1/nested2 && git rev-parse HEAD)
+nested3sha1=$(cd clone2/nested1/nested2/nested3 && git rev-parse HEAD)
+submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEAD)
+
+cat >expect <<EOF
+Entering '../nested1'
+$pwd/clone2-nested1-nested1-$nested1sha1
+Entering '../nested1/nested2'
+$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+Entering '../nested1/nested2/nested3'
+$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+Entering '../nested1/nested2/nested3/submodule'
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+Entering '../sub1'
+$pwd/clone2-foo1-sub1-$sub1sha1
+Entering '../sub2'
+$pwd/clone2-foo2-sub2-$sub2sha1
+Entering '../sub3'
+$pwd/clone2-foo3-sub3-$sub3sha1
+EOF
+
+test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
+	(
+		cd clone2/untracked &&
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+	) &&
+	test_i18ncmp expect actual
+'
 
 cat > expect <<EOF
 nested1-nested1
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH v2 10/13] submodule foreach: document '$sm_path' instead of '$path'
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (7 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
@ 2017-07-29 22:23   ` Prathamesh Chavan
  2017-07-29 22:23   ` [GSoC][PATCH v2 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
                     ` (2 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

As using a variable '$path' may be harmful to users due to
capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
use $path variable in eval_gettext string, 2012-04-17). Adjust
the documentation to advocate for using $sm_path,  which contains
the same value. We still make the 'path' variable available and
document it as a deprecated synonym of 'sm_path'.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
This patch is same as its previous version.
Although here I'll like to add a point that we aim to slowly drop the support
of the variable 'path'.

 Documentation/git-submodule.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index ff612001d..a23baef62 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,12 +183,14 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $path, $sha1 and
+	The command has access to the variables $name, $sm_path, $sha1 and
 	$toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
-	$path is the name of the submodule directory relative to the
-	superproject, $sha1 is the commit as recorded in the superproject,
-	and $toplevel is the absolute path to the top-level of the superproject.
+	$sm_path is the path of the submodule as recorded in the superproject,
+	$sha1 is the commit as recorded in the superproject, and
+	$toplevel is the absolute path to the top-level of the superproject.
+	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
+	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given `--quiet`, foreach prints the name
 	of each submodule before evaluating the command.
-- 
2.13.0


^ permalink raw reply	[relevance 23%]

* [GSoC][PATCH v2 11/13] submodule foreach: clarify the '$toplevel' variable documentation
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (8 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
@ 2017-07-29 22:23   ` Prathamesh Chavan
  2017-07-29 22:24   ` [GSoC][PATCH v2 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
  2017-07-29 22:24   ` [GSoC][PATCH v2 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:23 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

It does not contain the topmost superproject as the author assumed,
but the direct superproject, such that $toplevel/$sm_path is the
actual absolute path of the submodule.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index a23baef62..8e7930ebc 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -188,7 +188,8 @@ foreach [--recursive] <command>::
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
 	$sha1 is the commit as recorded in the superproject, and
-	$toplevel is the absolute path to the top-level of the superproject.
+	$toplevel is the absolute path to its superproject, such that
+	$toplevel/$sm_path is the absolute path of the submodule.
 	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
 	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
-- 
2.13.0


^ permalink raw reply	[relevance 24%]

* [GSoC][PATCH v2 12/13] submodule foreach: document variable '$displaypath'
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (9 preceding siblings ...)
  2017-07-29 22:23   ` [GSoC][PATCH v2 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
@ 2017-07-29 22:24   ` Prathamesh Chavan
  2017-07-29 22:24   ` [GSoC][PATCH v2 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:24 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

It was observed that the variable '$displaypath' was accessible but
undocumented. Hence, document it.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:

* Spelling mistake in the commit message was corrected.

 Documentation/git-submodule.txt |  6 ++++--
 t/t7407-submodule-foreach.sh    | 22 +++++++++++-----------
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 8e7930ebc..0cca702cb 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,10 +183,12 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $sm_path, $sha1 and
-	$toplevel:
+	The command has access to the variables $name, $sm_path, $displaypath,
+	$sha1 and $toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
+	$displaypath contains the relative path from the current working
+	directory to the submodules root directory,
 	$sha1 is the commit as recorded in the superproject, and
 	$toplevel is the absolute path to its superproject, such that
 	$toplevel/$sm_path is the absolute path of the submodule.
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 0663622a4..6ad57e061 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,16 +82,16 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-sub1-$sub1sha1
+$pwd/clone-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-sub3-$sub3sha1
+$pwd/clone-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
 	mkdir clone/sub &&
 	(
 		cd clone/sub &&
-		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
@@ -206,25 +206,25 @@ submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEA
 
 cat >expect <<EOF
 Entering '../nested1'
-$pwd/clone2-nested1-nested1-$nested1sha1
+$pwd/clone2-nested1-nested1-../nested1-$nested1sha1
 Entering '../nested1/nested2'
-$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+$pwd/clone2/nested1-nested2-nested2-../nested1/nested2-$nested2sha1
 Entering '../nested1/nested2/nested3'
-$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+$pwd/clone2/nested1/nested2-nested3-nested3-../nested1/nested2/nested3-$nested3sha1
 Entering '../nested1/nested2/nested3/submodule'
-$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-../nested1/nested2/nested3/submodule-$submodulesha1
 Entering '../sub1'
-$pwd/clone2-foo1-sub1-$sub1sha1
+$pwd/clone2-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub2'
-$pwd/clone2-foo2-sub2-$sub2sha1
+$pwd/clone2-foo2-sub2-../sub2-$sub2sha1
 Entering '../sub3'
-$pwd/clone2-foo3-sub3-$sub3sha1
+$pwd/clone2-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
 	(
 		cd clone2/untracked &&
-		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH v2 13/13] submodule: port submodule subcommand 'foreach' from shell to C
      [irrelevant] ` <20170729222401.12381-1-pc44800@gmail.com>
                     ` (10 preceding siblings ...)
  2017-07-29 22:24   ` [GSoC][PATCH v2 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
@ 2017-07-29 22:24   ` Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-29 22:24 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, bmwill, Prathamesh Chavan

This aims to make git-submodule foreach a builtin. This is the very
first step taken in this direction. Hence, 'foreach' is ported to
submodule--helper, and submodule--helper is called from git-submodule.sh.
The code is split up to have one function to obtain all the list of
submodules. This function acts as the front-end of git-submodule foreach
subcommand. It calls the function for_each_submodule_list, which basically
loops through the list and calls function fn, which in this case is
runcommand_in_submodule. This third function is a calling function that
takes care of running the command in that submodule, and recursively
perform the same when --recursive is flagged.

The first function module_foreach first parses the options present in
argv, and then with the help of module_list_compute, generates the list of
submodules present in the current working tree.

The second function for_each_submodule_list traverses through the
list, and calls function fn (which in case of submodule subcommand
foreach is runcommand_in_submodule) is called for each entry.

The third function runcommand_in_submodule, generates a submodule struct sub
for $name, value and then later prepends name=sub->name; and other
value assignment to the env argv_array structure of a child_process.
Also the <command> of submodule-foreach is push to args argv_array
structure and finally, using run_command the commands are executed
using a shell.

The third function also takes care of the recursive flag, by creating
a separate child_process structure and prepending "--super-prefix displaypath",
to the args argv_array structure. Other required arguments and the
input <command> of submodule-foreach is also appended to this argv_array.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:

* Comment style is improved in the function runcommand_in_submodule()

* Comment in added about why the variable "path" was exposed via args
  argv_array instead of exposing it via the env_array.

* This patch exposes the various variables when argc == 1 only, just
  for maintaining a faithful porting. You can also find discussion about
  the same at [1].

[1]: https://public-inbox.org/git/CAME+mvUSGAFbN5j-_hv7QpAS57hq4wgH+yZ7XJMPuyQN1gALaA@mail.gmail.com/

 builtin/submodule--helper.c | 136 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  39 +------------
 2 files changed, 137 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d8bf16f1d..d5527aa93 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -772,6 +772,141 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct cb_foreach {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define CB_FOREACH_INIT { 0, NULL, NULL, 0, 0 }
+
+static void runcommand_in_submodule(const struct cache_entry *list_item,
+				    void *cb_data)
+{
+	struct cb_foreach *info = cb_data;
+	const struct submodule *sub;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *displaypath;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub)
+		die(_("No url found for submodule path '%s' in .gitmodules"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+
+	/*
+	 * For the purpose of executing <command> in the submodule,
+	 * separate shell is used for the purpose of running the
+	 * child process.
+	 */
+	cp.use_shell = 1;
+	cp.dir = list_item->name;
+
+	if (info->argc == 1) {
+		char *toplevel = xgetcwd();
+
+		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
+		argv_array_pushf(&cp.env_array, "sm_path=%s", list_item->name);
+		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
+		argv_array_pushf(&cp.env_array, "sha1=%s",
+				 oid_to_hex(&list_item->oid));
+		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
+
+		/*
+		 * Since the path variable was accessible from the script
+		 * before porting, it is also made available after porting.
+		 * The environment variable "PATH" has a very special purpose
+		 * on windows. And since environment variables are
+		 * case-insensitive in windows, it interferes with the
+		 * existing PATH variable. Hence, to avoid that, we expose
+		 * path via the args argv_array and not via env_array.
+		 */
+		argv_array_pushf(&cp.args, "path=%s; %s",
+				 list_item->name, info->argv[0]);
+		free(toplevel);
+	} else {
+		argv_array_pushv(&cp.args, info->argv);
+	}
+
+	if (!info->quiet)
+		printf(_("Entering '%s'\n"), displaypath);
+
+	if (info->argv[0] && run_command(&cp))
+		die(_("run_command returned non-zero status for %s\n."),
+		      displaypath);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "foreach", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		argv_array_pushv(&cpr.args, info->argv);
+
+		if (run_command(&cpr))
+			die(_("run_command returned non-zero status while"
+			      "recursing in the nested submodules of %s\n."),
+			      displaypath);
+	}
+
+cleanup:
+	free(displaypath);
+}
+
+static int module_foreach(int argc, const char **argv, const char *prefix)
+{
+	struct cb_foreach info;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_foreach_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of entering each submodule command")),
+		OPT_BOOL(0, "recursive", &recursive,
+			 N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_foreach_options,
+			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, runcommand_in_submodule, &info);
+
+	return 0;
+}
+
 struct module_cb {
 	unsigned int mod_src;
 	unsigned int mod_dst;
@@ -2217,6 +2352,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
+	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 493a64372..e25b2c613 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -298,44 +298,7 @@ cmd_foreach()
 		shift
 	done
 
-	toplevel=$(pwd)
-
-	# dup stdin so that it can be restored when running the external
-	# command in the subshell (and a recursive call to this function)
-	exec 3<&0
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		if test -e "$sm_path"/.git
-		then
-			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-			say "$(eval_gettext "Entering '\$displaypath'")"
-			name=$(git submodule--helper name "$sm_path")
-			(
-				prefix="$prefix$sm_path/"
-				sanitize_submodule_env
-				cd "$sm_path" &&
-				# we make $path available to scripts ...
-				path=$sm_path &&
-				if test $# -eq 1
-				then
-					eval "$1"
-				else
-					"$@"
-				fi &&
-				if test -n "$recursive"
-				then
-					cmd_foreach "--recursive" "$@"
-				fi
-			) <&3 3<&- ||
-			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 #
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* Re: [PATCH 02/15] submodule: don't use submodule_from_name
  2017-07-26 21:06     ` Junio C Hamano
@ 2017-07-30 13:43       ` Jens Lehmann
  2017-07-31 20:43         ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Jens Lehmann @ 2017-07-30 13:43 UTC (permalink / raw)
  To: Junio C Hamano, Stefan Beller; +Cc: Brandon Williams, git, Jonathan Nieder

Am 26.07.2017 um 23:06 schrieb Junio C Hamano:
> Stefan Beller <sbeller@google.com> writes:
> 
>> Rereading the archives, there was quite some discussion on the design
>> of these patches, but these lines of code did not get any attention
>>
>>      https://public-inbox.org/git/4CDB3063.5010801@web.de/
>>
>> I cc'd Jens in the hope of him having a good memory why he
>> wrote the code that way. :)
> 
> Thanks for digging.  I wouldn't be surprised if this were a fallback
> to help a broken entry in .gitmodules that lack .path variable, but
> we shouldn't be sweeping the problem under the rug like that.

Sorry to disappoint you ;-) I added this in 7dce19d374 because
submodule by path lookup back then only parsed the checked out
.gitmodules file. So looking for it by name was a good guess to
fetch a new submodule that wasn't present in the current HEAD's
.gitmodules, as the path is used as the default name in "git
submodule add".

The refactoring in 851e18c385 could and should have removed that
because since then we use the .gitmodules path to name mapping
of the fetched commit.

> I wonder if we should barf loudly if there shouldn't be a submodule
> at that path, i.e.
> 
> 	if (!submodule)
> 		die("there is no submodule defined for path '%s'"...);
> 
> though.

Not sure if you want to die() or just issue a warning(), but yes.

^ permalink raw reply	[relevance 16%]

* Re: [PATCH 02/15] submodule: don't use submodule_from_name
  2017-07-30 13:43       ` Jens Lehmann
@ 2017-07-31 20:43         ` Stefan Beller
  2017-08-11 16:53           ` Heiko Voigt
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-31 20:43 UTC (permalink / raw)
  To: Jens Lehmann; +Cc: Junio C Hamano, Brandon Williams, git, Jonathan Nieder

On Sun, Jul 30, 2017 at 6:43 AM, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> Am 26.07.2017 um 23:06 schrieb Junio C Hamano:
>>
>> Stefan Beller <sbeller@google.com> writes:
>>
>>> Rereading the archives, there was quite some discussion on the design
>>> of these patches, but these lines of code did not get any attention
>>>
>>>      https://public-inbox.org/git/4CDB3063.5010801@web.de/
>>>
>>> I cc'd Jens in the hope of him having a good memory why he
>>> wrote the code that way. :)
>>
>>
>> Thanks for digging.  I wouldn't be surprised if this were a fallback
>> to help a broken entry in .gitmodules that lack .path variable, but
>> we shouldn't be sweeping the problem under the rug like that.
>
>
> Sorry to disappoint you ;-) I added this in 7dce19d374 because
> submodule by path lookup back then only parsed the checked out
> .gitmodules file.

This is still the case AFAICT, as we never ask for a specific .gitmodules
file identified by sha1 of the commit.

> So looking for it by name was a good guess to
> fetch a new submodule that wasn't present in the current HEAD's
> .gitmodules, as the path is used as the default name in "git
> submodule add".

3 things:
a) I think it is not as much a feature ('fallback to still make it work'),
   but rather a bug as when there is no (or wrong) entry in the .gitmodules
   file, reporting it is better than trying something.
b) in the case of moved submodules (2 submodules swapped their path)
   this may be harmful as we'd get a wrong submodule potentially.

c) I wonder if we want to use a different default for submodule names
   as I have seen people get confused by path and name being the same,
   e.g. to move a submodule they would have not just adapted the path,
   but any occurrence of the string that reads like the path.
   (i.e. also change the name, defeating the purpose of name/path
   separation).

   For a new name default, I would wager for some non-legible gibberish
   such as "hash( path/time )", as that sends a clear message to not mess
   with the value of the name.

>
> The refactoring in 851e18c385 could and should have removed that
> because since then we use the .gitmodules path to name mapping
> of the fetched commit.
>
>> I wonder if we should barf loudly if there shouldn't be a submodule
>> at that path, i.e.
>>
>>         if (!submodule)
>>                 die("there is no submodule defined for path '%s'"...);
>>
>> though.
>
>
> Not sure if you want to die() or just issue a warning(), but yes.

Either die() or "warning && return 0" is fine with me.

^ permalink raw reply	[relevance 25%]

* Re: [PATCH 03/15] add, reset: ensure submodules can be added or reset
  2017-07-26 21:25     ` Junio C Hamano
@ 2017-07-31 20:50       ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-07-31 20:50 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Stefan Beller, git, Jonathan Nieder

On 07/26, Junio C Hamano wrote:
> Stefan Beller <sbeller@google.com> writes:
> 
> > On Tue, Jul 25, 2017 at 2:39 PM, Brandon Williams <bmwill@google.com> wrote:
> >> Commit aee9c7d65 (Submodules: Add the new "ignore" config option for
> >> diff and status) ...
> >
> > introduced in 2010, so quite widely spread.
> >
> >> ...  introduced the ignore configuration option for
> >> submodules so that configured submodules could be omitted from the
> >> status and diff commands.  Because this flag is respected in the diff
> >> machinery it has the unintended consequence of potentially prohibiting
> >> users from adding or resetting a submodule, even when a path to the
> >> submodule is explicitly given.
> >>
> >> Ensure that submodules can be added or set, even if they are configured
> >> to be ignored, by setting the `DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG` diff
> >> flag.
> >>
> >> Signed-off-by: Brandon Williams <bmwill@google.com>
> >> ---
> >>  builtin/add.c   | 1 +
> >>  builtin/reset.c | 1 +
> >>  2 files changed, 2 insertions(+)
> >>
> >> diff --git a/builtin/add.c b/builtin/add.c
> >> index e888fb8c5..6f271512f 100644
> >> --- a/builtin/add.c
> >> +++ b/builtin/add.c
> >> @@ -116,6 +116,7 @@ int add_files_to_cache(const char *prefix,
> >>         rev.diffopt.output_format = DIFF_FORMAT_CALLBACK;
> >>         rev.diffopt.format_callback = update_callback;
> >>         rev.diffopt.format_callback_data = &data;
> >> +       rev.diffopt.flags |= DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
> >
> >
> > This flag occurs once in the code base, with the comment:
> >     /*
> >      * Unless the user did explicitly request a submodule
> >      * ignore mode by passing a command line option we do
> >      * not ignore any changed submodule SHA-1s when
> >      * comparing index and parent, no matter what is
> >      * configured. Otherwise we won't commit any
> >      * submodules which were manually staged, which would
> >      * be really confusing.
> >      */
> >     int diff_flags = DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
> >
> > in prepare_commit, so commit ignores the .gitmodules file.
> >
> > This allows git-add to add ignored submodules, currently ignored submodules
> > would have to be added using the plumbing
> >     git update-index --add --cacheinfo 160000,$SHA1,<gitlink>
> 
> Let me play devil's advocate (as I have this suspicion that .ignore
> thing specific for submodule is probably misdesigned and certainly
> its implementation is backwards).  Is the primary use case for this
> .ignore thing to be able to do
> 
> 	git add .
> 
> without having to worry about adding the submodule marked as such?  
> And if so, wouldn't it surprise these users who do use .ignore if
> "git add" suddenly started adding them?
> 
> I think the right tool to use these days for excluding some paths
> when adding all others is the negative pathspec; perhaps back when
> the .ignore thing was added, it didn't exist or not widely known?  
> 
> I suspect that it may result in a better system overall if we can
> deprecate and remove the submodule-specific .ignore thing.  At
> least, I think the DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG is backwards
> in that .ignore causes a submodule to be excluded from the diff by
> default and forces paths that care about differences to opt into the
> "override" thing, which is wrong---the specific UI thing that wants
> not to show them should instead opt into ignoring, while keeping the
> default not to special case such a flag that can only be set to a
> submodule path.

It looks like .ignore was added with aee9c7d65 (Submodules: Add the new
"ignore" config option for diff and status, 2010-08-06) in order to
ignore particular submodules with 'status' and 'diff' commands.  I don't
think it was intended to ignore submodules with commands like add and
reset.  Either way I agree that some of the things with most of the
submodules config seem a bit backwards and we may want to migrate away
from them completely as we begin to add more support for submodules into
the builtin commands.

> 
> > This makes sense, though a test demonstrating the change in behavior
> > would be nice, but git-add doesn't seem to change as it doesn't even load
> > the git modules config?
> >
> >>         rev.max_count = 0; /* do not compare unmerged paths with stage #2 */
> >>         run_diff_files(&rev, DIFF_RACY_IS_MODIFIED);
> >>         return !!data.add_errors;
> >> diff --git a/builtin/reset.c b/builtin/reset.c
> >> index 046403ed6..772d078b8 100644
> >> --- a/builtin/reset.c
> >> +++ b/builtin/reset.c
> >> @@ -156,6 +156,7 @@ static int read_from_tree(const struct pathspec *pathspec,
> >>         opt.output_format = DIFF_FORMAT_CALLBACK;
> >>         opt.format_callback = update_index_from_diff;
> >>         opt.format_callback_data = &intent_to_add;
> >> +       opt.flags |= DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG;
> >
> > same here? Also as this is not failing any test, it may be worth adding one
> > to document the behavior of the "submodule.<name>.ignore" flag in tests?
> >
> >>
> >>         if (do_diff_cache(tree_oid, &opt))
> >>                 return 1;
> >> --
> >> 2.14.0.rc0.400.g1c36432dff-goog
> >>

-- 
Brandon Williams

^ permalink raw reply	[relevance 17%]

* [GSoC][PATCH 00/13] Update: Week-11
@ 2017-07-31 20:56 Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
                   ` (11 more replies)
  0 siblings, 12 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

SUMMARY OF MY PROJECT:

Git submodule subcommands are currently implemented by using shell script
'git-submodule.sh'. There are several reasons why we'll prefer not to
use the shell script. My project intends to convert the subcommands into
C code, thus making them builtins. This will increase Git's portability
and hence the efficiency of working with the git-submodule commands.
Link to the complete proposal: [1]

Mentors:
Stefan Beller <sbeller@google.com>
Christian Couder <christian.couder@gmail.com>

UPDATES:

Following are the updates about my ongoing project:

* Following patches were updated after the previous reviews:
  - set_name_rev()
  submodule subcommands:
  - status
  - sync
  - deinit
  - summary
  - foreach

* Reviews from both Christian Couder <christian.couder@gmail.com> and
  Brandon Williams <bmwill@google.com> helped in improvising these patches
  and their suggestions were implemented.

* Porting of submodule subcommand add is still underway. Its progess can be
  viewed at [2].

PLAN FOR WEEK-12 (1 August 2017 to 7 August 2017):

* summary: One of the problems pointed out by Brandon this week was that
  the function print_submodule_summary() was too big to keep track of
  all the things that are happening. Hence, I will be splitting the
  function into smaller functions.

* displaypath: There is some confusion produced with the way the
  value of displaypath is being generated, using super-prefix. [3]
  Via having discussion on this, I'll try to resolve the issues
  regarding it. In the patches following the update, I have addressed
  this issue as well.

* add: Porting of this subcommand is still underway and will be working
  on to completely port this subcommand.

A complete build report of these series of patches is available at: [4].
Build #145
Branch: week-11

The work is push on github and is available at: [5].

[1]: https://docs.google.com/document/d/1krxVLooWl--75Pot3dazhfygR3wCUUWZWzTXtK1    L-xU/
[2]: https://github.com/pratham-pc/git/commits/sub-add
[3]: https://public-inbox.org/git/20170724213028.GB92874@google.com/
[4]: https://travis-ci.org/pratham-pc/git/builds/
[5]: https://github.com/pratham-pc/git/commits/week-11

  submodule--helper: introduce get_submodule_displaypath()
  submodule--helper: introduce for_each_submodule_list()
  submodule: port set_name_rev() from shell to C
  submodule: port submodule subcommand 'status' from shell to C
  submodule: port submodule subcommand 'sync' from shell to C
  submodule: port submodule subcommand 'deinit' from shell to C
  diff: change scope of the function count_lines()
  submodule: port submodule subcommand 'summary' from shell to C
  submodule foreach: correct '$path' in nested submodules from a
    subdirectory
  submodule foreach: document '$sm_path' instead of '$path'
  submodule foreach: clarify the '$toplevel' variable documentation
  submodule foreach: document variable '$displaypath'
  submodule: port submodule subcommand 'foreach' from shell to C

 Documentation/git-submodule.txt |   15 +-
 builtin/submodule--helper.c     | 1175 ++++++++++++++++++++++++++++++++++++++-
 diff.c                          |    2 +-
 diff.h                          |    1 +
 git-submodule.sh                |  394 +------------
 t/t7407-submodule-foreach.sh    |   38 +-
 6 files changed, 1207 insertions(+), 418 deletions(-)

-- 
2.13.0


^ permalink raw reply	[relevance 18%]

* [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath()
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function get_submodule_displaypath() to replace the code
occurring in submodule_init() for generating displaypath of the
submodule with a call to it.

This new function will also be used in other parts of the system
in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 6abdad329..7af4de09b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -220,6 +220,27 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+
+	if (prefix && super_prefix) {
+		BUG("cannot have prefix '%s' and superprefix '%s'",
+		    prefix, super_prefix);
+	} else if (prefix) {
+		struct strbuf sb = STRBUF_INIT;
+		char *displaypath = xstrdup(relative_path(path, prefix, &sb));
+		strbuf_release(&sb);
+		return displaypath;
+	} else if (super_prefix) {
+		int len = strlen(super_prefix);
+		const char *format = is_dir_sep(super_prefix[len - 1]) ? "%s%s" : "%s/%s";
+		return xstrfmt(format, super_prefix, path);
+	} else {
+		return xstrdup(path);
+	}
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -339,16 +360,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 	/* Only loads from .gitmodules, no overlay with .git/config */
 	gitmodules_config();
-
-	if (prefix && get_super_prefix())
-		die("BUG: cannot have prefix and superprefix");
-	else if (prefix)
-		displaypath = xstrdup(relative_path(path, prefix, &sb));
-	else if (get_super_prefix()) {
-		strbuf_addf(&sb, "%s%s", get_super_prefix(), path);
-		displaypath = strbuf_detach(&sb, NULL);
-	} else
-		displaypath = xstrdup(path);
+	displaypath = get_submodule_displaypath(path, prefix);
 
 	sub = submodule_from_path(null_sha1, path);
 
@@ -363,7 +375,6 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 * Set active flag for the submodule being initialized
 	 */
 	if (!is_submodule_active(the_repository, path)) {
-		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list()
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function for_each_submodule_list() and
replace a loop in module_init() with a call to it.

The new function will also be used in other parts of the
system in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7af4de09b..e41572f7a 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -14,6 +14,9 @@
 #include "refs.h"
 #include "connect.h"
 
+typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
+				      void *cb_data);
+
 static char *get_default_remote(void)
 {
 	char *dest = NULL, *ret;
@@ -352,17 +355,30 @@ static int module_list(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static void init_submodule(const char *path, const char *prefix, int quiet)
+static void for_each_submodule_list(const struct module_list list,
+				    submodule_list_func_t fn, void *cb_data)
 {
+	int i;
+	for (i = 0; i < list.nr; i++)
+		fn(list.entries[i], cb_data);
+}
+
+struct init_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+};
+#define INIT_CB_INIT { NULL, 0 }
+
+static void init_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct init_cb *info = cb_data;
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	/* Only loads from .gitmodules, no overlay with .git/config */
-	gitmodules_config();
-	displaypath = get_submodule_displaypath(path, prefix);
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
 
-	sub = submodule_from_path(null_sha1, path);
+	sub = submodule_from_path(null_sha1, list_item->name);
 
 	if (!sub)
 		die(_("No url found for submodule path '%s' in .gitmodules"),
@@ -374,7 +390,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_active(the_repository, path)) {
+	if (!is_submodule_active(the_repository, list_item->name)) {
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
@@ -416,7 +432,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 		if (git_config_set_gently(sb.buf, url))
 			die(_("Failed to register url for submodule path '%s'"),
 			    displaypath);
-		if (!quiet)
+		if (!info->quiet)
 			fprintf(stderr,
 				_("Submodule '%s' (%s) registered for path '%s'\n"),
 				sub->name, url, displaypath);
@@ -445,10 +461,10 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 static int module_init(int argc, const char **argv, const char *prefix)
 {
+	struct init_cb info = INIT_CB_INIT;
 	struct pathspec pathspec;
 	struct module_list list = MODULE_LIST_INIT;
 	int quiet = 0;
-	int i;
 
 	struct option module_init_options[] = {
 		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
@@ -473,8 +489,11 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	if (!argc && git_config_get_value_multi("submodule.active"))
 		module_list_active(&list);
 
-	for (i = 0; i < list.nr; i++)
-		init_submodule(list.entries[i]->name, prefix, quiet);
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+
+	gitmodules_config();
+	for_each_submodule_list(list, init_submodule, &info);
 
 	return 0;
 }
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Function set_name_rev() is ported from git-submodule to the
submodule--helper builtin. The function get_name_rev() generates the
value of the revision name as required, and the function
print_name_rev() handles the formating and printing of the obtained
revision name.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* The variable namerev from print_name_rev is now freed at the end of the 
  function.

 builtin/submodule--helper.c | 64 +++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 16 ++----------
 2 files changed, 66 insertions(+), 14 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index e41572f7a..2cb72d68e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -244,6 +244,69 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+static char *get_name_rev(const char *sub_path, const char* object_id)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char ***d;
+
+	static const char *describe_bare[] = {
+		NULL
+	};
+
+	static const char *describe_tags[] = {
+		"--tags", NULL
+	};
+
+	static const char *describe_contains[] = {
+		"--contains", NULL
+	};
+
+	static const char *describe_all_always[] = {
+		"--all", "--always", NULL
+	};
+
+	static const char **describe_argv[] = {
+		describe_bare, describe_tags, describe_contains,
+		describe_all_always, NULL
+	};
+
+	for (d = describe_argv; *d; d++) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+		prepare_submodule_repo_env(&cp.env_array);
+		cp.dir = sub_path;
+		cp.git_cmd = 1;
+		cp.no_stderr = 1;
+
+		argv_array_push(&cp.args, "describe");
+		argv_array_pushv(&cp.args, *d);
+		argv_array_push(&cp.args, object_id);
+
+		if (!capture_command(&cp, &sb, 0) && sb.len) {
+			strbuf_strip_suffix(&sb, "\n");
+			return strbuf_detach(&sb, NULL);
+		}
+
+	}
+
+	strbuf_release(&sb);
+	return NULL;
+}
+
+static int print_name_rev(int argc, const char **argv, const char *prefix)
+{
+	char *namerev;
+	if (argc != 3)
+		die("print-name-rev only accepts two arguments: <path> <sha1>");
+
+	namerev = get_name_rev(argv[1], argv[2]);
+	if (namerev && namerev[0])
+		printf(" (%s)", namerev);
+	printf("\n");
+
+	free(namerev);
+	return 0;
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -1242,6 +1305,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index e131760ee..e988167e0 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -759,18 +759,6 @@ cmd_update()
 	}
 }
 
-set_name_rev () {
-	revname=$( (
-		sanitize_submodule_env
-		cd "$1" && {
-			git describe "$2" 2>/dev/null ||
-			git describe --tags "$2" 2>/dev/null ||
-			git describe --contains "$2" 2>/dev/null ||
-			git describe --all --always "$2"
-		}
-	) )
-	test -z "$revname" || revname=" ($revname)"
-}
 #
 # Show commit summary for submodules in index or working tree
 #
@@ -1042,14 +1030,14 @@ cmd_status()
 		fi
 		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
 		then
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say " $sha1 $displaypath$revname"
 		else
 			if test -z "$cached"
 			then
 				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
 			fi
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say "+$sha1 $displaypath$revname"
 		fi
 
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' from shell to C
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (2 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
@ 2017-07-31 20:56 ` " Prathamesh Chavan
  2017-07-31 21:12   ` Stefan Beller
  2017-07-31 20:56 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
                   ` (7 subsequent siblings)
  11 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

This aims to make git-submodule 'status' a built-in. Hence, the function
cmd_status() is ported from shell to C. This is done by introducing
three functions: module_status(), submodule_status() and print_status().

The function module_status() acts as the front-end of the subcommand.
It parses subcommand's options and then calls the function
module_list_compute() for computing the list of submodules. Then
this functions calls for_each_submodule_list() looping through the
list obtained.

Then for_each_submodule_list() calls submodule_status() for each of the
submodule in its list. The function submodule_status() is responsible
for generating the status each submodule it is called for, and
then calls print_status().

Finally, the function print_status() handles the printing of submodule's
status.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* parameters passed to the function print_status() have been changed.
  Instead of passing char *sub_sha1, instead the object_id is being passed.
 
* Also, since the passed parameter displaypath's value isn't changed
  by the function, it is passed to the funcition as const char *displaypath
  instead of char *displaypath.

* the output type of the function handle_submodule_head_ref() is changed
  from strbuf to object_id, as we will use the object_id instead of the
  hex of sha1 being stored in a struct strbuf.

* diff_files_args is cleared after using it by passing it as args in the
  function cmd_diff_files.

* In the function status_submodule(), for checking if a submodule has merge
  conflicts, the patch currently checks if the value of any of the ce_flags
  is non-zero. Currently, I think the we aren't interested in a partiular flag,
  but I'm not sure on this.

* Debugging leftovers and suprious new-lines are removed.

* The confusion with displaypath being passed as te super-prefix in many
  of the ported subcommands may be a result of the fact that the
  function generating the displaypath: get_submodule_displaypath()
  uses the super-prefix as simply a path concatenated with the current
  submodule name to denote our current location.
  The function get_super_prefix() is declared in cache.h and defined in
  environment.c, but is majorly used in the builtin/submodule--helper.c
  and also in unpack-trees.c
  Also, for generating any submodule's displaypath, it would be important to
  have ".." passed to the submodule, and currently it is possible only via the
  super-prefix.
  This is also other instaces where the super-prefix contained ".." as well.
  One of such instance is Test 4 from t7406-submodule-update.sh
  Hence, maybe documenting the value of displaypath might a solution
  for the above problem.
  I'm just stating my views and would like to recieve your opinion on this
  matter.

 builtin/submodule--helper.c | 154 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  49 +-------------
 2 files changed, 155 insertions(+), 48 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2cb72d68e..a6e6a48cc 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -561,6 +561,159 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct status_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+	unsigned int cached: 1;
+};
+#define STATUS_CB_INIT { NULL, 0, 0, 0 }
+
+static void print_status(struct status_cb *info, char state, const char *path,
+			 const struct object_id *oid, const char *displaypath)
+{
+	if (info->quiet)
+		return;
+
+	printf("%c%s %s", state, oid_to_hex(oid), displaypath);
+
+	if (state == ' ' || state == '+') {
+		struct argv_array name_rev_args = ARGV_ARRAY_INIT;
+
+		argv_array_pushl(&name_rev_args, "print-name-rev",
+				 path, oid_to_hex(oid), NULL);
+		print_name_rev(name_rev_args.argc, name_rev_args.argv,
+			       info->prefix);
+	} else {
+		printf("\n");
+	}
+}
+
+static int handle_submodule_head_ref(const char *refname,
+				     const struct object_id *oid, int flags,
+				     void *cb_data)
+{
+	struct object_id *output = cb_data;
+	if (oid)
+		oidcpy(output, oid);
+
+	return 0;
+}
+
+static void status_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct status_cb *info = cb_data;
+	char *displaypath;
+	struct argv_array diff_files_args = ARGV_ARRAY_INIT;
+
+	if (!submodule_from_path(null_sha1, list_item->name))
+		die(_("no submodule mapping found in .gitmodules for path '%s'"),
+		      list_item->name);
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (list_item->ce_flags) {
+		print_status(info, 'U', list_item->name,
+			     &null_oid, displaypath);
+		goto cleanup;
+	}
+
+	if (!is_submodule_active(the_repository, list_item->name)) {
+		print_status(info, '-', list_item->name, &list_item->oid,
+			     displaypath);
+		goto cleanup;
+	}
+
+	argv_array_pushl(&diff_files_args, "diff-files",
+			 "--ignore-submodules=dirty", "--quiet", "--",
+			 list_item->name, NULL);
+
+	if (!cmd_diff_files(diff_files_args.argc, diff_files_args.argv,
+			    info->prefix)) {
+		print_status(info, ' ', list_item->name, &list_item->oid,
+			     displaypath);
+	} else {
+		if (!info->cached) {
+			struct object_id oid;
+
+			if (head_ref_submodule(list_item->name,
+					       handle_submodule_head_ref, &oid))
+				die(_("could not resolve HEAD ref inside the"
+				      "submodule '%s'"), list_item->name);
+
+			print_status(info, '+', list_item->name, &oid,
+				     displaypath);
+		} else {
+			print_status(info, '+', list_item->name,
+				     &list_item->oid, displaypath);
+		}
+	}
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "status", "--recursive",
+				 NULL);
+
+		if (info->cached)
+			argv_array_push(&cpr.args, "--cached");
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	argv_array_clear(&diff_files_args);
+	free(displaypath);
+}
+
+static int module_status(int argc, const char **argv, const char *prefix)
+{
+	struct status_cb info = STATUS_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int cached = 0;
+	int recursive = 0;
+
+	struct option module_status_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT_BOOL(0, "cached", &cached, N_("Use commit stored in the index instead of the one stored in the submodule HEAD")),
+		OPT_BOOL(0, "recursive", &recursive, N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule status [--quiet] [--cached] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_status_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+	info.cached = !!cached;
+
+	gitmodules_config();
+	for_each_submodule_list(list, status_submodule, &info);
+
+	return 0;
+}
+
 static int module_name(int argc, const char **argv, const char *prefix)
 {
 	const struct submodule *sub;
@@ -1307,6 +1460,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
+	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index e988167e0..51b057d82 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1005,54 +1005,7 @@ cmd_status()
 		shift
 	done
 
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		if test "$stage" = U
-		then
-			say "U$sha1 $displaypath"
-			continue
-		fi
-		if ! git submodule--helper is-active "$sm_path" ||
-		{
-			! test -d "$sm_path"/.git &&
-			! test -f "$sm_path"/.git
-		}
-		then
-			say "-$sha1 $displaypath"
-			continue;
-		fi
-		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
-		then
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say " $sha1 $displaypath$revname"
-		else
-			if test -z "$cached"
-			then
-				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
-			fi
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say "+$sha1 $displaypath$revname"
-		fi
-
-		if test -n "$recursive"
-		then
-			(
-				prefix="$displaypath/"
-				sanitize_submodule_env
-				wt_prefix=
-				cd "$sm_path" &&
-				eval cmd_status
-			) ||
-			die "$(eval_gettext "Failed to recurse into submodule path '\$sm_path'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} "$@"
 }
 #
 # Sync remote urls for submodules
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' from shell to C
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (3 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-07-31 20:56 ` " Prathamesh Chavan
  2017-07-31 21:19   ` Stefan Beller
  2017-07-31 20:56 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
                   ` (6 subsequent siblings)
  11 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Port the submodule subcommand 'sync' from shell to C using the same
mechanism as that used for porting submodule subcommand 'status'.
Hence, here the function cmd_sync() is ported from shell to C.
This is done by introducing three functions: module_sync(),
sync_submodule() and print_default_remote().

The function print_default_remote() is introduced for getting
the default remote as stdout.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* There was no good reason for using puts in the function print_default_remote()
  Hence, in this patch, we instead use printf to do the same, as it is what
  is generally used throughout the codebase.

* As suggested, this patch ensures a more efficient use of variables, and
  removes most of the variables by reusing 'strbuf sb' at places required.

 builtin/submodule--helper.c | 182 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  56 +-------------
 2 files changed, 183 insertions(+), 55 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index a6e6a48cc..91945337f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -44,6 +44,20 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static int print_default_remote(int argc, const char **argv, const char *prefix)
+{
+	const char *remote;
+
+	if (argc != 1)
+		die(_("submodule--helper print-default-remote takes no arguments"));
+
+	remote = get_default_remote();
+	if (remote)
+		printf("%s\n", remote);
+
+	return 0;
+}
+
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -380,6 +394,25 @@ static void module_list_active(struct module_list *list)
 	*list = active_modules;
 }
 
+static char *get_up_path(const char *path)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	for (i = count_slashes(path); i; i--)
+		strbuf_addstr(&sb, "../");
+
+	/*
+	 * Check if 'path' ends with slash or not
+	 * for having the same output for dir/sub_dir
+	 * and dir/sub_dir/
+	 */
+	if (!is_dir_sep(path[strlen(path) - 1]))
+		strbuf_addstr(&sb, "../");
+
+	return strbuf_detach(&sb, NULL);
+}
+
 static int module_list(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -733,6 +766,153 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct sync_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define SYNC_CB_INIT { NULL, 0, 0 }
+
+static void sync_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct sync_cb *info = cb_data;
+	const struct submodule *sub;
+	char *remote_key;
+	char *sub_origin_url, *super_config_url, *displaypath;
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *sub_config_path = NULL;
+
+	if (!is_submodule_active(the_repository, list_item->name))
+		return;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (sub && sub->url) {
+		if (starts_with_dot_dot_slash(sub->url) || starts_with_dot_slash(sub->url)) {
+			char *remote_url, *up_path;
+			char *remote = get_default_remote();
+			strbuf_addf(&sb, "remote.%s.url", remote);
+
+			if (git_config_get_string(sb.buf, &remote_url))
+				remote_url = xgetcwd();
+
+			up_path = get_up_path(list_item->name);
+			sub_origin_url = relative_url(remote_url, sub->url, up_path);
+			super_config_url = relative_url(remote_url, sub->url, NULL);
+
+			free(remote);
+			free(up_path);
+			free(remote_url);
+		} else {
+			sub_origin_url = xstrdup(sub->url);
+			super_config_url = xstrdup(sub->url);
+		}
+	} else {
+		sub_origin_url = "";
+		super_config_url = "";
+	}
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (!info->quiet)
+		printf(_("Synchronizing submodule url for '%s'\n"),
+			 displaypath);
+
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "submodule.%s.url", sub->name);
+	if (git_config_set_gently(sb.buf, super_config_url))
+		die(_("failed to register url for submodule path '%s'"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.dir = list_item->name;
+	argv_array_pushl(&cp.args, "submodule--helper",
+			 "print-default-remote", NULL);
+
+	strbuf_reset(&sb);
+	if (capture_command(&cp, &sb, 0))
+		die(_("failed to get the default remote for submodule '%s'"),
+		      list_item->name);
+
+	strbuf_strip_suffix(&sb, "\n");
+	remote_key = xstrfmt("remote.%s.url", sb.buf);
+
+	strbuf_reset(&sb);
+	submodule_to_gitdir(&sb, list_item->name);
+	strbuf_addstr(&sb, "/config");
+
+	if (git_config_set_in_file_gently(sb.buf, remote_key, sub_origin_url))
+		die(_("failed to update remote for submodule '%s'"),
+		      list_item->name);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "sync", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	strbuf_release(&sb);
+	free(super_config_url);
+	free(displaypath);
+	free(sub_config_path);
+	free(sub_origin_url);
+}
+
+static int module_sync(int argc, const char **argv, const char *prefix)
+{
+	struct sync_cb info = SYNC_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_sync_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of synchronizing submodule url")),
+		OPT_BOOL(0, "recursive", &recursive,
+			N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper sync [--quiet] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_sync_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, sync_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1461,6 +1641,8 @@ static struct cmd_struct commands[] = {
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
+	{"print-default-remote", print_default_remote, 0},
+	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 51b057d82..6bfc5e17d 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1037,63 +1037,9 @@ cmd_sync()
 			;;
 		esac
 	done
-	cd_to_toplevel
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-
-		# skip inactive submodules
-		if ! git submodule--helper is-active "$sm_path"
-		then
-			continue
-		fi
-
-		name=$(git submodule--helper name "$sm_path")
-		url=$(git config -f .gitmodules --get submodule."$name".url)
-
-		# Possibly a url relative to parent
-		case "$url" in
-		./*|../*)
-			# rewrite foo/bar as ../.. to find path from
-			# submodule work tree to superproject work tree
-			up_path="$(printf '%s\n' "$sm_path" | sed "s/[^/][^/]*/../g")" &&
-			# guarantee a trailing /
-			up_path=${up_path%/}/ &&
-			# path from submodule work tree to submodule origin repo
-			sub_origin_url=$(git submodule--helper resolve-relative-url "$url" "$up_path") &&
-			# path from superproject work tree to submodule origin repo
-			super_config_url=$(git submodule--helper resolve-relative-url "$url") || exit
-			;;
-		*)
-			sub_origin_url="$url"
-			super_config_url="$url"
-			;;
-		esac
 
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		say "$(eval_gettext "Synchronizing submodule url for '\$displaypath'")"
-		git config submodule."$name".url "$super_config_url"
-
-		if test -e "$sm_path"/.git
-		then
-		(
-			sanitize_submodule_env
-			cd "$sm_path"
-			remote=$(get_default_remote)
-			git config remote."$remote".url "$sub_origin_url"
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 
-			if test -n "$recursive"
-			then
-				prefix="$prefix$sm_path/"
-				eval cmd_sync
-			fi
-		)
-		fi
-	done
 }
 
 cmd_absorbgitdirs()
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' from shell to C
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (4 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-07-31 20:56 ` " Prathamesh Chavan
  2017-07-31 21:42   ` Stefan Beller
  2017-07-31 20:56 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
                   ` (5 subsequent siblings)
  11 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The same mechanism is used even for porting this submodule
subcommand, as used in the ported subcommands till now.
The function cmd_deinit in split up after porting into three
functions: module_deinit(), for_each_submodule_list() and
deinit_submodule().

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* In the function deinit_submodule, since the test is_git_directory()
  adds an additional condition, instead is_directory() is used to check
  if "sm_path/.git" is a directory.

* since it was possible in the previous path that the value st.st_mode passed
  to the function mkdir contained a garbage value, instead we intrduce a
  mode_t variable mode, initially containing a default mode value '0777'.
  This is what the default of mode is set in case, that the value is
  not set after the lstat call. 

 builtin/submodule--helper.c | 144 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  55 +----------------
 2 files changed, 145 insertions(+), 54 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 91945337f..f642f9889 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -913,6 +913,149 @@ static int module_sync(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct deinit_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int force: 1;
+	unsigned int all: 1;
+};
+#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
+
+static void deinit_submodule(const struct cache_entry *list_item,
+			     void *cb_data)
+{
+	struct deinit_cb *info = cb_data;
+	const struct submodule *sub;
+	char *displaypath = NULL;
+	struct child_process cp_config = CHILD_PROCESS_INIT;
+	struct strbuf sb_config = STRBUF_INIT;
+	char *sub_git_dir = xstrfmt("%s/.git", list_item->name);
+	mode_t mode = 0777;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub || !sub->name)
+		goto cleanup;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	/* remove the submodule work tree (unless the user already did it) */
+	if (is_directory(list_item->name)) {
+		struct stat st;
+		/* protect submodules containing a .git directory */
+		if (is_directory(sub_git_dir))
+			die(_("Submodule work tree '%s' contains a .git "
+			      "directory use 'rm -rf' if you really want "
+			      "to remove it including all of its history"),
+			      displaypath);
+
+		if (!info->force) {
+			struct child_process cp_rm = CHILD_PROCESS_INIT;
+			cp_rm.git_cmd = 1;
+			argv_array_pushl(&cp_rm.args, "rm", "-qn",
+					 list_item->name, NULL);
+
+			if (run_command(&cp_rm))
+				die(_("Submodule work tree '%s' contains local "
+				      "modifications; use '-f' to discard them"),
+				      displaypath);
+		}
+
+		if (!lstat(list_item->name, &st)) {
+			struct strbuf sb_rm = STRBUF_INIT;
+			const char *format;
+
+			strbuf_addstr(&sb_rm, list_item->name);
+
+			if (!remove_dir_recursively(&sb_rm, 0))
+				format = _("Cleared directory '%s'\n");
+			else
+				format = _("Could not remove submodule work tree '%s'\n");
+
+			if (!info->quiet)
+				printf(format, displaypath);
+
+			mode = st.st_mode;
+
+			strbuf_release(&sb_rm);
+		}
+	}
+
+	if (mkdir(list_item->name, mode))
+		die(_("could not create empty submodule directory %s"),
+		      displaypath);
+
+	cp_config.git_cmd = 1;
+	argv_array_pushl(&cp_config.args, "config", "--get-regexp", NULL);
+	argv_array_pushf(&cp_config.args, "submodule.%s\\.", sub->name);
+
+	/* remove the .git/config entries (unless the user already did it) */
+	if (!capture_command(&cp_config, &sb_config, 0) && sb_config.len) {
+		char *sub_key = xstrfmt("submodule.%s", sub->name);
+		/*
+		 * remove the whole section so we have a clean state when
+		 * the user later decides to init this submodule again
+		 */
+		git_config_rename_section_in_file(NULL, sub_key, NULL);
+		if (!info->quiet)
+			printf(_("Submodule '%s' (%s) unregistered for path '%s'\n"),
+				 sub->name, sub->url, displaypath);
+		free(sub_key);
+	}
+
+cleanup:
+	free(displaypath);
+	free(sub_git_dir);
+	strbuf_release(&sb_config);
+}
+
+static int module_deinit(int argc, const char **argv, const char *prefix)
+{
+	struct deinit_cb info = DEINIT_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int force = 0;
+	int all = 0;
+
+	struct option module_deinit_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT__FORCE(&force, N_("Remove submodule working trees even if they contain local changes")),
+		OPT_BOOL(0, "all", &all, N_("Unregister all submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule deinit [--quiet] [-f | --force] [--all | [--] [<path>...]]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_deinit_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.all = !!all;
+	info.force = !!force;
+
+	if (all && argc) {
+		error("pathspec and --all are incompatible");
+		usage_with_options(git_submodule_helper_usage,
+				   module_deinit_options);
+	}
+
+	if (!argc && !all)
+		die(_("Use '--all' if you really want to deinitialize all submodules"));
+
+	gitmodules_config();
+	for_each_submodule_list(list, deinit_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1643,6 +1786,7 @@ static struct cmd_struct commands[] = {
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
+	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 6bfc5e17d..73e6f093f 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -428,60 +428,7 @@ cmd_deinit()
 		shift
 	done
 
-	if test -n "$deinit_all" && test "$#" -ne 0
-	then
-		echo >&2 "$(eval_gettext "pathspec and --all are incompatible")"
-		usage
-	fi
-	if test $# = 0 && test -z "$deinit_all"
-	then
-		die "$(eval_gettext "Use '--all' if you really want to deinitialize all submodules")"
-	fi
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-
-		displaypath=$(git submodule--helper relative-path "$sm_path" "$wt_prefix")
-
-		# Remove the submodule work tree (unless the user already did it)
-		if test -d "$sm_path"
-		then
-			# Protect submodules containing a .git directory
-			if test -d "$sm_path/.git"
-			then
-				die "$(eval_gettext "\
-Submodule work tree '\$displaypath' contains a .git directory
-(use 'rm -rf' if you really want to remove it including all of its history)")"
-			fi
-
-			if test -z "$force"
-			then
-				git rm -qn "$sm_path" ||
-				die "$(eval_gettext "Submodule work tree '\$displaypath' contains local modifications; use '-f' to discard them")"
-			fi
-			rm -rf "$sm_path" &&
-			say "$(eval_gettext "Cleared directory '\$displaypath'")" ||
-			say "$(eval_gettext "Could not remove submodule work tree '\$displaypath'")"
-		fi
-
-		mkdir "$sm_path" || say "$(eval_gettext "Could not create empty submodule directory '\$displaypath'")"
-
-		# Remove the .git/config entries (unless the user already did it)
-		if test -n "$(git config --get-regexp submodule."$name\.")"
-		then
-			# Remove the whole section so we have a clean state when
-			# the user later decides to init this submodule again
-			url=$(git config submodule."$name".url)
-			git config --remove-section submodule."$name" 2>/dev/null &&
-			say "$(eval_gettext "Submodule '\$name' (\$url) unregistered for path '\$displaypath'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${force:+--force} ${deinit_all:+--all} "$@"
 }
 
 is_tip_reachable () (
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' from shell to C
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (5 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
@ 2017-07-31 20:56 ` " Prathamesh Chavan
  2017-07-31 22:15   ` Stefan Beller
  2017-07-31 20:56 ` [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
                   ` (4 subsequent siblings)
  11 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The submodule subcommand 'summary' is ported in the process of
making git-submodule a builtin. The function cmd_summary() from
git-submodule.sh is ported to functions module_summary(),
compute_summary_module_list(), prepare_submodule_summary() and
print_submodule_summary().

The first function module_summary() parses the options of submodule
subcommand and also acts as the front-end of this subcommand.
After parsing them, it calls the compute_summary_module_list()

The functions compute_summary_module_list() runs the diff_cmd,
and generates the modules list, as required by the subcommand.
The generation of this module list is done by the using the
callback function submodule_summary_callback(), and stored in the
structure module_cb.

Once the module list is generated, prepare_submodule_summary()
further goes through the list and filters the list, for
eventually calling the print_submodule_summary() function.

Finally, the print_submodule_summary() takes care of generating
and printing the summary for each submodule.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:

* Firstly, about the function compute_summary_module_list().
  This function is created to generate the list of modules, for which
  we will generate the summary further. Since the list is actually
  generated using the git-diff-files or git-diff-index command, but for
  porting this, we required to create a function similar to the builtin
  functions of the above commands. But we can't directly call cmd_diff_files() 
  and cmd_diff_index() since we don't have to display the output and instead
  need to store it. Hence, this function is introduced.
 
* Also, the module_cb_list *list is not freed since it is a non-heap object.
  Hence, free() can't be using on the non-heap objects.

* In the function prepare_submodule_summary(), as suggested
  'git_config_get_string_const' was used instead of instead of '_value'

* Some variables which weren't modified throughout the function-call were
  passed as const.

* The '!!' trick, which wasn't used in the last patch, is now used in this
  new version .

* the variables sha1_dst and sha1_src are removed from the function
  print_submodule_summary(), and instead the p->oid_src and p->oid_dst are
  used.

* The variable sm_git_dir is freed at the end of the function.

* variable head was no longer used in module_summary() and instead the strbuf
  was utilized.

 builtin/submodule--helper.c | 425 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 182 +------------------
 2 files changed, 426 insertions(+), 181 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f642f9889..94438d6ce 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -13,6 +13,9 @@
 #include "remote.h"
 #include "refs.h"
 #include "connect.h"
+#include "revision.h"
+#include "diffcore.h"
+#include "diff.h"
 
 typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
 				      void *cb_data);
@@ -766,6 +769,427 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct module_cb {
+	unsigned int mod_src;
+	unsigned int mod_dst;
+	struct object_id oid_src;
+	struct object_id oid_dst;
+	char status;
+	const char *sm_path;
+};
+#define MODULE_CB_INIT { 0, 0, NULL, NULL, '\0', NULL }
+
+struct module_cb_list {
+	struct module_cb **entries;
+	int alloc, nr;
+};
+#define MODULE_CB_LIST_INIT { NULL, 0, 0 }
+
+struct summary_cb {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	char *diff_cmd;
+	unsigned int cached: 1;
+	unsigned int for_status: 1;
+	unsigned int quiet: 1;
+	unsigned int files: 1;
+	int summary_limits;
+};
+#define SUMMARY_CB_INIT { 0, NULL, NULL, NULL, 0, 0, 0, 0, 0 }
+
+static int verify_submodule_object_name(const char *sm_path, const char *sha1)
+{
+	struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+	cp_rev_parse.git_cmd = 1;
+	cp_rev_parse.no_stdout = 1;
+	cp_rev_parse.dir = sm_path;
+	prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+	argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
+			 "--verify", NULL);
+	argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1);
+
+	if (run_command(&cp_rev_parse))
+		return 1;
+
+	return 0;
+}
+
+static void print_submodule_summary(struct summary_cb *info,
+				    struct module_cb *p)
+{
+	int missing_src = 0;
+	int missing_dst = 0;
+	char *displaypath;
+	const char *sha1_abbr_src;
+	const char *sha1_abbr_dst;
+	int errmsg = 0;
+	int total_commits = -1;
+	char *sm_git_dir = xstrfmt("%s/.git", p->sm_path);
+	int is_sm_git_dir = 0;
+
+	if (!info->cached && !oidcmp(&p->oid_dst, &null_oid)) {
+		if (S_ISGITLINK(p->mod_dst)) {
+			struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_parse = STRBUF_INIT;
+
+			cp_rev_parse.git_cmd = 1;
+			cp_rev_parse.no_stderr = 1;
+			cp_rev_parse.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+			argv_array_pushl(&cp_rev_parse.args,
+					 "rev-parse", "HEAD", NULL);
+			if (!capture_command(&cp_rev_parse, &sb_rev_parse, 0)) {
+				strbuf_strip_suffix(&sb_rev_parse, "\n");
+
+				get_oid_hex(sb_rev_parse.buf, &p->oid_dst);
+			}
+			strbuf_release(&sb_rev_parse);
+		} else if (S_ISLNK(p->mod_dst) || S_ISREG(p->mod_dst)) {
+			struct child_process cp_hash_object = CHILD_PROCESS_INIT;
+			struct strbuf sb_hash_object = STRBUF_INIT;
+
+			cp_hash_object.git_cmd = 1;
+			argv_array_pushl(&cp_hash_object.args,
+					 "hash-object", p->sm_path,
+					 NULL);
+			if (!capture_command(&cp_hash_object,
+					     &sb_hash_object, 0)) {
+				strbuf_strip_suffix(&sb_hash_object, "\n");
+
+				get_oid_hex(sb_hash_object.buf, &p->oid_dst);
+			}
+			strbuf_release(&sb_hash_object);
+		} else {
+			if (p->mod_dst)
+				die(_("unexpected mode %d\n"), p->mod_dst);
+		}
+	}
+
+	if (is_git_directory(sm_git_dir))
+		is_sm_git_dir = 1;
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_src))
+		missing_src = verify_submodule_object_name(p->sm_path,
+							   oid_to_hex(&p->oid_src));
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_dst))
+		missing_dst = verify_submodule_object_name(p->sm_path,
+							   oid_to_hex(&p->oid_dst));
+
+	displaypath = get_submodule_displaypath(p->sm_path, info->prefix);
+
+	if (!missing_dst && !missing_src) {
+		if (is_sm_git_dir) {
+			struct child_process cp_rev_list = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_list = STRBUF_INIT;
+			char *range;
+
+			if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst))
+				range = xstrfmt("%s...%s", oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+			else if (S_ISGITLINK(p->mod_src))
+				range = xstrdup(oid_to_hex(&p->oid_src));
+			else
+				range = xstrdup(oid_to_hex(&p->oid_dst));
+
+			cp_rev_list.git_cmd = 1;
+			cp_rev_list.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_list.env_array);
+
+			argv_array_pushl(&cp_rev_list.args, "rev-list",
+					 "--first-parent", range, "--", NULL);
+			if (!capture_command(&cp_rev_list, &sb_rev_list, 0)) {
+				if (sb_rev_list.len)
+					total_commits = count_lines(sb_rev_list.buf,
+								    sb_rev_list.len);
+				else
+					total_commits = 0;
+			}
+
+			free(range);
+			strbuf_release(&sb_rev_list);
+		}
+	} else {
+		errmsg = 1;
+	}
+
+	sha1_abbr_src = find_unique_abbrev(p->oid_src.hash, 7);
+	sha1_abbr_dst = find_unique_abbrev(p->oid_dst.hash, 7);
+
+	if (p->status == 'T') {
+		if (S_ISGITLINK(p->mod_dst))
+			printf(_("* %s %s(blob)->%s(submodule)"),
+				 displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+		else
+			printf(_("* %s %s(submodule)->%s(blob)"),
+				 displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+	} else {
+			printf("* %s %s...%s", displaypath, sha1_abbr_src,
+				 sha1_abbr_dst);
+	}
+
+	if (total_commits < 0)
+		printf(":\n");
+	else
+		printf(" (%d):\n", total_commits);
+
+	if (errmsg) {
+		/*
+		 * Don't give error msg for modification whose dst is not
+		 * submodule, i.e. deleted or changed to blob
+		 */
+		if (S_ISGITLINK(p->mod_src)) {
+			if (missing_src && missing_dst) {
+				printf(_("  Warn: %s doesn't contain commits %s and %s\n"),
+				 displaypath, oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+			} else if (missing_src) {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, oid_to_hex(&p->oid_src));
+			} else {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, oid_to_hex(&p->oid_dst));
+			}
+		}
+	} else if (is_sm_git_dir) {
+		struct child_process cp_log = CHILD_PROCESS_INIT;
+
+		cp_log.git_cmd = 1;
+		cp_log.dir = p->sm_path;
+		prepare_submodule_repo_env(&cp_log.env_array);
+		argv_array_pushl(&cp_log.args, "log", NULL);
+
+		if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst)) {
+			if (info->summary_limits > 0)
+				argv_array_pushf(&cp_log.args, "-%d", info->summary_limits);
+
+			argv_array_pushl(&cp_log.args, "--pretty=  %m %s",
+					 "--first-parent", NULL);
+			argv_array_pushf(&cp_log.args, "%s...%s", oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+		} else if (S_ISGITLINK(p->mod_dst)) {
+			argv_array_pushl(&cp_log.args, "--pretty=  > %s",
+					 "-1", oid_to_hex(&p->oid_dst), NULL);
+		} else {
+			argv_array_pushl(&cp_log.args, "--pretty=  < %s",
+					 "-1", oid_to_hex(&p->oid_src), NULL);
+		}
+
+		run_command(&cp_log);
+	}
+	printf("\n");
+
+	free(displaypath);
+	free(sm_git_dir);
+}
+
+static void prepare_submodule_summary(struct summary_cb *info,
+				      struct module_cb_list *list)
+{
+	int i;
+	for (i = 0; i < list->nr; i++) {
+		struct module_cb *p = list->entries[i];
+		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+		if (p->status == 'D' || p->status == 'T') {
+			print_submodule_summary(info, p);
+			continue;
+		}
+
+		if (info->for_status) {
+			char *config_key;
+			const char *ignore_config = "none";
+			const char *value;
+			const struct submodule *sub = submodule_from_path(null_sha1, p->sm_path);
+
+			if (sub && p->status != 'A') {
+				config_key = xstrfmt("submodule.%s.ignore",
+						     sub->name);
+				if (!git_config_get_string_const(config_key, &value))
+					ignore_config = value;
+				else if (sub->ignore)
+					ignore_config = sub->ignore;
+
+				free(config_key);
+
+				if (!strcmp(ignore_config, "all"))
+					continue;
+			}
+		}
+
+		/* Also show added or modified modules which are checked out */
+		cp_rev_parse.dir = p->sm_path;
+		cp_rev_parse.git_cmd = 1;
+		cp_rev_parse.no_stderr = 1;
+		cp_rev_parse.no_stdout = 1;
+
+		argv_array_pushl(&cp_rev_parse.args, "rev-parse",
+				 "--git-dir", NULL);
+
+		if (!run_command(&cp_rev_parse))
+			print_submodule_summary(info, p);
+	}
+}
+
+static void submodule_summary_callback(struct diff_queue_struct *q,
+				       struct diff_options *options,
+				       void *data)
+{
+	int i;
+	struct module_cb_list *list = data;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		struct module_cb *temp;
+
+		if (!S_ISGITLINK(p->one->mode) && !S_ISGITLINK(p->two->mode))
+			continue;
+		temp = (struct module_cb*)malloc(sizeof(struct module_cb));
+		temp->mod_src = p->one->mode;
+		temp->mod_dst = p->two->mode;
+		temp->oid_src = p->one->oid;
+		temp->oid_dst = p->two->oid;
+		temp->status = p->status;
+		temp->sm_path = xstrdup(p->one->path);
+
+		ALLOC_GROW(list->entries, list->nr + 1, list->alloc);
+		list->entries[list->nr++] = temp;
+	}
+}
+
+static int compute_summary_module_list(char *head, struct summary_cb *info)
+{
+	struct argv_array diff_args = ARGV_ARRAY_INIT;
+	struct rev_info rev;
+	struct module_cb_list list = MODULE_CB_LIST_INIT;
+
+	argv_array_push(&diff_args, info->diff_cmd);
+	if (info->cached)
+		argv_array_push(&diff_args, "--cached");
+	argv_array_pushl(&diff_args, "--ignore-submodules=dirty", "--raw",
+			 NULL);
+	if (head)
+		argv_array_push(&diff_args, head);
+	argv_array_push(&diff_args, "--");
+	if (info->argc)
+		argv_array_pushv(&diff_args, info->argv);
+
+	git_config(git_diff_basic_config, NULL);
+	init_revisions(&rev, info->prefix);
+	gitmodules_config();
+	rev.abbrev = 0;
+	precompose_argv(diff_args.argc, diff_args.argv);
+
+	diff_args.argc = setup_revisions(diff_args.argc, diff_args.argv,
+					 &rev, NULL);
+	rev.diffopt.output_format = DIFF_FORMAT_NO_OUTPUT | DIFF_FORMAT_CALLBACK;
+	rev.diffopt.format_callback = submodule_summary_callback;
+	rev.diffopt.format_callback_data = &list;
+
+	if (!info->cached) {
+		if (!strcmp(info->diff_cmd, "diff-index"))
+			setup_work_tree();
+		if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
+			perror("read_cache_preload");
+			return -1;
+		}
+	} else if (read_cache() < 0) {
+		perror("read_cache");
+		return -1;
+	}
+
+	if (!strcmp(info->diff_cmd, "diff-index"))
+		run_diff_index(&rev, info->cached);
+	else
+		run_diff_files(&rev, 0);
+	prepare_submodule_summary(info, &list);
+
+	free(head);
+	return 0;
+
+}
+
+static int module_summary(int argc, const char **argv, const char *prefix)
+{
+	struct summary_cb info = SUMMARY_CB_INIT;
+	int cached = 0;
+	char *diff_cmd = "diff-index";
+	int for_status = 0;
+	int quiet = 0;
+	int files = 0;
+	int summary_limits = -1;
+	struct child_process cp_rev = CHILD_PROCESS_INIT;
+	struct strbuf sb = STRBUF_INIT;
+
+	struct option module_summary_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
+		OPT_BOOL(0, "cached", &cached, N_("Use the commit stored in the index instead of the submodule HEAD")),
+		OPT_BOOL(0, "files", &files, N_("To compares the commit in the index with that in the submodule HEAD")),
+		OPT_BOOL(0, "for-status", &for_status, N_("Skip submodules with 'all' ignore_config value")),
+		OPT_INTEGER('n', "summary-limits", &summary_limits, N_("Limit the summary size")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper summary [<options>] [--] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_summary_options,
+			     git_submodule_helper_usage, 0);
+
+	if (!summary_limits)
+		return 0;
+
+	cp_rev.git_cmd = 1;
+	argv_array_pushl(&cp_rev.args, "rev-parse", "-q", "--verify",
+			 argc ? argv[0] : "HEAD", NULL);
+
+	if (!capture_command(&cp_rev, &sb, 0)) {
+		strbuf_strip_suffix(&sb, "\n");
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else if (!argc || !strcmp(argv[0], "HEAD")) {
+		/* before the first commit: compare with an empty tree */
+		struct stat st;
+		struct object_id oid;
+		if (fstat(0, &st) < 0 || index_fd(oid.hash, 0, &st, 2, prefix, 3))
+			die("Unable to add %s to database", oid.hash);
+		strbuf_addstr(&sb, oid_to_hex(&oid));
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else {
+		strbuf_addstr(&sb, "HEAD");
+	}
+
+	if (files) {
+		if (cached)
+			die(_("The --cached option cannot be used with the --files option"));
+		diff_cmd = "diff-files";
+
+		strbuf_reset(&sb);
+		sb.buf = NULL;
+	}
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.cached = !!cached;
+	info.for_status = !!for_status;
+	info.quiet = quiet;
+	info.files = files;
+	info.summary_limits = summary_limits;
+	info.diff_cmd = diff_cmd;
+
+	return compute_summary_module_list(strbuf_detach(&sb, NULL), &info);
+}
+
 struct sync_cb {
 	const char *prefix;
 	unsigned int quiet: 1;
@@ -1787,6 +2211,7 @@ static struct cmd_struct commands[] = {
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
+	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 73e6f093f..a427ddafd 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -51,31 +51,6 @@ die_if_unmatched ()
 	fi
 }
 
-#
-# Print a submodule configuration setting
-#
-# $1 = submodule name
-# $2 = option name
-# $3 = default value
-#
-# Checks in the usual git-config places first (for overrides),
-# otherwise it falls back on .gitmodules.  This allows you to
-# distribute project-wide defaults in .gitmodules, while still
-# customizing individual repositories if necessary.  If the option is
-# not in .gitmodules either, print a default value.
-#
-get_submodule_config () {
-	name="$1"
-	option="$2"
-	default="$3"
-	value=$(git config submodule."$name"."$option")
-	if test -z "$value"
-	then
-		value=$(git config -f .gitmodules submodule."$name"."$option")
-	fi
-	printf '%s' "${value:-$default}"
-}
-
 isnumber()
 {
 	n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
@@ -755,163 +730,8 @@ cmd_summary() {
 		shift
 	done
 
-	test $summary_limit = 0 && return
-
-	if rev=$(git rev-parse -q --verify --default HEAD ${1+"$1"})
-	then
-		head=$rev
-		test $# = 0 || shift
-	elif test -z "$1" || test "$1" = "HEAD"
-	then
-		# before the first commit: compare with an empty tree
-		head=$(git hash-object -w -t tree --stdin </dev/null)
-		test -z "$1" || shift
-	else
-		head="HEAD"
-	fi
-
-	if [ -n "$files" ]
-	then
-		test -n "$cached" &&
-		die "$(gettext "The --cached option cannot be used with the --files option")"
-		diff_cmd=diff-files
-		head=
-	fi
-
-	cd_to_toplevel
-	eval "set $(git rev-parse --sq --prefix "$wt_prefix" -- "$@")"
-	# Get modified modules cared by user
-	modules=$(git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- "$@" |
-		sane_egrep '^:([0-7]* )?160000' |
-		while read -r mod_src mod_dst sha1_src sha1_dst status sm_path
-		do
-			# Always show modules deleted or type-changed (blob<->module)
-			if test "$status" = D || test "$status" = T
-			then
-				printf '%s\n' "$sm_path"
-				continue
-			fi
-			# Respect the ignore setting for --for-status.
-			if test -n "$for_status"
-			then
-				name=$(git submodule--helper name "$sm_path")
-				ignore_config=$(get_submodule_config "$name" ignore none)
-				test $status != A && test $ignore_config = all && continue
-			fi
-			# Also show added or modified modules which are checked out
-			GIT_DIR="$sm_path/.git" git-rev-parse --git-dir >/dev/null 2>&1 &&
-			printf '%s\n' "$sm_path"
-		done
-	)
-
-	test -z "$modules" && return
-
-	git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- $modules |
-	sane_egrep '^:([0-7]* )?160000' |
-	cut -c2- |
-	while read -r mod_src mod_dst sha1_src sha1_dst status name
-	do
-		if test -z "$cached" &&
-			test $sha1_dst = 0000000000000000000000000000000000000000
-		then
-			case "$mod_dst" in
-			160000)
-				sha1_dst=$(GIT_DIR="$name/.git" git rev-parse HEAD)
-				;;
-			100644 | 100755 | 120000)
-				sha1_dst=$(git hash-object $name)
-				;;
-			000000)
-				;; # removed
-			*)
-				# unexpected type
-				eval_gettextln "unexpected mode \$mod_dst" >&2
-				continue ;;
-			esac
-		fi
-		missing_src=
-		missing_dst=
-
-		test $mod_src = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_src^0 >/dev/null &&
-		missing_src=t
-
-		test $mod_dst = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_dst^0 >/dev/null &&
-		missing_dst=t
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${for_status:+--for-status} ${files:+--files} ${cached:+--cached} ${summary_limit:+-n $summary_limit} "$@"
 
-		display_name=$(git submodule--helper relative-path "$name" "$wt_prefix")
-
-		total_commits=
-		case "$missing_src,$missing_dst" in
-		t,)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_src")"
-			;;
-		,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_dst")"
-			;;
-		t,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commits \$sha1_src and \$sha1_dst")"
-			;;
-		*)
-			errmsg=
-			total_commits=$(
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				range="$sha1_src...$sha1_dst"
-			elif test $mod_src = 160000
-			then
-				range=$sha1_src
-			else
-				range=$sha1_dst
-			fi
-			GIT_DIR="$name/.git" \
-			git rev-list --first-parent $range -- | wc -l
-			)
-			total_commits=" ($(($total_commits + 0)))"
-			;;
-		esac
-
-		sha1_abbr_src=$(echo $sha1_src | cut -c1-7)
-		sha1_abbr_dst=$(echo $sha1_dst | cut -c1-7)
-		if test $status = T
-		then
-			blob="$(gettext "blob")"
-			submodule="$(gettext "submodule")"
-			if test $mod_dst = 160000
-			then
-				echo "* $display_name $sha1_abbr_src($blob)->$sha1_abbr_dst($submodule)$total_commits:"
-			else
-				echo "* $display_name $sha1_abbr_src($submodule)->$sha1_abbr_dst($blob)$total_commits:"
-			fi
-		else
-			echo "* $display_name $sha1_abbr_src...$sha1_abbr_dst$total_commits:"
-		fi
-		if test -n "$errmsg"
-		then
-			# Don't give error msg for modification whose dst is not submodule
-			# i.e. deleted or changed to blob
-			test $mod_dst = 160000 && echo "$errmsg"
-		else
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				limit=
-				test $summary_limit -gt 0 && limit="-$summary_limit"
-				GIT_DIR="$name/.git" \
-				git log $limit --pretty='format:  %m %s' \
-				--first-parent $sha1_src...$sha1_dst
-			elif test $mod_dst = 160000
-			then
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  > %s' -1 $sha1_dst
-			else
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  < %s' -1 $sha1_src
-			fi
-			echo
-		fi
-		echo
-	done
 }
 #
 # List all submodules, prefixed with:
-- 
2.13.0


^ permalink raw reply	[relevance 15%]

* [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (6 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

When running 'git submodule foreach' from a subdirectory of your
repository, nested submodules get a bogus value for $sm_path:
For a submodule 'sub' that contains a nested submodule 'nested',
running 'git -C dir submodule foreach echo $path' would report
path='../nested' for the nested submodule. The first part '../' is
derived from the logic computing the relative path from $pwd to the
root of the superproject. The second part is the submodule path inside
the submodule. This value is of little use and is hard to document.

There are two different possible solutions that have more value:
(a) The path value is documented as the path from the toplevel of the
    superproject to the mount point of the submodule.
    In this case we would want to have path='sub/nested'.

(b) As Ramsay noticed the documented value is wrong. For the non-nested
    case the path is equal to the relative path from $pwd to the
    submodules working directory. When following this model,
    the expected value would be path='../sub/nested'.

The behavior for (b) was introduced in 091a6eb0fe (submodule: drop the
top-level requirement, 2013-06-16) the intent for $path seemed to be
relative to $cwd to the submodule worktree, but that did not work for
nested submodules, as the intermittent submodules were not included in
the path.

If we were to fix the meaning of the $path using (a) such that "path"
is "the path from the toplevel of the superproject to the mount point
of the submodule", we would break any existing submodule user that runs
foreach from non-root of the superproject as the non-nested submodule
'../sub' would change its path to 'sub'.

If we would fix the meaning of the $path using (b), such that "path"
is "the relative path from $pwd to the submodule", then we would break
any user that uses nested submodules (even from the root directory) as
the 'nested' would become 'sub/nested'.

Both groups can be found in the wild.  The author has no data if one group
outweighs the other by large margin, and offending each one seems equally
bad at first.  However in the authors imagination it is better to go with
(a) as running from a sub directory sounds like it is carried out
by a human rather than by some automation task.  With a human on
the keyboard the feedback loop is short and the changed behavior can be
adapted to quickly unlike some automation that can break silently.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh             |  1 -
 t/t7407-submodule-foreach.sh | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index a427ddafd..493a64372 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -320,7 +320,6 @@ cmd_foreach()
 				prefix="$prefix$sm_path/"
 				sanitize_submodule_env
 				cd "$sm_path" &&
-				sm_path=$(git submodule--helper relative-path "$sm_path" "$wt_prefix") &&
 				# we make $path available to scripts ...
 				path=$sm_path &&
 				if test $# -eq 1
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 6ba5daf42..0663622a4 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,9 +82,9 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-../sub1-$sub1sha1
+$pwd/clone-foo1-sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-../sub3-$sub3sha1
+$pwd/clone-foo3-sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
@@ -196,6 +196,38 @@ test_expect_success 'test messages from "foreach --recursive" from subdirectory'
 	) &&
 	test_i18ncmp expect actual
 '
+sub1sha1=$(cd clone2/sub1 && git rev-parse HEAD)
+sub2sha1=$(cd clone2/sub2 && git rev-parse HEAD)
+sub3sha1=$(cd clone2/sub3 && git rev-parse HEAD)
+nested1sha1=$(cd clone2/nested1 && git rev-parse HEAD)
+nested2sha1=$(cd clone2/nested1/nested2 && git rev-parse HEAD)
+nested3sha1=$(cd clone2/nested1/nested2/nested3 && git rev-parse HEAD)
+submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEAD)
+
+cat >expect <<EOF
+Entering '../nested1'
+$pwd/clone2-nested1-nested1-$nested1sha1
+Entering '../nested1/nested2'
+$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+Entering '../nested1/nested2/nested3'
+$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+Entering '../nested1/nested2/nested3/submodule'
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+Entering '../sub1'
+$pwd/clone2-foo1-sub1-$sub1sha1
+Entering '../sub2'
+$pwd/clone2-foo2-sub2-$sub2sha1
+Entering '../sub3'
+$pwd/clone2-foo3-sub3-$sub3sha1
+EOF
+
+test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
+	(
+		cd clone2/untracked &&
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+	) &&
+	test_i18ncmp expect actual
+'
 
 cat > expect <<EOF
 nested1-nested1
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path'
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (7 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

As using a variable '$path' may be harmful to users due to
capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
use $path variable in eval_gettext string, 2012-04-17). Adjust
the documentation to advocate for using $sm_path,  which contains
the same value. We still make the 'path' variable available and
document it as a deprecated synonym of 'sm_path'.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
This patch is same as its previous version.
Although here I'll like to add a point that we aim to slowly drop the support
of the variable 'path'.

 Documentation/git-submodule.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index ff612001d..a23baef62 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,12 +183,14 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $path, $sha1 and
+	The command has access to the variables $name, $sm_path, $sha1 and
 	$toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
-	$path is the name of the submodule directory relative to the
-	superproject, $sha1 is the commit as recorded in the superproject,
-	and $toplevel is the absolute path to the top-level of the superproject.
+	$sm_path is the path of the submodule as recorded in the superproject,
+	$sha1 is the commit as recorded in the superproject, and
+	$toplevel is the absolute path to the top-level of the superproject.
+	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
+	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given `--quiet`, foreach prints the name
 	of each submodule before evaluating the command.
-- 
2.13.0


^ permalink raw reply	[relevance 23%]

* [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (8 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

It does not contain the topmost superproject as the author assumed,
but the direct superproject, such that $toplevel/$sm_path is the
actual absolute path of the submodule.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index a23baef62..8e7930ebc 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -188,7 +188,8 @@ foreach [--recursive] <command>::
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
 	$sha1 is the commit as recorded in the superproject, and
-	$toplevel is the absolute path to the top-level of the superproject.
+	$toplevel is the absolute path to its superproject, such that
+	$toplevel/$sm_path is the absolute path of the submodule.
 	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
 	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
-- 
2.13.0


^ permalink raw reply	[relevance 24%]

* [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath'
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (9 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 20:56 ` [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

It was observed that the variable '$displaypath' was accessible but
undocumented. Hence, document it.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* Spelling mistake in the commit message was corrected.

 Documentation/git-submodule.txt |  6 ++++--
 t/t7407-submodule-foreach.sh    | 22 +++++++++++-----------
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 8e7930ebc..0cca702cb 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,10 +183,12 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $sm_path, $sha1 and
-	$toplevel:
+	The command has access to the variables $name, $sm_path, $displaypath,
+	$sha1 and $toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
+	$displaypath contains the relative path from the current working
+	directory to the submodules root directory,
 	$sha1 is the commit as recorded in the superproject, and
 	$toplevel is the absolute path to its superproject, such that
 	$toplevel/$sm_path is the absolute path of the submodule.
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 0663622a4..6ad57e061 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,16 +82,16 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-sub1-$sub1sha1
+$pwd/clone-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-sub3-$sub3sha1
+$pwd/clone-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
 	mkdir clone/sub &&
 	(
 		cd clone/sub &&
-		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
@@ -206,25 +206,25 @@ submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEA
 
 cat >expect <<EOF
 Entering '../nested1'
-$pwd/clone2-nested1-nested1-$nested1sha1
+$pwd/clone2-nested1-nested1-../nested1-$nested1sha1
 Entering '../nested1/nested2'
-$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+$pwd/clone2/nested1-nested2-nested2-../nested1/nested2-$nested2sha1
 Entering '../nested1/nested2/nested3'
-$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+$pwd/clone2/nested1/nested2-nested3-nested3-../nested1/nested2/nested3-$nested3sha1
 Entering '../nested1/nested2/nested3/submodule'
-$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-../nested1/nested2/nested3/submodule-$submodulesha1
 Entering '../sub1'
-$pwd/clone2-foo1-sub1-$sub1sha1
+$pwd/clone2-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub2'
-$pwd/clone2-foo2-sub2-$sub2sha1
+$pwd/clone2-foo2-sub2-../sub2-$sub2sha1
 Entering '../sub3'
-$pwd/clone2-foo3-sub3-$sub3sha1
+$pwd/clone2-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
 	(
 		cd clone2/untracked &&
-		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C
  2017-07-31 20:56 [GSoC][PATCH 00/13] Update: Week-11 Prathamesh Chavan
                   ` (10 preceding siblings ...)
  2017-07-31 20:56 ` [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
@ 2017-07-31 20:56 ` Prathamesh Chavan
  2017-07-31 22:20   ` Stefan Beller
  11 siblings, 1 reply; 200+ results
From: Prathamesh Chavan @ 2017-07-31 20:56 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

This aims to make git-submodule foreach a builtin. This is the very
first step taken in this direction. Hence, 'foreach' is ported to
submodule--helper, and submodule--helper is called from git-submodule.sh.
The code is split up to have one function to obtain all the list of
submodules. This function acts as the front-end of git-submodule foreach
subcommand. It calls the function for_each_submodule_list, which basically
loops through the list and calls function fn, which in this case is
runcommand_in_submodule. This third function is a calling function that
takes care of running the command in that submodule, and recursively
perform the same when --recursive is flagged.

The first function module_foreach first parses the options present in
argv, and then with the help of module_list_compute, generates the list of
submodules present in the current working tree.

The second function for_each_submodule_list traverses through the
list, and calls function fn (which in case of submodule subcommand
foreach is runcommand_in_submodule) is called for each entry.

The third function runcommand_in_submodule, generates a submodule struct sub
for $name, value and then later prepends name=sub->name; and other
value assignment to the env argv_array structure of a child_process.
Also the <command> of submodule-foreach is push to args argv_array
structure and finally, using run_command the commands are executed
using a shell.

The third function also takes care of the recursive flag, by creating
a separate child_process structure and prepending "--super-prefix displaypath",
to the args argv_array structure. Other required arguments and the
input <command> of submodule-foreach is also appended to this argv_array.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* Comment style is improved in the function runcommand_in_submodule()

* Comment in added about why the variable "path" was exposed via args
  argv_array instead of exposing it via the env_array.

* This patch exposes the various variables when argc == 1 only, just
  for maintaining a faithful porting. You can also find discussion about
  the same at [1].

[1]: https://public-inbox.org/git/CAME+mvUSGAFbN5j-_hv7QpAS57hq4wgH+yZ7XJMPuyQN1gALaA@mail.gmail.com/

 builtin/submodule--helper.c | 136 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  39 +------------
 2 files changed, 137 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 94438d6ce..935f56510 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -769,6 +769,141 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct cb_foreach {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define CB_FOREACH_INIT { 0, NULL, NULL, 0, 0 }
+
+static void runcommand_in_submodule(const struct cache_entry *list_item,
+				    void *cb_data)
+{
+	struct cb_foreach *info = cb_data;
+	const struct submodule *sub;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *displaypath;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub)
+		die(_("No url found for submodule path '%s' in .gitmodules"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+
+	/*
+	 * For the purpose of executing <command> in the submodule,
+	 * separate shell is used for the purpose of running the
+	 * child process.
+	 */
+	cp.use_shell = 1;
+	cp.dir = list_item->name;
+
+	if (info->argc == 1) {
+		char *toplevel = xgetcwd();
+
+		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
+		argv_array_pushf(&cp.env_array, "sm_path=%s", list_item->name);
+		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
+		argv_array_pushf(&cp.env_array, "sha1=%s",
+				 oid_to_hex(&list_item->oid));
+		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
+
+		/*
+		 * Since the path variable was accessible from the script
+		 * before porting, it is also made available after porting.
+		 * The environment variable "PATH" has a very special purpose
+		 * on windows. And since environment variables are
+		 * case-insensitive in windows, it interferes with the
+		 * existing PATH variable. Hence, to avoid that, we expose
+		 * path via the args argv_array and not via env_array.
+		 */
+		argv_array_pushf(&cp.args, "path=%s; %s",
+				 list_item->name, info->argv[0]);
+		free(toplevel);
+	} else {
+		argv_array_pushv(&cp.args, info->argv);
+	}
+
+	if (!info->quiet)
+		printf(_("Entering '%s'\n"), displaypath);
+
+	if (info->argv[0] && run_command(&cp))
+		die(_("run_command returned non-zero status for %s\n."),
+		      displaypath);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "foreach", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		argv_array_pushv(&cpr.args, info->argv);
+
+		if (run_command(&cpr))
+			die(_("run_command returned non-zero status while"
+			      "recursing in the nested submodules of %s\n."),
+			      displaypath);
+	}
+
+cleanup:
+	free(displaypath);
+}
+
+static int module_foreach(int argc, const char **argv, const char *prefix)
+{
+	struct cb_foreach info;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_foreach_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of entering each submodule command")),
+		OPT_BOOL(0, "recursive", &recursive,
+			 N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_foreach_options,
+			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, runcommand_in_submodule, &info);
+
+	return 0;
+}
+
 struct module_cb {
 	unsigned int mod_src;
 	unsigned int mod_dst;
@@ -2206,6 +2341,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
+	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 493a64372..e25b2c613 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -298,44 +298,7 @@ cmd_foreach()
 		shift
 	done
 
-	toplevel=$(pwd)
-
-	# dup stdin so that it can be restored when running the external
-	# command in the subshell (and a recursive call to this function)
-	exec 3<&0
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		if test -e "$sm_path"/.git
-		then
-			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-			say "$(eval_gettext "Entering '\$displaypath'")"
-			name=$(git submodule--helper name "$sm_path")
-			(
-				prefix="$prefix$sm_path/"
-				sanitize_submodule_env
-				cd "$sm_path" &&
-				# we make $path available to scripts ...
-				path=$sm_path &&
-				if test $# -eq 1
-				then
-					eval "$1"
-				else
-					"$@"
-				fi &&
-				if test -n "$recursive"
-				then
-					cmd_foreach "--recursive" "$@"
-				fi
-			) <&3 3<&- ||
-			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 #
-- 
2.13.0


^ permalink raw reply	[relevance 20%]

* Re: [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' from shell to C
  2017-07-31 20:56 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-07-31 21:12   ` Stefan Beller
  2017-08-01 21:14     ` Prathamesh Chavan
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-31 21:12 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Mon, Jul 31, 2017 at 1:56 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> This aims to make git-submodule 'status' a built-in. Hence, the function
> cmd_status() is ported from shell to C. This is done by introducing
> three functions: module_status(), submodule_status() and print_status().
>
> The function module_status() acts as the front-end of the subcommand.
> It parses subcommand's options and then calls the function
> module_list_compute() for computing the list of submodules. Then
> this functions calls for_each_submodule_list() looping through the
> list obtained.
>
> Then for_each_submodule_list() calls submodule_status() for each of the
> submodule in its list. The function submodule_status() is responsible
> for generating the status each submodule it is called for, and
> then calls print_status().
>
> Finally, the function print_status() handles the printing of submodule's
> status.
>
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
> In this new version, the following changes have been made:
> * parameters passed to the function print_status() have been changed.
>   Instead of passing char *sub_sha1, instead the object_id is being passed.
>
> * Also, since the passed parameter displaypath's value isn't changed
>   by the function, it is passed to the funcition as const char *displaypath
>   instead of char *displaypath.
>
> * the output type of the function handle_submodule_head_ref() is changed
>   from strbuf to object_id, as we will use the object_id instead of the
>   hex of sha1 being stored in a struct strbuf.
>
> * diff_files_args is cleared after using it by passing it as args in the
>   function cmd_diff_files.
>
> * In the function status_submodule(), for checking if a submodule has merge
>   conflicts, the patch currently checks if the value of any of the ce_flags
>   is non-zero. Currently, I think the we aren't interested in a partiular flag,
>   but I'm not sure on this.
>
> * Debugging leftovers and suprious new-lines are removed.
>
> * The confusion with displaypath being passed as te super-prefix in many
>   of the ported subcommands may be a result of the fact that the
>   function generating the displaypath: get_submodule_displaypath()
>   uses the super-prefix as simply a path concatenated with the current
>   submodule name to denote our current location.
>   The function get_super_prefix() is declared in cache.h and defined in
>   environment.c, but is majorly used in the builtin/submodule--helper.c
>   and also in unpack-trees.c
>   Also, for generating any submodule's displaypath, it would be important to
>   have ".." passed to the submodule, and currently it is possible only via the
>   super-prefix.
>   This is also other instaces where the super-prefix contained ".." as well.
>   One of such instance is Test 4 from t7406-submodule-update.sh
>   Hence, maybe documenting the value of displaypath might a solution
>   for the above problem.
>   I'm just stating my views and would like to recieve your opinion on this
>   matter.

Yes, I agree that the display path is not quite easily understood as it can be
ambiguous.  I am confused by this paragraph:
* does test 4 from 7406 fail here, or was it just the starting point
  of the discussion and it all works fine?

I have reviewed the patches up to (and including this) patch and
so far they look good to me.

Thanks,
Stefan

^ permalink raw reply	[relevance 18%]

* Re: [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' from shell to C
  2017-07-31 20:56 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-07-31 21:19   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-31 21:19 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Mon, Jul 31, 2017 at 1:56 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> In this new version, the following changes have been made:
> * There was no good reason for using puts in the function print_default_remote()
>   Hence, in this patch, we instead use printf to do the same, as it is what
>   is generally used throughout the codebase.

Hah! I refrained on suggesting the opposite at the last patch.
In older code there are lots of "puts", c.f. "git grep puts", so I assumed
it to be a micro optimization, hence I would have suggested it.
Either way is fine with me.

^ permalink raw reply	[relevance 18%]

* Re: [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' from shell to C
  2017-07-31 20:56 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
@ 2017-07-31 21:42   ` Stefan Beller
  2017-08-01 21:19     ` Prathamesh Chavan
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-31 21:42 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Mon, Jul 31, 2017 at 1:56 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> The same mechanism is used even for porting this submodule
> subcommand, as used in the ported subcommands till now.
> The function cmd_deinit in split up after porting into three
> functions: module_deinit(), for_each_submodule_list() and
> deinit_submodule().
>
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
> In this new version, the following changes have been made:
> * In the function deinit_submodule, since the test is_git_directory()
>   adds an additional condition, instead is_directory() is used to check
>   if "sm_path/.git" is a directory.

Thanks for writing these patches.
I wonder if (some of) these notes are best put into the code
as a comment such as

    /* NEEDSWORK: convert to is_submodule_active */

such that people reading this code later realize that checking
for a directory may not be the "correct" thing, but a thing which
was easy to express using shell.

> +struct deinit_cb {
> +       const char *prefix;
> +       unsigned int quiet: 1;
> +       unsigned int force: 1;
> +       unsigned int all: 1;

The value 'all' seems to be unused, i.e. we assign it but never read it?

> +};
> +#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
> +
> +static void deinit_submodule(const struct cache_entry *list_item,
> +                            void *cb_data)
> +{
> +       struct deinit_cb *info = cb_data;
> +       const struct submodule *sub;
> +       char *displaypath = NULL;
> +       struct child_process cp_config = CHILD_PROCESS_INIT;
> +       struct strbuf sb_config = STRBUF_INIT;
> +       char *sub_git_dir = xstrfmt("%s/.git", list_item->name);
> +       mode_t mode = 0777;
> +
> +       sub = submodule_from_path(null_sha1, list_item->name);
> +
> +       if (!sub || !sub->name)
> +               goto cleanup;
> +
> +       displaypath = get_submodule_displaypath(list_item->name, info->prefix);
> +
> +       /* remove the submodule work tree (unless the user already did it) */
> +       if (is_directory(list_item->name)) {
> +               struct stat st;
> +               /* protect submodules containing a .git directory */

Here may a good place to put:
  /* NEEDSWORK: automatically call absorbgitdirs before warning/die. */
(It was not in the shell version, so feel free to ignore)

> +               if (!info->force) {
> +                       struct child_process cp_rm = CHILD_PROCESS_INIT;
> +                       cp_rm.git_cmd = 1;
> +                       argv_array_pushl(&cp_rm.args, "rm", "-qn",
> +                                        list_item->name, NULL);

A bug that exists in the shell version as well as here:
What if the submodule has the name '--cached', which happens
to be a valid argument for git-rm?

The call to git-rm would die claiming that the <file> is missing,
as the file name was miss-interpreted as another flag.

To solve this problem we would insert a '--' after the options,
before the file name to state that the last argument is a <file>.

Not sure if we want to fix the bug while we're here or if we rather
want to add

    /* NEEDSWORK: add '--' to confirm <file> argument */

> +       if (mkdir(list_item->name, mode))
> +               die(_("could not create empty submodule directory %s"),
> +                     displaypath);

In the shell version we used the shell mkdir, which on failure
would print the error to stderr on its own. The added "|| say"
is just to clarify the bigger picture.

mkdir in C doesn't print the actual error (e.g.
no diskspace, permissions) so we have to do it.
Use 'die_errno' instead as then the reason will be given as well.

> +       cp_config.git_cmd = 1;
> +       argv_array_pushl(&cp_config.args, "config", "--get-regexp", NULL);
> +       argv_array_pushf(&cp_config.args, "submodule.%s\\.", sub->name);

It would be nice if the commit message could give hints as why a literal
translation was chosen here and not e.g. one of the builtin config
calls, as that would omit spawning a process.

^ permalink raw reply	[relevance 22%]

* Re: [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' from shell to C
  2017-07-31 20:56 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
@ 2017-07-31 22:15   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-31 22:15 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Mon, Jul 31, 2017 at 1:56 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> The submodule subcommand 'summary' is ported in the process of
> making git-submodule a builtin. The function cmd_summary() from
> git-submodule.sh is ported to functions module_summary(),
> compute_summary_module_list(), prepare_submodule_summary() and
> print_submodule_summary().
>
> The first function module_summary() parses the options of submodule
> subcommand and also acts as the front-end of this subcommand.
> After parsing them, it calls the compute_summary_module_list()
>
> The functions compute_summary_module_list() runs the diff_cmd,
> and generates the modules list, as required by the subcommand.
> The generation of this module list is done by the using the
> callback function submodule_summary_callback(), and stored in the
> structure module_cb.
>
> Once the module list is generated, prepare_submodule_summary()
> further goes through the list and filters the list, for
> eventually calling the print_submodule_summary() function.
>
> Finally, the print_submodule_summary() takes care of generating
> and printing the summary for each submodule.
>
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
> In this new version, the following changes have been made:
>
> * Firstly, about the function compute_summary_module_list().
>   This function is created to generate the list of modules, for which
>   we will generate the summary further. Since the list is actually
>   generated using the git-diff-files or git-diff-index command, but for
>   porting this, we required to create a function similar to the builtin
>   functions of the above commands. But we can't directly call cmd_diff_files()
>   and cmd_diff_index() since we don't have to display the output and instead
>   need to store it. Hence, this function is introduced.
>
> * Also, the module_cb_list *list is not freed since it is a non-heap object.
>   Hence, free() can't be using on the non-heap objects.
>
> * In the function prepare_submodule_summary(), as suggested
>   'git_config_get_string_const' was used instead of instead of '_value'
>
> * Some variables which weren't modified throughout the function-call were
>   passed as const.
>
> * The '!!' trick, which wasn't used in the last patch, is now used in this
>   new version .
>
> * the variables sha1_dst and sha1_src are removed from the function
>   print_submodule_summary(), and instead the p->oid_src and p->oid_dst are
>   used.
>
> * The variable sm_git_dir is freed at the end of the function.
>
> * variable head was no longer used in module_summary() and instead the strbuf
>   was utilized.
>
>  builtin/submodule--helper.c | 425 ++++++++++++++++++++++++++++++++++++++++++++
>  git-submodule.sh            | 182 +------------------
>  2 files changed, 426 insertions(+), 181 deletions(-)
>
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index f642f9889..94438d6ce 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -13,6 +13,9 @@
>  #include "remote.h"
>  #include "refs.h"
>  #include "connect.h"
> +#include "revision.h"
> +#include "diffcore.h"
> +#include "diff.h"
>
>  typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
>                                       void *cb_data);
> @@ -766,6 +769,427 @@ static int module_name(int argc, const char **argv, const char *prefix)
>         return 0;
>  }
>
> +struct module_cb {
> +       unsigned int mod_src;
> +       unsigned int mod_dst;
> +       struct object_id oid_src;
> +       struct object_id oid_dst;
> +       char status;
> +       const char *sm_path;
> +};
> +#define MODULE_CB_INIT { 0, 0, NULL, NULL, '\0', NULL }
> +
> +struct module_cb_list {
> +       struct module_cb **entries;
> +       int alloc, nr;
> +};
> +#define MODULE_CB_LIST_INIT { NULL, 0, 0 }
> +
> +struct summary_cb {
> +       int argc;
> +       const char **argv;
> +       const char *prefix;
> +       char *diff_cmd;
> +       unsigned int cached: 1;
> +       unsigned int for_status: 1;
> +       unsigned int quiet: 1;
> +       unsigned int files: 1;
> +       int summary_limits;
> +};
> +#define SUMMARY_CB_INIT { 0, NULL, NULL, NULL, 0, 0, 0, 0, 0 }
> +
> +static int verify_submodule_object_name(const char *sm_path, const char *sha1)
> +{
> +       struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
> +
> +       cp_rev_parse.git_cmd = 1;
> +       cp_rev_parse.no_stdout = 1;
> +       cp_rev_parse.dir = sm_path;
> +       prepare_submodule_repo_env(&cp_rev_parse.env_array);
> +
> +       argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
> +                        "--verify", NULL);
> +       argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1);
> +
> +       if (run_command(&cp_rev_parse))
> +               return 1;
> +
> +       return 0;
> +}
> +
> +static void print_submodule_summary(struct summary_cb *info,
> +                                   struct module_cb *p)
> +{
> +       int missing_src = 0;
> +       int missing_dst = 0;
> +       char *displaypath;
> +       const char *sha1_abbr_src;
> +       const char *sha1_abbr_dst;
> +       int errmsg = 0;
> +       int total_commits = -1;
> +       char *sm_git_dir = xstrfmt("%s/.git", p->sm_path);
> +       int is_sm_git_dir = 0;
> +
> +       if (!info->cached && !oidcmp(&p->oid_dst, &null_oid)) {
> +               if (S_ISGITLINK(p->mod_dst)) {
> +                       struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
> +                       struct strbuf sb_rev_parse = STRBUF_INIT;
> +
> +                       cp_rev_parse.git_cmd = 1;
> +                       cp_rev_parse.no_stderr = 1;
> +                       cp_rev_parse.dir = p->sm_path;
> +                       prepare_submodule_repo_env(&cp_rev_parse.env_array);
> +
> +                       argv_array_pushl(&cp_rev_parse.args,
> +                                        "rev-parse", "HEAD", NULL);
> +                       if (!capture_command(&cp_rev_parse, &sb_rev_parse, 0)) {
> +                               strbuf_strip_suffix(&sb_rev_parse, "\n");
> +
> +                               get_oid_hex(sb_rev_parse.buf, &p->oid_dst);
> +                       }
> +                       strbuf_release(&sb_rev_parse);

I think this could be replaced via
head_ref_submodule(sub->path, callback function, &where_to_store)
or is there some trickery going on, that this also works on
non-compliant submodules?
(Maybe add that as a NEEDSWORK/TODO)

> +static int compute_summary_module_list(char *head, struct summary_cb *info)
> +{
> +       struct argv_array diff_args = ARGV_ARRAY_INIT;
> +       struct rev_info rev;
> +       struct module_cb_list list = MODULE_CB_LIST_INIT;
> +
> +       argv_array_push(&diff_args, info->diff_cmd);
> +       if (info->cached)
> +               argv_array_push(&diff_args, "--cached");
> +       argv_array_pushl(&diff_args, "--ignore-submodules=dirty", "--raw",
> +                        NULL);
> +       if (head)
> +               argv_array_push(&diff_args, head);
> +       argv_array_push(&diff_args, "--");
> +       if (info->argc)
> +               argv_array_pushv(&diff_args, info->argv);
> +
> +       git_config(git_diff_basic_config, NULL);
> +       init_revisions(&rev, info->prefix);
> +       gitmodules_config();
> +       rev.abbrev = 0;

Recently there was a discussion how to operate the
revision machinery best (search for earlier versions of
js/rebase-i-final if interested), whether we can and want
to directly set flags such as .abbrev or if we'd rather
want to push "--abbrev=0" to the diff_args before the --

bisect and archive both assign abbrev directly, so I think
we're fine here.

> +       precompose_argv(diff_args.argc, diff_args.argv);
> +
> +       diff_args.argc = setup_revisions(diff_args.argc, diff_args.argv,
> +                                        &rev, NULL);
> +       rev.diffopt.output_format = DIFF_FORMAT_NO_OUTPUT | DIFF_FORMAT_CALLBACK;
> +       rev.diffopt.format_callback = submodule_summary_callback;
> +       rev.diffopt.format_callback_data = &list;
> +
> +       if (!info->cached) {
> +               if (!strcmp(info->diff_cmd, "diff-index"))

This strcmp smells like we're encoding the state not optimally
in 'info'.  Maybe we can have an enum { DIFF_FILES, DIFF_INDEX }
instead of a string (that we assign earlier) and then have to
compare to it again.

> +                       setup_work_tree();
> +               if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
> +                       perror("read_cache_preload");
> +                       return -1;
> +               }
> +       } else if (read_cache() < 0) {
> +               perror("read_cache");
> +               return -1;
> +       }

This cascaded decision whether to use
setup_work_tree / read_cache_preload / read_cache
seems quite optimized, hence complicated to read. :)
I like it, though.

> +
> +       if (!summary_limits)
> +               return 0;

Good call for converting "test $summary_limit = 0 && return".
I suspected this may be an overeager optimization (as no
error checking is done at all, but that is what it is)
f2dc06a344 (git-submodule summary: limit summary size,
2008-03-11) introduced it like this.

^ permalink raw reply	[relevance 25%]

* Re: [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C
  2017-07-31 20:56 ` [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
@ 2017-07-31 22:20   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-07-31 22:20 UTC (permalink / raw)
  To: Prathamesh Chavan; +Cc: git, Christian Couder

On Mon, Jul 31, 2017 at 1:56 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
> This aims to make git-submodule foreach a builtin. This is the very
> first step taken in this direction. Hence, 'foreach' is ported to
> submodule--helper, and submodule--helper is called from git-submodule.sh.
> The code is split up to have one function to obtain all the list of
> submodules. This function acts as the front-end of git-submodule foreach
> subcommand. It calls the function for_each_submodule_list, which basically
> loops through the list and calls function fn, which in this case is
> runcommand_in_submodule. This third function is a calling function that
> takes care of running the command in that submodule, and recursively
> perform the same when --recursive is flagged.
>
> The first function module_foreach first parses the options present in
> argv, and then with the help of module_list_compute, generates the list of
> submodules present in the current working tree.
>
> The second function for_each_submodule_list traverses through the
> list, and calls function fn (which in case of submodule subcommand
> foreach is runcommand_in_submodule) is called for each entry.
>
> The third function runcommand_in_submodule, generates a submodule struct sub
> for $name, value and then later prepends name=sub->name; and other
> value assignment to the env argv_array structure of a child_process.
> Also the <command> of submodule-foreach is push to args argv_array
> structure and finally, using run_command the commands are executed
> using a shell.
>
> The third function also takes care of the recursive flag, by creating
> a separate child_process structure and prepending "--super-prefix displaypath",
> to the args argv_array structure. Other required arguments and the
> input <command> of submodule-foreach is also appended to this argv_array.
>
> Helped-by: Brandon Williams <bmwill@google.com>
> Mentored-by: Christian Couder <christian.couder@gmail.com>
> Mentored-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
> ---
> In this new version, the following changes have been made:
> * Comment style is improved in the function runcommand_in_submodule()
>
> * Comment in added about why the variable "path" was exposed via args
>   argv_array instead of exposing it via the env_array.
>
> * This patch exposes the various variables when argc == 1 only, just
>   for maintaining a faithful porting. You can also find discussion about
>   the same at [1].
>
> [1]: https://public-inbox.org/git/CAME+mvUSGAFbN5j-_hv7QpAS57hq4wgH+yZ7XJMPuyQN1gALaA@mail.gmail.com/

Ah right, maybe add a NEEDSWORK or have the commit message explain
this behavior.

^ permalink raw reply	[relevance 18%]

* [PATCH] convert any hard coded .gitmodules file string to the MACRO
      [irrelevant] <20170718190527.78049-4-bmwill@google.com>
@ 2017-07-31 23:11 ` Stefan Beller
  2017-08-01 13:14   ` Jeff Hostetler
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-31 23:11 UTC (permalink / raw)
  To: bmwill; +Cc: git, gitster, jrnieder, sbeller

I used these commands:
  $ cat sem.cocci
  @@
  @@
  - ".gitmodules"
  + GITMODULES_FILE

  $ spatch --in-place --sp-file sem.cocci builtin/*.c *.c *.h

Feel free to regenerate or squash it in or have it as a separate commit.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c    | 18 +++++++++---------
 unpack-trees.c |  2 +-
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/submodule.c b/submodule.c
index 37f4a92872..b75d02ba7b 100644
--- a/submodule.c
+++ b/submodule.c
@@ -63,7 +63,7 @@ int update_path_in_gitmodules(const char *oldpath, const char *newpath)
 	struct strbuf entry = STRBUF_INIT;
 	const struct submodule *submodule;
 
-	if (!file_exists(".gitmodules")) /* Do nothing without .gitmodules */
+	if (!file_exists(GITMODULES_FILE)) /* Do nothing without .gitmodules */
 		return -1;
 
 	if (gitmodules_is_unmerged)
@@ -77,7 +77,7 @@ int update_path_in_gitmodules(const char *oldpath, const char *newpath)
 	strbuf_addstr(&entry, "submodule.");
 	strbuf_addstr(&entry, submodule->name);
 	strbuf_addstr(&entry, ".path");
-	if (git_config_set_in_file_gently(".gitmodules", entry.buf, newpath) < 0) {
+	if (git_config_set_in_file_gently(GITMODULES_FILE, entry.buf, newpath) < 0) {
 		/* Maybe the user already did that, don't error out here */
 		warning(_("Could not update .gitmodules entry %s"), entry.buf);
 		strbuf_release(&entry);
@@ -97,7 +97,7 @@ int remove_path_from_gitmodules(const char *path)
 	struct strbuf sect = STRBUF_INIT;
 	const struct submodule *submodule;
 
-	if (!file_exists(".gitmodules")) /* Do nothing without .gitmodules */
+	if (!file_exists(GITMODULES_FILE)) /* Do nothing without .gitmodules */
 		return -1;
 
 	if (gitmodules_is_unmerged)
@@ -110,7 +110,7 @@ int remove_path_from_gitmodules(const char *path)
 	}
 	strbuf_addstr(&sect, "submodule.");
 	strbuf_addstr(&sect, submodule->name);
-	if (git_config_rename_section_in_file(".gitmodules", sect.buf, NULL) < 0) {
+	if (git_config_rename_section_in_file(GITMODULES_FILE, sect.buf, NULL) < 0) {
 		/* Maybe the user already did that, don't error out here */
 		warning(_("Could not remove .gitmodules entry for %s"), path);
 		strbuf_release(&sect);
@@ -122,7 +122,7 @@ int remove_path_from_gitmodules(const char *path)
 
 void stage_updated_gitmodules(void)
 {
-	if (add_file_to_cache(".gitmodules", 0))
+	if (add_file_to_cache(GITMODULES_FILE, 0))
 		die(_("staging updated .gitmodules failed"));
 }
 
@@ -233,18 +233,18 @@ void gitmodules_config(void)
 		strbuf_addstr(&gitmodules_path, "/.gitmodules");
 		if (read_cache() < 0)
 			die("index file corrupt");
-		pos = cache_name_pos(".gitmodules", 11);
+		pos = cache_name_pos(GITMODULES_FILE, 11);
 		if (pos < 0) { /* .gitmodules not found or isn't merged */
 			pos = -1 - pos;
 			if (active_nr > pos) {  /* there is a .gitmodules */
 				const struct cache_entry *ce = active_cache[pos];
 				if (ce_namelen(ce) == 11 &&
-				    !memcmp(ce->name, ".gitmodules", 11))
+				    !memcmp(ce->name, GITMODULES_FILE, 11))
 					gitmodules_is_unmerged = 1;
 			}
 		} else if (pos < active_nr) {
 			struct stat st;
-			if (lstat(".gitmodules", &st) == 0 &&
+			if (lstat(GITMODULES_FILE, &st) == 0 &&
 			    ce_match_stat(active_cache[pos], &st, 0) & DATA_CHANGED)
 				gitmodules_is_modified = 1;
 		}
@@ -264,7 +264,7 @@ static int gitmodules_cb(const char *var, const char *value, void *data)
 
 void repo_read_gitmodules(struct repository *repo)
 {
-	char *gitmodules_path = repo_worktree_path(repo, ".gitmodules");
+	char *gitmodules_path = repo_worktree_path(repo, GITMODULES_FILE);
 
 	git_config_from_file(gitmodules_cb, gitmodules_path, repo);
 	free(gitmodules_path);
diff --git a/unpack-trees.c b/unpack-trees.c
index dd535bc849..05335fe5bf 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -286,7 +286,7 @@ static void reload_gitmodules_file(struct index_state *index,
 	for (i = 0; i < index->cache_nr; i++) {
 		struct cache_entry *ce = index->cache[i];
 		if (ce->ce_flags & CE_UPDATE) {
-			int r = strcmp(ce->name, ".gitmodules");
+			int r = strcmp(ce->name, GITMODULES_FILE);
 			if (r < 0)
 				continue;
 			else if (r == 0) {
-- 
2.14.0.rc0.3.g6c2e499285


^ permalink raw reply	[relevance 21%]

* Re: [PATCH v3 07/10] submodule: check for unstaged .gitmodules outside of config parsing
      [irrelevant]   ` <20170718190527.78049-8-bmwill@google.com>
@ 2017-07-31 23:41     ` Stefan Beller
  2017-08-02 17:41       ` Brandon Williams
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-07-31 23:41 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Jonathan Nieder, Junio C Hamano

On Tue, Jul 18, 2017 at 12:05 PM, Brandon Williams <bmwill@google.com> wrote:
> Teach 'is_staging_gitmodules_ok()' to be able to determine in the
> '.gitmodules' file has unstaged changes based on the passed in index
> instead of relying on a global varible which is set during the

variable

> submodule-config parsing.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
>  builtin/mv.c |  2 +-
>  builtin/rm.c |  2 +-
>  submodule.c  | 32 +++++++++++++++++---------------
>  submodule.h  |  2 +-
>  4 files changed, 20 insertions(+), 18 deletions(-)
>
> diff --git a/builtin/mv.c b/builtin/mv.c
> index dcf6736b5..94fbaaa5d 100644
> --- a/builtin/mv.c
> +++ b/builtin/mv.c
> @@ -81,7 +81,7 @@ static void prepare_move_submodule(const char *src, int first,
>         struct strbuf submodule_dotgit = STRBUF_INIT;
>         if (!S_ISGITLINK(active_cache[first]->ce_mode))
>                 die(_("Directory %s is in index and no submodule?"), src);
> -       if (!is_staging_gitmodules_ok())
> +       if (!is_staging_gitmodules_ok(&the_index))
>                 die(_("Please stage your changes to .gitmodules or stash them to proceed"));
>         strbuf_addf(&submodule_dotgit, "%s/.git", src);
>         *submodule_gitfile = read_gitfile(submodule_dotgit.buf);
> diff --git a/builtin/rm.c b/builtin/rm.c
> index 52826d137..4057e73fa 100644
> --- a/builtin/rm.c
> +++ b/builtin/rm.c
> @@ -286,7 +286,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
>                 list.entry[list.nr].name = xstrdup(ce->name);
>                 list.entry[list.nr].is_submodule = S_ISGITLINK(ce->ce_mode);
>                 if (list.entry[list.nr++].is_submodule &&
> -                   !is_staging_gitmodules_ok())
> +                   !is_staging_gitmodules_ok(&the_index))
>                         die (_("Please stage your changes to .gitmodules or stash them to proceed"));
>         }
>
> diff --git a/submodule.c b/submodule.c
> index b1965290f..46ec04d7c 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -37,18 +37,25 @@ static struct oid_array ref_tips_after_fetch;
>  static int gitmodules_is_unmerged;
>
>  /*
> - * This flag is set if the .gitmodules file had unstaged modifications on
> - * startup. This must be checked before allowing modifications to the
> - * .gitmodules file with the intention to stage them later, because when
> - * continuing we would stage the modifications the user didn't stage herself
> - * too. That might change in a future version when we learn to stage the
> - * changes we do ourselves without staging any previous modifications.
> + * Check if the .gitmodules file has unstaged modifications.  This must be
> + * checked before allowing modifications to the .gitmodules file with the
> + * intention to stage them later, because when continuing we would stage the
> + * modifications the user didn't stage herself too. That might change in a
> + * future version when we learn to stage the changes we do ourselves without
> + * staging any previous modifications.
>   */
> -static int gitmodules_is_modified;
> -
> -int is_staging_gitmodules_ok(void)
> +int is_staging_gitmodules_ok(const struct index_state *istate)
>  {
> -       return !gitmodules_is_modified;
> +       int pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE));
> +
> +       if ((pos >= 0) && (pos < istate->cache_nr)) {

Why do we need the second check (pos < istate->cache_nr) ?

I would have assumed the first one suffices,
it might read better if turned around:


    if (pos < 0)
        return 1;

    return (lstat(GITMODULES_FILE, &st) == 0 &&
        ce_match_stat(istate->cache[pos], &st, 0) & DATA_CHANGED);
  }

> @@ -231,11 +238,6 @@ void gitmodules_config(void)
>                                     !memcmp(ce->name, ".gitmodules", 11))
>                                         gitmodules_is_unmerged = 1;
>                         }
> -               } else if (pos < active_nr) {
> -                       struct stat st;
> -                       if (lstat(".gitmodules", &st) == 0 &&
> -                           ce_match_stat(active_cache[pos], &st, 0) & DATA_CHANGED)
> -                               gitmodules_is_modified = 1;
>                 }

So this is where the check "pos < active_nr" is coming from,
introduced in 5fee995244 (submodule.c: add .gitmodules staging
helper functions, 2013-07-30) as well as d4e98b581b (Submodules:
Don't parse .gitmodules when it contains, merge conflicts, 2011-05-14).

If I am reading the docs for cache_name_pos correctly, we would
not need to check for the index exceeding active_cache,
but checking for the index not being out of bounds seems
to be wide spread.

^ permalink raw reply	[relevance 24%]

* Re: [PATCH] convert any hard coded .gitmodules file string to the MACRO
  2017-07-31 23:11 ` [PATCH] convert any hard coded .gitmodules file string to the MACRO Stefan Beller
@ 2017-08-01 13:14   ` Jeff Hostetler
  2017-08-01 17:35     ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Jeff Hostetler @ 2017-08-01 13:14 UTC (permalink / raw)
  To: Stefan Beller, bmwill; +Cc: git, gitster, jrnieder



On 7/31/2017 7:11 PM, Stefan Beller wrote:
> I used these commands:
>    $ cat sem.cocci
>    @@
>    @@
>    - ".gitmodules"
>    + GITMODULES_FILE
> 
>    $ spatch --in-place --sp-file sem.cocci builtin/*.c *.c *.h
> 
> Feel free to regenerate or squash it in or have it as a separate commit.
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>   submodule.c    | 18 +++++++++---------
>   unpack-trees.c |  2 +-
>   2 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/submodule.c b/submodule.c
> index 37f4a92872..b75d02ba7b 100644
> --- a/submodule.c
> +++ b/submodule.c
>   
> @@ -233,18 +233,18 @@ void gitmodules_config(void)
>   		strbuf_addstr(&gitmodules_path, "/.gitmodules");

Did you mean to also change "/.gitmodules" ??

>   		if (read_cache() < 0)
>   			die("index file corrupt");
> -		pos = cache_name_pos(".gitmodules", 11);
> +		pos = cache_name_pos(GITMODULES_FILE, 11);
>   		if (pos < 0) { /* .gitmodules not found or isn't merged */
>   			pos = -1 - pos;
>   			if (active_nr > pos) {  /* there is a .gitmodules */

It might also be nice to change the literals in the comments to
use the macro.

Jeff


^ permalink raw reply	[relevance 8%]

* Re: [PATCH] convert any hard coded .gitmodules file string to the MACRO
  2017-08-01 13:14   ` Jeff Hostetler
@ 2017-08-01 17:35     ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-01 17:35 UTC (permalink / raw)
  To: Jeff Hostetler; +Cc: Brandon Williams, git, Junio C Hamano, Jonathan Nieder

On Tue, Aug 1, 2017 at 6:14 AM, Jeff Hostetler <git@jeffhostetler.com> wrote:
>
>
> On 7/31/2017 7:11 PM, Stefan Beller wrote:
>>
>> I used these commands:
>>    $ cat sem.cocci
>>    @@
>>    @@
>>    - ".gitmodules"
>>    + GITMODULES_FILE
>>
>>    $ spatch --in-place --sp-file sem.cocci builtin/*.c *.c *.h
>>
>> Feel free to regenerate or squash it in or have it as a separate commit.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>   submodule.c    | 18 +++++++++---------
>>   unpack-trees.c |  2 +-
>>   2 files changed, 10 insertions(+), 10 deletions(-)
>>
>> diff --git a/submodule.c b/submodule.c
>> index 37f4a92872..b75d02ba7b 100644
>> --- a/submodule.c
>> +++ b/submodule.c
>>   @@ -233,18 +233,18 @@ void gitmodules_config(void)
>>                 strbuf_addstr(&gitmodules_path, "/.gitmodules");
>
>
> Did you mean to also change "/.gitmodules" ??

Goog point. We should pick that up as well. However as we do not have
a macro for that, we'd have to have 2 calls to strbuf API

    strbuf_addch(&sb, '/');
    strbuf_addstr(&sb, GITMODULES);

>
>>                 if (read_cache() < 0)
>>                         die("index file corrupt");
>> -               pos = cache_name_pos(".gitmodules", 11);
>> +               pos = cache_name_pos(GITMODULES_FILE, 11);
>>                 if (pos < 0) { /* .gitmodules not found or isn't merged */
>>                         pos = -1 - pos;
>>                         if (active_nr > pos) {  /* there is a .gitmodules
>> */
>
>
> It might also be nice to change the literals in the comments to
> use the macro.

Yes, I wondered if sed would have been better for this job.

>
> Jeff
>

^ permalink raw reply	[relevance 8%]

* Re: [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' from shell to C
  2017-07-31 21:12   ` Stefan Beller
@ 2017-08-01 21:14     ` Prathamesh Chavan
  0 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-01 21:14 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Christian Couder, Brandon Williams

On Tue, Aug 1, 2017 at 2:42 AM, Stefan Beller <sbeller@google.com> wrote:
> On Mon, Jul 31, 2017 at 1:56 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
>> This aims to make git-submodule 'status' a built-in. Hence, the function
>> cmd_status() is ported from shell to C. This is done by introducing
>> three functions: module_status(), submodule_status() and print_status().
>>
>> The function module_status() acts as the front-end of the subcommand.
>> It parses subcommand's options and then calls the function
>> module_list_compute() for computing the list of submodules. Then
>> this functions calls for_each_submodule_list() looping through the
>> list obtained.
>>
>> Then for_each_submodule_list() calls submodule_status() for each of the
>> submodule in its list. The function submodule_status() is responsible
>> for generating the status each submodule it is called for, and
>> then calls print_status().
>>
>> Finally, the function print_status() handles the printing of submodule's
>> status.
>>
>> Mentored-by: Christian Couder <christian.couder@gmail.com>
>> Mentored-by: Stefan Beller <sbeller@google.com>
>> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
>> ---
>> In this new version, the following changes have been made:
>> * parameters passed to the function print_status() have been changed.
>>   Instead of passing char *sub_sha1, instead the object_id is being passed.
>>
>> * Also, since the passed parameter displaypath's value isn't changed
>>   by the function, it is passed to the funcition as const char *displaypath
>>   instead of char *displaypath.
>>
>> * the output type of the function handle_submodule_head_ref() is changed
>>   from strbuf to object_id, as we will use the object_id instead of the
>>   hex of sha1 being stored in a struct strbuf.
>>
>> * diff_files_args is cleared after using it by passing it as args in the
>>   function cmd_diff_files.
>>
>> * In the function status_submodule(), for checking if a submodule has merge
>>   conflicts, the patch currently checks if the value of any of the ce_flags
>>   is non-zero. Currently, I think the we aren't interested in a partiular flag,
>>   but I'm not sure on this.
>>
>> * Debugging leftovers and suprious new-lines are removed.
>>
>> * The confusion with displaypath being passed as te super-prefix in many
>>   of the ported subcommands may be a result of the fact that the
>>   function generating the displaypath: get_submodule_displaypath()
>>   uses the super-prefix as simply a path concatenated with the current
>>   submodule name to denote our current location.
>>   The function get_super_prefix() is declared in cache.h and defined in
>>   environment.c, but is majorly used in the builtin/submodule--helper.c
>>   and also in unpack-trees.c
>>   Also, for generating any submodule's displaypath, it would be important to
>>   have ".." passed to the submodule, and currently it is possible only via the
>>   super-prefix.
>>   This is also other instaces where the super-prefix contained ".." as well.
>>   One of such instance is Test 4 from t7406-submodule-update.sh
>>   Hence, maybe documenting the value of displaypath might a solution
>>   for the above problem.
>>   I'm just stating my views and would like to recieve your opinion on this
>>   matter.
>
> Yes, I agree that the display path is not quite easily understood as it can be
> ambiguous.  I am confused by this paragraph:
> * does test 4 from 7406 fail here, or was it just the starting point
>   of the discussion and it all works fine?

Sorry for misleading there. In the discussion earlier, Brandon mentioned that
displaypath's value may contain ".." as well.
To this, I replied pointing out one of the test which checks for the value of
displaypath.
In this test, ( Test 4 from t7406-submodule-update.sh), the value of displaypath
is being calculated via the submodule_init() function present in
submodule--helper.c
Here, I wanted to point out that, even in the case of this function,
the variable
displaypath is evaluated after receiving the value of "../super/" as
the value returned
by the function get_super_prefix().
Hence, to correct this confusion about the usage of super-prefix,
(which as pointed out by Brandon has been traditionally defined
as the path from the root of the superproject down to the root of
the submodule which further means that there should not ever be
any relative '..'s.), we may either have to get the value of super-prefix's
Documentation to accomodate this change, and to accept the way it is
used in submodule--helper and the current patch series, or else,
change the way it is being used in the various function, and used
another method for evaluation of displaypath.

Thanks,
Prathamesh Chavan

^ permalink raw reply	[relevance 26%]

* Re: [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' from shell to C
  2017-07-31 21:42   ` Stefan Beller
@ 2017-08-01 21:19     ` Prathamesh Chavan
  0 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-01 21:19 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Christian Couder

On Tue, Aug 1, 2017 at 3:12 AM, Stefan Beller <sbeller@google.com> wrote:
> On Mon, Jul 31, 2017 at 1:56 PM, Prathamesh Chavan <pc44800@gmail.com> wrote:
>> The same mechanism is used even for porting this submodule
>> subcommand, as used in the ported subcommands till now.
>> The function cmd_deinit in split up after porting into three
>> functions: module_deinit(), for_each_submodule_list() and
>> deinit_submodule().
>>
>> Mentored-by: Christian Couder <christian.couder@gmail.com>
>> Mentored-by: Stefan Beller <sbeller@google.com>
>> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
>> ---
>> In this new version, the following changes have been made:
>> * In the function deinit_submodule, since the test is_git_directory()
>>   adds an additional condition, instead is_directory() is used to check
>>   if "sm_path/.git" is a directory.
>
> Thanks for writing these patches.
> I wonder if (some of) these notes are best put into the code
> as a comment such as
>
>     /* NEEDSWORK: convert to is_submodule_active */
>
> such that people reading this code later realize that checking
> for a directory may not be the "correct" thing, but a thing which
> was easy to express using shell.
>
>> +struct deinit_cb {
>> +       const char *prefix;
>> +       unsigned int quiet: 1;
>> +       unsigned int force: 1;
>> +       unsigned int all: 1;
>
> The value 'all' seems to be unused, i.e. we assign it but never read it?
>
>> +};
>> +#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
>> +
>> +static void deinit_submodule(const struct cache_entry *list_item,
>> +                            void *cb_data)
>> +{
>> +       struct deinit_cb *info = cb_data;
>> +       const struct submodule *sub;
>> +       char *displaypath = NULL;
>> +       struct child_process cp_config = CHILD_PROCESS_INIT;
>> +       struct strbuf sb_config = STRBUF_INIT;
>> +       char *sub_git_dir = xstrfmt("%s/.git", list_item->name);
>> +       mode_t mode = 0777;
>> +
>> +       sub = submodule_from_path(null_sha1, list_item->name);
>> +
>> +       if (!sub || !sub->name)
>> +               goto cleanup;
>> +
>> +       displaypath = get_submodule_displaypath(list_item->name, info->prefix);
>> +
>> +       /* remove the submodule work tree (unless the user already did it) */
>> +       if (is_directory(list_item->name)) {
>> +               struct stat st;
>> +               /* protect submodules containing a .git directory */
>
> Here may a good place to put:
>   /* NEEDSWORK: automatically call absorbgitdirs before warning/die. */
> (It was not in the shell version, so feel free to ignore)
>
>> +               if (!info->force) {
>> +                       struct child_process cp_rm = CHILD_PROCESS_INIT;
>> +                       cp_rm.git_cmd = 1;
>> +                       argv_array_pushl(&cp_rm.args, "rm", "-qn",
>> +                                        list_item->name, NULL);
>
> A bug that exists in the shell version as well as here:
> What if the submodule has the name '--cached', which happens
> to be a valid argument for git-rm?
>
> The call to git-rm would die claiming that the <file> is missing,
> as the file name was miss-interpreted as another flag.
>
> To solve this problem we would insert a '--' after the options,
> before the file name to state that the last argument is a <file>.
>
> Not sure if we want to fix the bug while we're here or if we rather
> want to add
>
>     /* NEEDSWORK: add '--' to confirm <file> argument */
>
IMO, I would first port the subcommand, and add an additional comment
for pointing the bug out. And then later, we may have a bug-fix patch in this
series of patch itself for tackling this bug out.

Thanks,
Prathamesh Chavan

^ permalink raw reply	[relevance 18%]

* Re: [PATCH v3 07/10] submodule: check for unstaged .gitmodules outside of config parsing
  2017-07-31 23:41     ` [PATCH v3 07/10] submodule: check for unstaged .gitmodules outside of config parsing Stefan Beller
@ 2017-08-02 17:41       ` Brandon Williams
  2017-08-02 18:00         ` Brandon Williams
  0 siblings, 1 reply; 200+ results
From: Brandon Williams @ 2017-08-02 17:41 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Jonathan Nieder, Junio C Hamano

On 07/31, Stefan Beller wrote:
> On Tue, Jul 18, 2017 at 12:05 PM, Brandon Williams <bmwill@google.com> wrote:
> > Teach 'is_staging_gitmodules_ok()' to be able to determine in the
> > '.gitmodules' file has unstaged changes based on the passed in index
> > instead of relying on a global varible which is set during the
> 
> variable
> 

Will change.

> > submodule-config parsing.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> > ---
> >  builtin/mv.c |  2 +-
> >  builtin/rm.c |  2 +-
> >  submodule.c  | 32 +++++++++++++++++---------------
> >  submodule.h  |  2 +-
> >  4 files changed, 20 insertions(+), 18 deletions(-)
> >
> > diff --git a/builtin/mv.c b/builtin/mv.c
> > index dcf6736b5..94fbaaa5d 100644
> > --- a/builtin/mv.c
> > +++ b/builtin/mv.c
> > @@ -81,7 +81,7 @@ static void prepare_move_submodule(const char *src, int first,
> >         struct strbuf submodule_dotgit = STRBUF_INIT;
> >         if (!S_ISGITLINK(active_cache[first]->ce_mode))
> >                 die(_("Directory %s is in index and no submodule?"), src);
> > -       if (!is_staging_gitmodules_ok())
> > +       if (!is_staging_gitmodules_ok(&the_index))
> >                 die(_("Please stage your changes to .gitmodules or stash them to proceed"));
> >         strbuf_addf(&submodule_dotgit, "%s/.git", src);
> >         *submodule_gitfile = read_gitfile(submodule_dotgit.buf);
> > diff --git a/builtin/rm.c b/builtin/rm.c
> > index 52826d137..4057e73fa 100644
> > --- a/builtin/rm.c
> > +++ b/builtin/rm.c
> > @@ -286,7 +286,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
> >                 list.entry[list.nr].name = xstrdup(ce->name);
> >                 list.entry[list.nr].is_submodule = S_ISGITLINK(ce->ce_mode);
> >                 if (list.entry[list.nr++].is_submodule &&
> > -                   !is_staging_gitmodules_ok())
> > +                   !is_staging_gitmodules_ok(&the_index))
> >                         die (_("Please stage your changes to .gitmodules or stash them to proceed"));
> >         }
> >
> > diff --git a/submodule.c b/submodule.c
> > index b1965290f..46ec04d7c 100644
> > --- a/submodule.c
> > +++ b/submodule.c
> > @@ -37,18 +37,25 @@ static struct oid_array ref_tips_after_fetch;
> >  static int gitmodules_is_unmerged;
> >
> >  /*
> > - * This flag is set if the .gitmodules file had unstaged modifications on
> > - * startup. This must be checked before allowing modifications to the
> > - * .gitmodules file with the intention to stage them later, because when
> > - * continuing we would stage the modifications the user didn't stage herself
> > - * too. That might change in a future version when we learn to stage the
> > - * changes we do ourselves without staging any previous modifications.
> > + * Check if the .gitmodules file has unstaged modifications.  This must be
> > + * checked before allowing modifications to the .gitmodules file with the
> > + * intention to stage them later, because when continuing we would stage the
> > + * modifications the user didn't stage herself too. That might change in a
> > + * future version when we learn to stage the changes we do ourselves without
> > + * staging any previous modifications.
> >   */
> > -static int gitmodules_is_modified;
> > -
> > -int is_staging_gitmodules_ok(void)
> > +int is_staging_gitmodules_ok(const struct index_state *istate)
> >  {
> > -       return !gitmodules_is_modified;
> > +       int pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE));
> > +
> > +       if ((pos >= 0) && (pos < istate->cache_nr)) {
> 
> Why do we need the second check (pos < istate->cache_nr) ?
> 
> I would have assumed the first one suffices,
> it might read better if turned around:
> 
> 
>     if (pos < 0)
>         return 1;
> 
>     return (lstat(GITMODULES_FILE, &st) == 0 &&
>         ce_match_stat(istate->cache[pos], &st, 0) & DATA_CHANGED);
>   }
> 
> > @@ -231,11 +238,6 @@ void gitmodules_config(void)
> >                                     !memcmp(ce->name, ".gitmodules", 11))
> >                                         gitmodules_is_unmerged = 1;
> >                         }
> > -               } else if (pos < active_nr) {
> > -                       struct stat st;
> > -                       if (lstat(".gitmodules", &st) == 0 &&
> > -                           ce_match_stat(active_cache[pos], &st, 0) & DATA_CHANGED)
> > -                               gitmodules_is_modified = 1;
> >                 }
> 
> So this is where the check "pos < active_nr" is coming from,
> introduced in 5fee995244 (submodule.c: add .gitmodules staging
> helper functions, 2013-07-30) as well as d4e98b581b (Submodules:
> Don't parse .gitmodules when it contains, merge conflicts, 2011-05-14).
> 
> If I am reading the docs for cache_name_pos correctly, we would
> not need to check for the index exceeding active_cache,
> but checking for the index not being out of bounds seems
> to be wide spread.

I can drop the pos < active_nr requirement.

-- 
Brandon Williams

^ permalink raw reply	[relevance 16%]

* Re: [PATCH v3 07/10] submodule: check for unstaged .gitmodules outside of config parsing
  2017-08-02 17:41       ` Brandon Williams
@ 2017-08-02 18:00         ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-08-02 18:00 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Jonathan Nieder, Junio C Hamano

On 08/02, Brandon Williams wrote:
> On 07/31, Stefan Beller wrote:
> > 
> > So this is where the check "pos < active_nr" is coming from,
> > introduced in 5fee995244 (submodule.c: add .gitmodules staging
> > helper functions, 2013-07-30) as well as d4e98b581b (Submodules:
> > Don't parse .gitmodules when it contains, merge conflicts, 2011-05-14).
> > 
> > If I am reading the docs for cache_name_pos correctly, we would
> > not need to check for the index exceeding active_cache,
> > but checking for the index not being out of bounds seems
> > to be wide spread.
> 
> I can drop the pos < active_nr requirement.

From our conversation offline i'll leave this just in case there is some
subtle reason why it exists.  Also makes it more of a 1:1 conversion.

-- 
Brandon Williams

^ permalink raw reply	[relevance 7%]

* [PATCH v4 03/10] cache.h: add GITMODULES_FILE macro
      [irrelevant]   ` <20170802194923.88239-1-bmwill@google.com>
@ 2017-08-02 19:49     ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2017-08-02 19:49 UTC (permalink / raw)
  To: git; +Cc: sbeller, jrnieder, gitster, Brandon Williams

Add a macro to be used when specifying the '.gitmodules' file and
convert any existing hard coded '.gitmodules' file strings to use the
new macro.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h        |  1 +
 submodule.c    | 20 ++++++++++----------
 unpack-trees.c |  2 +-
 3 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/cache.h b/cache.h
index 71fe09264..d59f767e2 100644
--- a/cache.h
+++ b/cache.h
@@ -433,6 +433,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define GITATTRIBUTES_FILE ".gitattributes"
 #define INFOATTRIBUTES_FILE "info/attributes"
 #define ATTRIBUTE_MACRO_PREFIX "[attr]"
+#define GITMODULES_FILE ".gitmodules"
 #define GIT_NOTES_REF_ENVIRONMENT "GIT_NOTES_REF"
 #define GIT_NOTES_DEFAULT_REF "refs/notes/commits"
 #define GIT_NOTES_DISPLAY_REF_ENVIRONMENT "GIT_NOTES_DISPLAY_REF"
diff --git a/submodule.c b/submodule.c
index 6531c5d60..64ad5c12d 100644
--- a/submodule.c
+++ b/submodule.c
@@ -63,7 +63,7 @@ int update_path_in_gitmodules(const char *oldpath, const char *newpath)
 	struct strbuf entry = STRBUF_INIT;
 	const struct submodule *submodule;
 
-	if (!file_exists(".gitmodules")) /* Do nothing without .gitmodules */
+	if (!file_exists(GITMODULES_FILE)) /* Do nothing without .gitmodules */
 		return -1;
 
 	if (gitmodules_is_unmerged)
@@ -77,7 +77,7 @@ int update_path_in_gitmodules(const char *oldpath, const char *newpath)
 	strbuf_addstr(&entry, "submodule.");
 	strbuf_addstr(&entry, submodule->name);
 	strbuf_addstr(&entry, ".path");
-	if (git_config_set_in_file_gently(".gitmodules", entry.buf, newpath) < 0) {
+	if (git_config_set_in_file_gently(GITMODULES_FILE, entry.buf, newpath) < 0) {
 		/* Maybe the user already did that, don't error out here */
 		warning(_("Could not update .gitmodules entry %s"), entry.buf);
 		strbuf_release(&entry);
@@ -97,7 +97,7 @@ int remove_path_from_gitmodules(const char *path)
 	struct strbuf sect = STRBUF_INIT;
 	const struct submodule *submodule;
 
-	if (!file_exists(".gitmodules")) /* Do nothing without .gitmodules */
+	if (!file_exists(GITMODULES_FILE)) /* Do nothing without .gitmodules */
 		return -1;
 
 	if (gitmodules_is_unmerged)
@@ -110,7 +110,7 @@ int remove_path_from_gitmodules(const char *path)
 	}
 	strbuf_addstr(&sect, "submodule.");
 	strbuf_addstr(&sect, submodule->name);
-	if (git_config_rename_section_in_file(".gitmodules", sect.buf, NULL) < 0) {
+	if (git_config_rename_section_in_file(GITMODULES_FILE, sect.buf, NULL) < 0) {
 		/* Maybe the user already did that, don't error out here */
 		warning(_("Could not remove .gitmodules entry for %s"), path);
 		strbuf_release(&sect);
@@ -122,7 +122,7 @@ int remove_path_from_gitmodules(const char *path)
 
 void stage_updated_gitmodules(void)
 {
-	if (add_file_to_cache(".gitmodules", 0))
+	if (add_file_to_cache(GITMODULES_FILE, 0))
 		die(_("staging updated .gitmodules failed"));
 }
 
@@ -230,21 +230,21 @@ void gitmodules_config(void)
 		struct strbuf gitmodules_path = STRBUF_INIT;
 		int pos;
 		strbuf_addstr(&gitmodules_path, work_tree);
-		strbuf_addstr(&gitmodules_path, "/.gitmodules");
+		strbuf_addstr(&gitmodules_path, "/" GITMODULES_FILE);
 		if (read_cache() < 0)
 			die("index file corrupt");
-		pos = cache_name_pos(".gitmodules", 11);
+		pos = cache_name_pos(GITMODULES_FILE, 11);
 		if (pos < 0) { /* .gitmodules not found or isn't merged */
 			pos = -1 - pos;
 			if (active_nr > pos) {  /* there is a .gitmodules */
 				const struct cache_entry *ce = active_cache[pos];
 				if (ce_namelen(ce) == 11 &&
-				    !memcmp(ce->name, ".gitmodules", 11))
+				    !memcmp(ce->name, GITMODULES_FILE, 11))
 					gitmodules_is_unmerged = 1;
 			}
 		} else if (pos < active_nr) {
 			struct stat st;
-			if (lstat(".gitmodules", &st) == 0 &&
+			if (lstat(GITMODULES_FILE, &st) == 0 &&
 			    ce_match_stat(active_cache[pos], &st, 0) & DATA_CHANGED)
 				gitmodules_is_modified = 1;
 		}
@@ -264,7 +264,7 @@ static int gitmodules_cb(const char *var, const char *value, void *data)
 
 void repo_read_gitmodules(struct repository *repo)
 {
-	char *gitmodules_path = repo_worktree_path(repo, ".gitmodules");
+	char *gitmodules_path = repo_worktree_path(repo, GITMODULES_FILE);
 
 	git_config_from_file(gitmodules_cb, gitmodules_path, repo);
 	free(gitmodules_path);
diff --git a/unpack-trees.c b/unpack-trees.c
index dd535bc84..05335fe5b 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -286,7 +286,7 @@ static void reload_gitmodules_file(struct index_state *index,
 	for (i = 0; i < index->cache_nr; i++) {
 		struct cache_entry *ce = index->cache[i];
 		if (ce->ce_flags & CE_UPDATE) {
-			int r = strcmp(ce->name, ".gitmodules");
+			int r = strcmp(ce->name, GITMODULES_FILE);
 			if (r < 0)
 				continue;
 			else if (r == 0) {
-- 
2.14.0.rc1.383.gd1ce394fe2-goog


^ permalink raw reply	[relevance 15%]

* Re: [RFC PATCH] clone: add clone.recursesubmodules config option
      [irrelevant]           ` <598215C8.4000100@game-point.net>
@ 2017-08-02 20:34             ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-02 20:34 UTC (permalink / raw)
  To: Jeremy Morton; +Cc: Heiko Voigt, Junio C Hamano, Jens Lehmann, Chris Packham, git, mara.kim, Jonathan Nieder

On Wed, Aug 2, 2017 at 11:11 AM, Jeremy Morton <admin@game-point.net> wrote:
> Did this ever get anywhere?  If not why not?  It would be very useful to me
> to be able to clone recursively by default, especially considering you can't
> use 'alias' to override the existing 'clone' command.
>

Note that there is 3c548de378 (Merge branch 'sb/submodule-blanket-recursive',
2017-06-13), which adds recursing into submodules to a couple of commands.

clone is not one of them, because at that time I thought you'd want to select
explicitly at clone time which submodules you want. Unlike most other commands
that can recurse into submodules, clone supports a pathspec for the recurse
parameter, such that you can express a fine grained selection of submodules
that you are interested in.

I wonder if submodule.recurse is set if we'd just want to recurse into
all submodules for clone? That may have negative consequences though
as people may have forgotten that they set that config a long time ago and then
are surprised to get so many submodules.

^ permalink raw reply	[relevance 19%]

* Re: [PATCH v2 02/15] submodule: don't use submodule_from_name
      [irrelevant]   ` <20170803182000.179328-3-bmwill@google.com>
@ 2017-08-03 18:57     ` Stefan Beller
  2017-08-04 21:53       ` Brandon Williams
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-08-03 18:57 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Junio C Hamano, Jonathan Nieder, Jens Lehmann

On Thu, Aug 3, 2017 at 11:19 AM, Brandon Williams <bmwill@google.com> wrote:
> The function 'submodule_from_name()' is being used incorrectly here as a
> submodule path is being used instead of a submodule name.  Since the
> correct function to use with a path to a submodule is already being used
> ('submodule_from_path()') let's remove the call to
> 'submodule_from_name()'.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>

In case a reroll is needed, you could incorperate Jens feedback
stating that 851e18c385 should have done it.

> ---
>  submodule.c | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/submodule.c b/submodule.c
> index 5139b9256..19bd13bb2 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1177,8 +1177,6 @@ static int get_next_submodule(struct child_process *cp,
>                         continue;
>
>                 submodule = submodule_from_path(&null_oid, ce->name);
> -               if (!submodule)
> -                       submodule = submodule_from_name(&null_oid, ce->name);
>
>                 default_argv = "yes";
>                 if (spf->command_line_option == RECURSE_SUBMODULES_DEFAULT) {
> --
> 2.14.0.rc1.383.gd1ce394fe2-goog
>

^ permalink raw reply	[relevance 16%]

* Re: [PATCH v2 08/15] unpack-trees: don't respect submodule.update
      [irrelevant]   ` <20170803182000.179328-9-bmwill@google.com>
@ 2017-08-03 20:26     ` Stefan Beller
      [irrelevant]     ` <xmqqpoccwfpl.fsf@gitster.mtv.corp.google.com>
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-03 20:26 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git, Junio C Hamano, Jonathan Nieder, Jens Lehmann

On Thu, Aug 3, 2017 at 11:19 AM, Brandon Williams <bmwill@google.com> wrote:
> The 'submodule.update' config was historically used and respected by the
> 'submodule update' command because update handled a variety of different
> ways it updated a submodule.  As we begin teaching other commands about
> submodules it makes more sense for the different settings of
> 'submodule.update' to be handled by the individual commands themselves
> (checkout, rebase, merge, etc) so it shouldn't be respected by the
> native checkout command.
>
> Also remove the overlaying of the repository's config (via using
> 'submodule_config()') from the commands which use the unpack-trees
> logic (checkout, read-tree, reset).

That was a mistake that I introduced with the checkout series.

Thanks for fixing it.

^ permalink raw reply	[relevance 16%]

* Re: [PATCH v2 08/15] unpack-trees: don't respect submodule.update
      [irrelevant]     ` <xmqqpoccwfpl.fsf@gitster.mtv.corp.google.com>
@ 2017-08-03 20:43       ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-03 20:43 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Brandon Williams, git, Jonathan Nieder, Jens Lehmann

On Thu, Aug 3, 2017 at 1:37 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Brandon Williams <bmwill@google.com> writes:
>
>> The 'submodule.update' config was historically used and respected by the
>> 'submodule update' command because update handled a variety of different
>> ways it updated a submodule.  As we begin teaching other commands about
>> submodules it makes more sense for the different settings of
>> 'submodule.update' to be handled by the individual commands themselves
>> (checkout, rebase, merge, etc) so it shouldn't be respected by the
>> native checkout command.
>
> Soooo... what's the externally observable effect of this change?  Is
> it something that can be illustrated in a set of new tests?

The illustration can be as follows

    git config submodule.NAME.update none
    git checkout -f --recurse-submodules HEAD
    git status
    # observe dirty submodule, which is
    # not what checkout -f promises

> IOW does this commit by itself want to change the behaviour of
> "submodule update" and existing (indirect) users of unpack-trees?
> Or does it want to keep the documented behaviour of "submodule
> update" while correcting unintended triggering in other (indirect)
> users of unpack-trees of the same machinery that is being removed in
> this patch?

"submodule update" is unaffected, only the recently introduced submodule
awareness of checkout/reset/read-tree are changed.

This option is documented as
    submodule.<name>.update
    The default update procedure for a submodule. This variable is
    populated by git submodule init from the gitmodules(5) file. See
    description of update command in git-submodule(1).

which doesn't indicate that any other command apart from
"submodule update" should respect it.

>
>> -     switch (sub->update_strategy.type) {
>> -     case SM_UPDATE_UNSPECIFIED:
>> -     case SM_UPDATE_CHECKOUT:
>> -             if (submodule_move_head(ce->name, old_id, new_id, flags))
>> -                     return o->gently ? -1 :
>> -                             add_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);
>> -             return 0;
>> -     case SM_UPDATE_NONE:
>> -             return 0;
>> -     case SM_UPDATE_REBASE:
>> -     case SM_UPDATE_MERGE:
>> -     case SM_UPDATE_COMMAND:
>> -     default:
>> -             warning(_("submodule update strategy not supported for submodule '%s'"), ce->name);
>> -             return -1;
>> -     }
>> +     if (submodule_move_head(ce->name, old_id, new_id, flags))
>> +             return o->gently ? -1 :
>> +                                add_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);
>> +     return 0;
>
> With this update, we always behave as if update_strategy.type were
> either left unspecified or explicitly set to checkout.  Other arms
> in this switch (and the other switch too), especially "none", were
> not expecting a call to submodule_move_head() to be made, but now
> the call is unconditional.
>

Yes. This is because each command (reset/checkout) should provide
one expected behavior. It is not that we can configure reset to omit certain
(tracked) files from being reset?

^ permalink raw reply	[relevance 27%]

* Re: [PATCH] clone: teach recursive clones to respect -q
      [irrelevant] <20170803222544.17216-1-bmwill@google.com>
@ 2017-08-03 22:32 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-03 22:32 UTC (permalink / raw)
  To: Brandon Williams; +Cc: git

On Thu, Aug 3, 2017 at 3:25 PM, Brandon Williams <bmwill@google.com> wrote:
> Teach 'git clone --recurse-submodules' to respect the '-q' option by
> passing down the quiet flag to the process which handles cloning of
> submodules.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---

Yay! Thanks for this.
Looks good to me.

^ permalink raw reply	[relevance 8%]

* [ANNOUNCE] Git v2.14.0
@ 2017-08-04 16:54 Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2017-08-04 16:54 UTC (permalink / raw)
  To: git; +Cc: Linux Kernel

The latest feature release Git v2.14.0 is now available at the
usual places.  It is comprised of 727 non-merge commits since
v2.13.0, contributed by 66 people, 18 of which are new faces.

The tarballs are found at:

    https://www.kernel.org/pub/software/scm/git/

when kernel.org mirrors catch up.

The following public repositories all have a copy of the 'v2.14.0'
tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.13.0 are as follows.
Welcome to the Git development community!

  Anthony Sottile, A. Wilcox, Ben Peart, Brian Malehorn, Hugues
  Peccatte, James Clarke, Jeff Smith, Kaartic Sivaraam, Liam
  Beguin, Louis, Phillip Wood, Rikard Falkeborn, Sahil Dua,
  Samuel Lijin, Stephen Kent, Sylvestre Ledru, Tyler Brazier,
  and xiaoqiang zhao.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Adam Dinwoodie, Ævar Arnfjörð Bjarmason, Alejandro R. Sedeño,
  Alexander Shopov, Andreas Heiduk, Beat Bolli, Brandon Williams,
  brian m. carlson, Changwoo Ryu, Christian Couder, David Aguilar,
  David Turner, Dennis Kaarsemaker, Dimitriy Ryazantcev, Eric
  Wong, Hartmut Henkel, Jean-Noel Avila, Jeff Hostetler, Jeff
  King, Jiang Xin, Johannes Schindelin, Johannes Sixt, Jonathan
  Nieder, Jonathan Tan, Jordi Mas, Junio C Hamano, Kyle J. McKay,
  Kyle Meyer, Lars Schneider, Marc Branchaud, Michael Haggerty,
  Miguel Torroja, Mike Hommey, Nguyễn Thái Ngọc Duy, Patrick
  Steinhardt, Peter Krefting, Prathamesh Chavan, Ralf Thielow,
  Ramsay Jones, René Scharfe, Stefan Beller, Štěpán Němec,
  Sven Strickroth, SZEDER Gábor, Thomas Gummerer, Torsten
  Bögershausen, Trần Ngọc Quân, and Ville Skyttä.

----------------------------------------------------------------

Git 2.14 Release Notes
======================

Backward compatibility notes and other notable changes.

 * Use of an empty string as a pathspec element that is used for
   'everything matches' is still warned and Git asks users to use a
   more explicit '.' for that instead.  The hope is that existing
   users will not mind this change, and eventually the warning can be
   turned into a hard error, upgrading the deprecation into removal of
   this (mis)feature.  That is not scheduled to happen in the upcoming
   release (yet).

 * Git now avoids blindly falling back to ".git" when the setup
   sequence said we are _not_ in Git repository.  A corner case that
   happens to work right now may be broken by a call to die("BUG").
   We've tried hard to locate such cases and fixed them, but there
   might still be cases that need to be addressed--bug reports are
   greatly appreciated.

 * The experiment to improve the hunk-boundary selection of textual
   diff output has finished, and the "indent heuristics" has now
   become the default.

 * Git can now be built with PCRE v2 instead of v1 of the PCRE
   library. Replace USE_LIBPCRE=YesPlease with USE_LIBPCRE2=YesPlease
   in existing build scripts to build against the new version.  As the
   upstream PCRE maintainer has abandoned v1 maintenance for all but
   the most critical bug fixes, use of v2 is recommended.


Updates since v2.13
-------------------

UI, Workflows & Features

 * The colors in which "git status --short --branch" showed the names
   of the current branch and its remote-tracking branch are now
   configurable.

 * "git clone" learned the "--no-tags" option not to fetch all tags
   initially, and also set up the tagopt not to follow any tags in
   subsequent fetches.

 * "git archive --format=zip" learned to use zip64 extension when
   necessary to go beyond the 4GB limit.

 * "git reset" learned "--recurse-submodules" option.

 * "git diff --submodule=diff" now recurses into nested submodules.

 * "git repack" learned to accept the --threads=<n> option and pass it
   to pack-objects.

 * "git send-email" learned to run sendemail-validate hook to inspect
   and reject a message before sending it out.

 * There is no good reason why "git fetch $there $sha1" should fail
   when the $sha1 names an object at the tip of an advertised ref,
   even when the other side hasn't enabled allowTipSHA1InWant.

 * The "[includeIf "gitdir:$dir"] path=..." mechanism introduced in
   2.13.0 would canonicalize the path of the gitdir being matched,
   and did not match e.g. "gitdir:~/work/*" against a repo in
   "~/work/main" if "~/work" was a symlink to "/mnt/storage/work".
   Now we match both the resolved canonical path and what "pwd" would
   show. The include will happen if either one matches.

 * The "indent" heuristics is now the default in "diff". The
   diff.indentHeuristic configuration variable can be set to "false"
   for those who do not want it.

 * Many commands learned to pay attention to submodule.recurse
   configuration.

 * The convention for a command line is to follow "git cmdname
   --options" with revisions followed by an optional "--"
   disambiguator and then finally pathspecs.  When "--" is not there,
   we make sure early ones are all interpretable as revs (and do not
   look like paths) and later ones are the other way around.  A
   pathspec with "magic" (e.g. ":/p/a/t/h" that matches p/a/t/h from
   the top-level of the working tree, no matter what subdirectory you
   are working from) are conservatively judged as "not a path", which
   required disambiguation more often.  The command line parser
   learned to say "it's a pathspec" a bit more often when the syntax
   looks like so.

 * Update "perl-compatible regular expression" support to enable JIT
   and also allow linking with the newer PCRE v2 library.

 * "filter-branch" learned a pseudo filter "--setup" that can be used
   to define common functions/variables that can be used by other
   filters.

 * Using "git add d/i/r" when d/i/r is the top of the working tree of
   a separate repository would create a gitlink in the index, which
   would appear as a not-quite-initialized submodule to others.  We
   learned to give warnings when this happens.

 * "git status" learned to optionally give how many stash entries there
   are in its output.

 * "git status" has long shown essentially the same message as "git
   commit"; the message it gives while preparing for the root commit,
   i.e. "Initial commit", was hard to understand for some new users.
   Now it says "No commits yet" to stress more on the current status
   (rather than the commit the user is preparing for, which is more in
   line with the focus of "git commit").

 * "git send-email" now has --batch-size and --relogin-delay options
    which can be used to overcome limitations on SMTP servers that
    restrict on how many of e-mails can be sent in a single session.

 * An old message shown in the commit log template was removed, as it
   has outlived its usefulness.

 * "git pull --rebase --recurse-submodules" learns to rebase the
   branch in the submodules to an updated base.

 * "git log" learned -P as a synonym for --perl-regexp, "git grep"
   already had such a synonym.

 * "git log" didn't understand --regexp-ignore-case when combined with
   --perl-regexp. This has been fixed.

Performance, Internal Implementation, Development Support etc.

 * The default packed-git limit value has been raised on larger
   platforms to save "git fetch" from a (recoverable) failure while
   "gc" is running in parallel.

 * Code to update the cache-tree has been tightened so that we won't
   accidentally write out any 0{40} entry in the tree object.

 * Attempt to allow us notice "fishy" situation where we fail to
   remove the temporary directory used during the test.

 * Travis CI gained a task to format the documentation with both
   AsciiDoc and AsciiDoctor.

 * Some platforms have ulong that is smaller than time_t, and our
   historical use of ulong for timestamp would mean they cannot
   represent some timestamp that the platform allows.  Invent a
   separate and dedicated timestamp_t (so that we can distingiuish
   timestamps and a vanilla ulongs, which along is already a good
   move), and then declare uintmax_t is the type to be used as the
   timestamp_t.

 * We can trigger Windows auto-build tester (credits: Dscho &
   Microsoft) from our existing Travis CI tester now.

 * Conversion from uchar[20] to struct object_id continues.

 * Simplify parse_pathspec() codepath and stop it from looking at the
   default in-core index.

 * Add perf-test for wildmatch.

 * Code from "conversion using external process" codepath has been
   extracted to a separate sub-process.[ch] module.

 * When "git checkout", "git merge", etc. manipulates the in-core
   index, various pieces of information in the index extensions are
   discarded from the original state, as it is usually not the case
   that they are kept up-to-date and in-sync with the operation on the
   main index.  The untracked cache extension is copied across these
   operations now, which would speed up "git status" (as long as the
   cache is properly invalidated).

 * The internal implementation of "git grep" has seen some clean-up.

 * Update the C style recommendation for notes for translators, as
   recent versions of gettext tools can work with our style of
   multi-line comments.

 * The implementation of "ref" API around the "packed refs" have been
   cleaned up, in preparation for further changes.

 * The internal logic used in "git blame" has been libified to make it
   easier to use by cgit.

 * Our code often opens a path to an optional file, to work on its
   contents when we can successfully open it.  We can ignore a failure
   to open if such an optional file does not exist, but we do want to
   report a failure in opening for other reasons (e.g. we got an I/O
   error, or the file is there, but we lack the permission to open).

   The exact errors we need to ignore are ENOENT (obviously) and
   ENOTDIR (less obvious).  Instead of repeating comparison of errno
   with these two constants, introduce a helper function to do so.

 * We often try to open a file for reading whose existence is
   optional, and silently ignore errors from open/fopen; report such
   errors if they are not due to missing files.

 * When an existing repository is used for t/perf testing, we first
   create bit-for-bit copy of it, which may grab a transient state of
   the repository and freeze it into the repository used for testing,
   which then may cause Git operations to fail.  Single out "the index
   being locked" case and forcibly drop the lock from the copy.

 * Three instances of the same helper function have been consolidated
   to one.

 * "fast-import" uses a default pack chain depth that is consistent
   with other parts of the system.

 * A new test to show the interaction between the pattern [^a-z]
   (which matches '/') and a slash in a path has been added.  The
   pattern should not match the slash with "pathmatch", but should
   with "wildmatch".

 * The 'diff-highlight' program (in contrib/) has been restructured
   for easier reuse by an external project 'diff-so-fancy'.

 * A common pattern to free a piece of memory and assign NULL to the
   pointer that used to point at it has been replaced with a new
   FREE_AND_NULL() macro.

 * Traditionally, the default die() routine had a code to prevent it
   from getting called multiple times, which interacted badly when a
   threaded program used it (one downside is that the real error may
   be hidden and instead the only error message given to the user may
   end up being "die recursion detected", which is not very useful).

 * Introduce a "repository" object to eventually make it easier to
   work in multiple repositories (the primary focus is to work with
   the superproject and its submodules) in a single process.

 * Optimize "what are the object names already taken in an alternate
   object database?" query that is used to derive the length of prefix
   an object name is uniquely abbreviated to.

 * The hashmap API has been updated so that data to customize the
   behaviour of the comparison function can be specified at the time a
   hashmap is initialized.

 * The "collision detecting" SHA-1 implementation shipped with 2.13 is
   now integrated into git.git as a submodule (the first submodule to
   ship with git.git). Clone git.git with --recurse-submodules to get
   it. For now a non-submodule copy of the same code is also shipped
   as part of the tree.

 * A recent update made it easier to use "-fsanitize=" option while
   compiling but supported only one sanitize option.  Allow more than
   one to be combined, joined with a comma, like "make SANITIZE=foo,bar".

 * Use "p4 -G" to make "p4 changes" output more Python-friendly
   to parse.

 * We started using "%" PRItime, imitating "%" PRIuMAX and friends, as
   a way to format the internal timestamp value, but this does not
   play well with gettext(1) i18n framework, and causes "make pot"
   that is run by the l10n coordinator to create a broken po/git.pot
   file.  This is a possible workaround for that problem.

 * It turns out that Cygwin also needs the fopen() wrapper that
   returns failure when a directory is opened for reading.

Also contains various documentation updates and code clean-ups.


Fixes since v2.13
-----------------

Unless otherwise noted, all the fixes since v2.13 in the maintenance
track are contained in this release (see the maintenance releases'
notes for details).

 * "git gc" did not interact well with "git worktree"-managed
   per-worktree refs.

 * "git cherry-pick" and other uses of the sequencer machinery
   mishandled a trailer block whose last line is an incomplete line.
   This has been fixed so that an additional sign-off etc. are added
   after completing the existing incomplete line.

 * The codepath in "git am" that is used when running "git rebase"
   leaked memory held for the log message of the commits being rebased.

 * "git clone --config var=val" is a way to populate the
   per-repository configuration file of the new repository, but it did
   not work well when val is an empty string.  This has been fixed.

 * Setting "log.decorate=false" in the configuration file did not take
   effect in v2.13, which has been corrected.

 * A few codepaths in "checkout" and "am" working on an unborn branch
   tried to access an uninitialized piece of memory.

 * The Web interface to gmane news archive is long gone, even though
   the articles are still accessible via NTTP.  Replace the links with
   ones to public-inbox.org.  Because their message identification is
   based on the actual message-id, it is likely that it will be easier
   to migrate away from it if/when necessary.

 * The receive-pack program now makes sure that the push certificate
   records the same set of push options used for pushing.

 * Tests have been updated to pass under GETTEXT_POISON (a mechanism
   to ensure that output strings that should not be translated are
   not translated by mistake), and TravisCI is told to run them.

 * "git checkout --recurse-submodules" did not quite work with a
   submodule that itself has submodules.

 * "pack-objects" can stream a slice of an existing packfile out when
   the pack bitmap can tell that the reachable objects are all needed
   in the output, without inspecting individual objects.  This
   strategy however would not work well when "--local" and other
   options are in use, and need to be disabled.

 * Fix memory leaks pointed out by Coverity (and people).

 * "git read-tree -m" (no tree-ish) gave a nonsense suggestion "use
   --empty if you want to clear the index".  With "-m", such a request
   will still fail anyway, as you'd need to name at least one tree-ish
   to be merged.

 * Make sure our tests would pass when the sources are checked out
   with "platform native" line ending convention by default on
   Windows.  Some "text" files out tests use and the test scripts
   themselves that are meant to be run with /bin/sh, ought to be
   checked out with eol=LF even on Windows.

 * Introduce the BUG() macro to improve die("BUG: ...").

 * Clarify documentation for include.path and includeIf.<condition>.path
   configuration variables.

 * Git sometimes gives an advice in a rhetorical question that does
   not require an answer, which can confuse new users and non native
   speakers.  Attempt to rephrase them.

 * A few http:// links that are redirected to https:// in the
   documentation have been updated to https:// links.

 * "git for-each-ref --format=..." with %(HEAD) in the format used to
   resolve the HEAD symref as many times as it had processed refs,
   which was wasteful, and "git branch" shared the same problem.

 * Regression fix to topic recently merged to 'master'.

 * The shell completion script (in contrib/) learned "git stash" has
   a new "push" subcommand.

 * "git interpret-trailers", when used as GIT_EDITOR for "git commit
   -v", looked for and appended to a trailer block at the very end,
   i.e. at the end of the "diff" output.  The command has been
   corrected to pay attention to the cut-mark line "commit -v" adds to
   the buffer---the real trailer block should appear just before it.

 * A test allowed both "git push" and "git receive-pack" on the other
   end write their traces into the same file.  This is OK on platforms
   that allows atomically appending to a file opened with O_APPEND,
   but on other platforms led to a mangled output, causing
   intermittent test failures.  This has been fixed by disabling
   traces from "receive-pack" in the test.

 * Tag objects, which are not reachable from any ref, that point at
   missing objects were mishandled by "git gc" and friends (they
   should silently be ignored instead)

 * "git describe --contains" penalized light-weight tags so much that
   they were almost never considered.  Instead, give them about the
   same chance to be considered as an annotated tag that is the same
   age as the underlying commit would.

 * The "run-command" API implementation has been made more robust
   against dead-locking in a threaded environment.

 * A recent update to t5545-push-options.sh started skipping all the
   tests in the script when a web server testing is disabled or
   unavailable, not just the ones that require a web server.  Non HTTP
   tests have been salvaged to always run in this script.

 * "git send-email" now uses Net::SMTP::SSL, which is obsolete, only
   when needed.  Recent versions of Net::SMTP can do TLS natively.

 * "foo\bar\baz" in "git fetch foo\bar\baz", even though there is no
   slashes in it, cannot be a nickname for a remote on Windows, as
   that is likely to be a pathname on a local filesystem.

 * "git clean -d" used to clean directories that has ignored files,
   even though the command should not lose ignored ones without "-x".
   "git status --ignored"  did not list ignored and untracked files
   without "-uall".  These have been corrected.

 * The result from "git diff" that compares two blobs, e.g. "git diff
   $commit1:$path $commit2:$path", used to be shown with the full
   object name as given on the command line, but it is more natural to
   use the $path in the output and use it to look up .gitattributes.

 * The "collision detecting" SHA-1 implementation shipped with 2.13
   was quite broken on some big-endian platforms and/or platforms that
   do not like unaligned fetches.  Update to the upstream code which
   has already fixed these issues.

 * "git am -h" triggered a BUG().

 * The interaction of "url.*.insteadOf" and custom URL scheme's
   whitelisting is now documented better.

 * The timestamp of the index file is now taken after the file is
   closed, to help Windows, on which a stale timestamp is reported by
   fstat() on a file that is opened for writing and data was written
   but not yet closed.

 * "git pull --rebase --autostash" didn't auto-stash when the local history
   fast-forwards to the upstream.

 * A flaky test has been corrected.

 * "git $cmd -h" for builtin commands calls the implementation of the
   command (i.e. cmd_$cmd() function) without doing any repository
   set-up, and the commands that expect RUN_SETUP is done by the Git
   potty needs to be prepared to show the help text without barfing.
   (merge d691551192 jk/consistent-h later to maint).

 * Help contributors that visit us at GitHub.

 * "git stash push <pathspec>" did not work from a subdirectory at all.
   Bugfix for a topic in v2.13

 * As there is no portable way to pass timezone information to
   strftime, some output format from "git log" and friends are
   impossible to produce.  Teach our own strbuf_addftime to replace %z
   and %Z with caller-supplied values to help working around this.
   (merge 6eced3ec5e rs/strbuf-addftime-zZ later to maint).

 * "git mergetool" learned to work around a wrapper MacOS X adds
   around underlying meld.

 * An example in documentation that does not work in multi worktree
   configuration has been corrected.

 * The pretty-format specifiers like '%h', '%t', etc. had an
   optimization that no longer works correctly.  In preparation/hope
   of getting it correctly implemented, first discard the optimization
   that is broken.

 * The code to pick up and execute command alias definition from the
   configuration used to switch to the top of the working tree and
   then come back when the expanded alias was executed, which was
   unnecessarilyl complex.  Attempt to simplify the logic by using the
   early-config mechanism that does not chdir around.

 * Fix configuration codepath to pay proper attention to commondir
   that is used in multi-worktree situation, and isolate config API
   into its own header file.
   (merge dc8441fdb4 bw/config-h later to maint).

 * "git add -p" were updated in 2.12 timeframe to cope with custom
   core.commentchar but the implementation was buggy and a
   metacharacter like $ and * did not work.

 * A recent regression in "git rebase -i" has been fixed and tests
   that would have caught it and others have been added.

 * An unaligned 32-bit access in pack-bitmap code has been corrected.

 * Tighten error checks for invalid "git apply" input.

 * The split index code did not honor core.sharedRepository setting
   correctly.

 * The Makefile rule in contrib/subtree for building documentation
   learned to honour USE_ASCIIDOCTOR just like the main documentation
   set does.

 * Code clean-up to fix possible buffer over-reading.

 * A few tests that tried to verify the contents of push certificates
   did not use 'git rev-parse' to formulate the line to look for in
   the certificate correctly.

 * Update the character width tables.

 * After "git branch --move" of the currently checked out branch, the
   code to walk the reflog of HEAD via "log -g" and friends
   incorrectly stopped at the reflog entry that records the renaming
   of the branch.

 * The rewrite of "git branch --list" using for-each-ref's internals
   that happened in v2.13 regressed its handling of color.branch.local;
   this has been fixed.

 * The build procedure has been improved to allow building and testing
   Git with address sanitizer more easily.
   (merge 425ca6710b jk/build-with-asan later to maint).

 * On Cygwin, similar to Windows, "git push //server/share/repository"
   ought to mean a repository on a network share that can be accessed
   locally, but this did not work correctly due to stripping the double
   slashes at the beginning.

 * The progress meter did not give a useful output when we haven't had
   0.5 seconds to measure the throughput during the interval.  Instead
   show the overall throughput rate at the end, which is a much more
   useful number.

 * Code clean-up, that makes us in sync with Debian by one patch.

 * We run an early part of "git gc" that deals with refs before
   daemonising (and not under lock) even when running a background
   auto-gc, which caused multiple gc processes attempting to run the
   early part at the same time.  This is now prevented by running the
   early part also under the GC lock.

 * A recent update broke an alias that contained an uppercase letter.

 * Other minor doc, test and build updates and code cleanups.
   (merge 5053313562 rs/urlmatch-cleanup later to maint).
   (merge 42c78a216e rs/use-div-round-up later to maint).
   (merge 5e8d2729ae rs/wt-status-cleanup later to maint).
   (merge bc9b7e207f as/diff-options-grammofix later to maint).
   (merge ac05222b31 ah/patch-id-doc later to maint).

----------------------------------------------------------------

Changes since v2.13.0 are as follows:

A. Wilcox (1):
      subtree: honour USE_ASCIIDOCTOR when set

Adam Dinwoodie (1):
      docs: fix formatting and grammar

Alejandro R. Sedeño (1):
      ref-filter.c: drop return from void function

Alexander Shopov (2):
      l10n: bg.po: Updated Bulgarian translation (3206t)
      l10n: bg.po: Updated Bulgarian translation (3213t)

Andreas Heiduk (11):
      Documentation: fix reference to ifExists for interpret-trailers
      Documentation: fix formatting typo in pretty-formats.txt
      doc: filter-branch does not require re-export of vars
      doc: rewrite description for rev-parse --short
      doc: describe git svn init --ignore-refs
      filter-branch: add `--setup` step
      filter-branch: add [--] to usage
      git-svn: document special options for commit-diff
      doc: do not use `rm .git/index` when normalizing line endings
      doc: clarify syntax for %C(auto,...) in pretty formats
      doc: remove unsupported parameter from patch-id

Anthony Sottile (1):
      diff-options doc: grammar fix

Beat Bolli (1):
      unicode: update the width tables to Unicode 10

Ben Peart (10):
      convert: remove erroneous tests for errno == EPIPE
      pkt-line: fix packet_read_line() to handle len < 0 errors
      pkt-line: add packet_read_line_gently()
      convert: move packet_write_line() into pkt-line as packet_writel()
      convert: split start_multi_file_filter() into two separate functions
      convert: separate generic structures and variables from the filter specific ones
      convert: update generic functions to only use generic data structures
      convert: rename reusable sub-process functions
      sub-process: move sub-process functions into separate files
      convert: update subprocess_read_status() to not die on EOF

Brandon Williams (107):
      t5550: use write_script to generate post-update hook
      t0061: run_command executes scripts without a #! line
      run-command: prepare command before forking
      run-command: use the async-signal-safe execv instead of execvp
      string-list: add string_list_remove function
      run-command: prepare child environment before forking
      run-command: don't die in child when duping /dev/null
      run-command: eliminate calls to error handling functions in child
      run-command: handle dup2 and close errors in child
      run-command: add note about forking and threading
      run-command: expose is_executable function
      run-command: restrict PATH search to executable files
      submodule: rename add_sha1_to_array()
      submodule: rename free_submodules_sha1s()
      submodule: remove add_oid_to_argv()
      submodule: change string_list changed_submodule_paths
      submodule: improve submodule_has_commits()
      submodule: refactor logic to determine changed submodules
      dir: stop using the index compatibility macros
      dir: convert read_skip_worktree_file_from_index to take an index
      dir: convert directory_exists_in_index to take index
      dir: convert get_dtype to take index
      dir: convert dir_add* to take an index
      dir: convert last_exclude_matching_from_list to take an index
      dir: convert is_excluded_from_list to take an index
      dir: convert add_excludes to take an index
      dir: convert prep_exclude to take an index
      dir: convert is_excluded to take an index
      dir: convert open_cached_dir to take an index
      dir: convert read_directory_recursive to take an index
      dir: convert read_directory to take an index
      dir: convert fill_directory to take an index
      pathspec: provide a more descriptive die message
      submodule: add die_in_unpopulated_submodule function
      pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag
      ls-files: prevent prune_cache from overeagerly pruning submodules
      pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP
      pathspec: convert find_pathspecs_matching_against_index to take an index
      grep: convert to struct object_id
      diff: convert get_stat_data to struct object_id
      diff: convert diff_index_show_file to struct object_id
      diff: convert diff_addremove to struct object_id
      diff: convert run_diff_files to struct object_id
      diff: convert diff_change to struct object_id
      diff: convert fill_filespec to struct object_id
      diff: convert reuse_worktree_file to struct object_id
      diff: finish conversion for prepare_temp_file to struct object_id
      patch-ids: convert to struct object_id
      diff: convert diff_flush_patch_id to struct object_id
      combine-diff: convert diff_tree_combined to struct object_id
      combine-diff: convert find_paths_* to struct object_id
      tree-diff: convert diff_root_tree_sha1 to struct object_id
      notes-merge: convert notes_merge* to struct object_id
      notes-merge: convert merge_from_diffs to struct object_id
      notes-merge: convert find_notes_merge_pair_ps to struct object_id
      notes-merge: convert verify_notes_filepair to struct object_id
      notes-merge: convert write_note_to_worktree to struct object_id
      diff-tree: convert diff_tree_sha1 to struct object_id
      builtin/diff-tree: cleanup references to sha1
      tree-diff: convert try_to_follow_renames to struct object_id
      tree-diff: convert diff_tree_paths to struct object_id
      tree-diff: convert path_appendnew to object_id
      diffcore-rename: use is_empty_blob_oid
      diff: rename diff_fill_sha1_info to diff_fill_oid_info
      convert: convert get_cached_convert_stats_ascii to take an index
      convert: convert crlf_to_git to take an index
      convert: convert convert_to_git_filter_fd to take an index
      convert: convert convert_to_git to take an index
      convert: convert renormalize_buffer to take an index
      tree: convert read_tree to take an index parameter
      ls-files: convert overlay_tree_on_cache to take an index
      ls-files: convert write_eolinfo to take an index
      ls-files: convert show_killed_files to take an index
      ls-files: convert show_other_files to take an index
      ls-files: convert show_ru_info to take an index
      ls-files: convert ce_excluded to take an index
      ls-files: convert prune_cache to take an index
      ls-files: convert show_ce_entry to take an index
      ls-files: convert show_files to take an index
      ls-files: factor out debug info into a function
      ls-files: factor out tag calculation
      config: create config.h
      config: remove git_config_iter
      config: don't include config.h by default
      setup: teach discover_git_directory to respect the commondir
      config: respect commondir
      config: don't implicitly use gitdir or commondir
      setup: don't perform lazy initialization of repository state
      setup: add comment indicating a hack
      environment: remove namespace_len variable
      repository: introduce the repository object
      environment: place key repository state in the_repository
      environment: store worktree in the_repository
      path: create path.h
      path: always pass in commondir to update_common_dir
      path: convert strbuf_git_common_path to take a 'struct repository'
      path: convert do_git_path to take a 'struct repository'
      path: worktree_git_path() should not use file relocation
      path: add repo_git_path and strbuf_repo_git_path
      path: add repo_worktree_path and strbuf_repo_worktree_path
      config: read config from a repository object
      repository: add index_state to struct repo
      submodule-config: store the_submodule_cache in the_repository
      submodule: add repo_read_gitmodules
      submodule: convert is_submodule_initialized to work on a repository
      repository: enable initialization of submodules
      ls-files: use repository object

Brian Malehorn (1):
      interpret-trailers: honor the cut line

Changwoo Ryu (2):
      l10n: ko.po: Update Korean translation
      l10n: ko.po: Update Korean translation

Christian Couder (5):
      p3400: add perf tests for rebasing many changes
      sub-process: correct path to API docs in a comment
      read-cache: use shared perms when writing shared index
      t1301: move modebits() to test-lib-functions.sh
      t1700: make sure split-index respects core.sharedrepository

David Aguilar (1):
      mergetools/meld: improve compatibiilty with Meld on macOS X

David Turner (2):
      Increase core.packedGitLimit
      unpack-trees: preserve index extensions

Dennis Kaarsemaker (1):
      send-email: Net::SMTP::SSL is obsolete, use only when necessary

Dimitriy Ryazantcev (2):
      l10n: ru.po: update Russian translation
      l10n: ru.po: update Russian translation

Eric Wong (2):
      run-command: block signals between fork and execve
      set FD_CLOEXEC properly when O_CLOEXEC is not supported

Hartmut Henkel (1):
      l10n: de.po: various fixes in German translation

Hugues Peccatte (1):
      l10n: fr.po Fix some translations

James Clarke (1):
      pack-bitmap: don't perform unaligned memory access

Jean-Noel Avila (4):
      usability: don't ask questions if no reply is required
      read-tree -m: make error message for merging 0 trees less smart aleck
      git-filter-branch: be more direct in an error message
      l10n: fr.po v2.14.0 rnd 2

Jeff Hostetler (1):
      read-cache: close index.lock in do_write_index

Jeff King (78):
      cache-tree: reject entries with null sha1
      am: fix commit buffer leak in get_commit_info()
      am: simplify allocations in get_commit_info()
      am: shorten ident_split variable name in get_commit_info()
      status: fix missing newline when comment chars are disabled
      pack-objects: disable pack reuse for object-selection options
      t5310: fix "; do" style
      add--interactive: drop diff.indentHeuristic handling
      docs/config: clarify include/includeIf relationship
      docs/config: give a relative includeIf example
      docs/config: avoid the term "expand" for includes
      docs/config: consistify include.path examples
      usage.c: add BUG() function
      setup_git_env: convert die("BUG") to BUG()
      config: complain about --local outside of a git repo
      usage.c: drop set_error_handle()
      pkt-line: annotate packet_writel with LAST_ARG_MUST_BE_NULL
      t5400: avoid concurrent writes into a trace file
      ref-filter: resolve HEAD when parsing %(HEAD) atom
      revision.c: ignore broken tags with ignore_missing_links
      ref-filter: limit traversal to prefix
      handle_revision_arg: reset "dotdot" consistently
      handle_revision_arg: simplify commit reference lookups
      handle_revision_arg: stop using "dotdot" as a generic pointer
      handle_revision_arg: hoist ".." check out of range parsing
      handle_revision_arg: add handle_dotdot() helper
      sha1_name: consistently refer to object_context as "oc"
      get_sha1_with_context: always initialize oc->symlink_path
      get_sha1_with_context: dynamically allocate oc->path
      t4063: add tests of direct blob diffs
      handle_revision_arg: record modes for "a..b" endpoints
      handle_revision_arg: record paths for pending objects
      diff: pass whole pending entry in blobinfo
      diff: use the word "path" instead of "name" for blobs
      diff: use pending "path" if it is available
      diff: use blob path for blob/file diffs
      connect.c: fix leak in parse_one_symref_info()
      t4208: add check for ":/" without matching file
      check_filename(): refactor ":/" handling
      check_filename(): use skip_prefix
      check_filename(): handle ":^" path magic
      verify_filename(): treat ":(magic)" as a pathspec
      verify_filename(): flip order of checks
      am: handle "-h" argument earlier
      credential: handle invalid arguments earlier
      upload-archive: handle "-h" option early
      remote-{ext,fd}: print usage message on invalid arguments
      submodule--helper: show usage for "-h"
      remote: drop free_refspecs() function
      docs/config: mention protocol implications of url.insteadOf
      version: convert to parse-options
      git: add hidden --list-builtins option
      t0012: test "-h" with builtins
      t5313: make extended-table test more deterministic
      sha1dc: ignore indent-with-non-tab whitespace violations
      add: warn when adding an embedded repository
      t: move "git add submodule" into test blocks
      diff-highlight: split code into module
      configure.ac: loosen FREAD_READS_DIRECTORIES test program
      t0006: check --date=format zone offsets
      date: use localtime() for "-local" time formats
      docs: update 64-bit core.packedGitLimit default
      add--interactive: handle EOF in prompt_yesno
      add--interactive: quote commentChar regex
      reflog-walk: skip over double-null oid due to HEAD rename
      reflog-walk: duplicate strings in complete_reflogs list
      reflog-walk: don't free reflogs added to cache
      reflog-walk: include all fields when freeing complete_reflogs
      branch: only perform HEAD check for local branches
      branch: use BRANCH_COLOR_LOCAL in ref-filter format
      branch: set remote color in ref-filter branch immediately
      test-lib: set ASAN_OPTIONS variable before we run git
      test-lib: turn on ASan abort_on_error by default
      Makefile: add helper for compiling with -fsanitize
      Makefile: turn off -fomit-frame-pointer with sanitizers
      Makefile: disable unaligned loads with UBSan
      gc: run pre-detach operations under lock
      t: handle EOF in test_copy_bytes()

Jeff Smith (29):
      blame: remove unneeded dependency on blob.h
      blame: move textconv_object with related functions
      blame: remove unused parameters
      blame: rename origin structure to blame_origin
      blame: rename scoreboard structure to blame_scoreboard
      blame: rename origin-related functions
      blame: rename coalesce function
      blame: rename ent_score function
      blame: rename nth_line function
      blame: move stat counters to scoreboard
      blame: move copy/move thresholds to scoreboard
      blame: move contents_from to scoreboard
      blame: move reverse flag to scoreboard
      blame: move show_root flag to scoreboard
      blame: move xdl_opts flags to scoreboard
      blame: move no_whole_file_rename flag to scoreboard
      blame: make sanity_check use a callback in scoreboard
      blame: move progress updates to a scoreboard callback
      blame: wrap blame_sort and compare_blame_final
      blame: rework methods that determine 'final' commit
      blame: create scoreboard init function
      blame: create scoreboard setup function
      blame: create entry prepend function
      blame: move core structures to header
      blame: move origin-related methods to libgit
      blame: move fake-commit-related methods to libgit
      blame: move scoreboard-related methods to libgit
      blame: move scoreboard setup to libgit
      blame: move entry prepend to libgit

Jiang Xin (4):
      l10n: git.pot: v2.14.0 round 1 (34 new, 23 removed)
      l10n: git.pot: v2.14.0 round 2 (9 new, 2 removed)
      l10n: zh_CN: for git v2.14.0 l10n round 2
      l10n: zh_CN: review for git v2.14.0 l10n

Johannes Schindelin (50):
      ref-filter: avoid using `unsigned long` for catch-all data type
      t0006 & t5000: prepare for 64-bit timestamps
      t0006 & t5000: skip "far in the future" test when time_t is too limited
      parse_timestamp(): specify explicitly where we parse timestamps
      PRItime: introduce a new "printf format" for timestamps
      timestamp_t: a new data type for timestamps
      date.c: abort if the system time cannot handle one of our timestamps
      use uintmax_t for timestamps
      mingw: avoid memory leak when splitting PATH
      winansi: avoid use of uninitialized value
      winansi: avoid buffer overrun
      add_commit_patch_id(): avoid allocating memory unnecessarily
      git_config_rename_section_in_file(): avoid resource leak
      get_mail_commit_oid(): avoid resource leak
      difftool: address a couple of resource/memory leaks
      status: close file descriptor after reading git-rebase-todo
      mailinfo & mailsplit: check for EOF while parsing
      cat-file: fix memory leak
      checkout: fix memory leak
      split_commit_in_progress(): simplify & fix memory leak
      setup_bare_git_dir(): help static analysis
      setup_discovered_git_dir(): plug memory leak
      pack-redundant: plug memory leak
      mktree: plug memory leaks reported by Coverity
      fast-export: avoid leaking memory in handle_tag()
      receive-pack: plug memory leak in update()
      line-log: avoid memory leak
      shallow: avoid memory leak
      add_reflog_for_walk: avoid memory leak
      remote: plug memory leak in match_explicit()
      name-rev: avoid leaking memory in the `deref` case
      show_worktree(): plug memory leak
      submodule_uses_worktrees(): plug memory leak
      Fix build with core.autocrlf=true
      git-new-workdir: mark script as LF-only
      completion: mark bash script as LF-only
      t3901: move supporting files into t/t3901/
      Fix the remaining tests that failed with core.autocrlf=true
      t4051: mark supporting files as requiring LF-only line endings
      mingw: verify that paths are not mistaken for remote nicknames
      discover_git_directory(): avoid setting invalid git_dir
      config: report correct line number upon error
      help: use early config when autocorrecting aliases
      t1308: relax the test verifying that empty alias values are disallowed
      t7006: demonstrate a problem with aliases in subdirectories
      alias: use the early config machinery to expand aliases
      sequencer: print autostash messages to stderr
      t5534: fix misleading grep invocation
      t1300: demonstrate that CamelCased aliases regressed
      alias: compare alias name *case-insensitively*

Johannes Sixt (3):
      mingw.h: permit arguments with side effects for is_dir_sep
      Windows: do not treat a path with backslashes as a remote's nick name
      mingw_fopen: report ENOENT for invalid file names

Jonathan Nieder (4):
      credential doc: make multiple-helper behavior more prominent
      clone: handle empty config values in -c
      send-email: Net::SMTP::starttls was introduced in v2.34
      pre-rebase hook: capture documentation in a <<here document

Jonathan Tan (14):
      sequencer: add newline before adding footers
      docs: correct receive.advertisePushOptions default
      receive-pack: verify push options in cert
      fetch-pack: always allow fetching of literal SHA1s
      send-email: support validate hook
      send-email: check for repo before invoking hook
      sha1_file: teach packed_object_info about typename
      sha1_file: rename LOOKUP_UNKNOWN_OBJECT
      sha1_file: rename LOOKUP_REPLACE_OBJECT
      sha1_file: move delta base cache code up
      sha1_file: refactor read_object
      sha1_file: teach sha1_object_info_extended more flags
      sha1_file: do not access pack if unneeded
      sha1_file: refactor has_sha1_file_with_flags

Jordi Mas (3):
      l10n: Fixes to Catalan translation
      l10n: Update Catalan translation
      l10n: Update Catalan translation

Junio C Hamano (54):
      name-rev: refactor logic to see if a new candidate is a better name
      name-rev: favor describing with tags and use committer date to tiebreak
      test-lib.sh: do not barf under --debug at the end of the test
      test-lib: retire $remove_trash variable
      repack: accept --threads=<n> and pass it down to pack-objects
      apply.c: fix whitespace-only mismerge
      checkout: fix memory leak
      doc: replace more gmane links
      read-tree: "read-tree -m --empty" does not make sense
      Start post 2.13 cycle
      test: allow skipping the remainder
      name-rev: change a "long" variable to timestamp_t
      Second batch for 2.14
      config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too
      Third batch for 2.14
      compat-util: is_missing_file_error()
      treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checked
      Fourth batch for 2.14
      diff-tree: update stale in-code comments
      Fifth batch for 2.14
      Sixth batch for 2.14
      Prepare for 2.13.1; more topics to follow
      Git 2.13.1
      Seventh batch for 2.14
      diff- and log- family: handle "git cmd -h" early
      Prepare for 2.13.2
      Eighth batch for 2.14
      Ninth batch for 2.14
      Tenth batch for 2.14
      t3420: fix under GETTEXT_POISON build
      Revert "split-index: add and use unshare_split_index()"
      Eleventh batch for 2.14
      Git 2.13.2
      Twelfth batch for 2.14
      t1450: use egrep for regexp "alternation"
      submodule--helper: do not call utf8_fprintf() unnecessarily
      Thirteenth batch for 2.14
      sha1collisiondetection: automatically enable when submodule is populated
      Fourteenth batch for 2.14
      Fifteenth batch for 2.14
      Sixteenth batch for 2.14
      Prepare for 2.13.3
      Hopefully the last batch before -rc0
      Git 2.13.3
      Git 2.14-rc0
      A few more topics before 2.14-rc1
      Makefile: help gettext tools to cope with our custom PRItime format
      A few more topics while waiting for the po/PRItime resolution
      Hopefully the final last-minute fix before -rc1
      fixes from 'master' for 2.13.4
      Git 2.14-rc1
      Preparation for 2.13.4 continues
      Git 2.13.4
      Git 2.14

Kaartic Sivaraam (9):
      t7508: fix a broken indentation
      status: contextually notify user about an initial commit
      Documentation/git-submodule: cleanup "add" section
      commit-template: remove outdated notice about explicit paths
      commit-template: distinguish status information unconditionally
      builtin/commit.c: fix a typo in the comment
      doc: correct a mistake in an illustration
      doc: camelCase the i18n config variables to improve readability
      doc: reformat the paragraph containing the 'cut-line'

Kyle J. McKay (1):
      t5100: add some more mailinfo tests

Kyle Meyer (1):
      config.txt: add an entry for log.showSignature

Lars Schneider (9):
      travis-ci: build documentation with AsciiDoc and Asciidoctor
      travis-ci: parallelize documentation build
      travis-ci: unset compiler for jobs that do not need one
      travis-ci: check AsciiDoc/AsciiDoctor stderr output
      travis-ci: handle Git for Windows CI status "failed" explicitly
      travis-ci: retry if Git for Windows CI returns HTTP error 502 or 503
      travis-ci: setup "prove cache" in "script" step
      travis-ci: add job to run tests with GETTEXT_POISON
      Configure Git contribution guidelines for github.com

Liam Beguin (3):
      stash: update documentation to use 'stash entry'
      status: add optional stash count information
      glossary: define 'stash entry'

Louis (1):
      l10n: fr.po Fix typo

Marc Branchaud (3):
      diff: make the indent heuristic part of diff's basic configuration
      diff: have the diff-* builtins configure diff before initializing revisions
      auto-correct: tweak phrasing

Michael Haggerty (26):
      t3600: clean up permissions test properly
      refs.h: clarify docstring for the ref_transaction_update()-related fns
      ref_iterator_begin_fn(): fix docstring
      files-backend: use `die("BUG: ...")`, not `die("internal error: ...")`
      prefix_ref_iterator: don't trim too much
      refs_ref_iterator_begin(): don't check prefixes redundantly
      refs: use `size_t` indexes when iterating over ref transaction updates
      ref_store: take a `msg` parameter when deleting references
      lockfile: add a new method, is_lock_file_locked()
      files-backend: move `lock` member to `files_ref_store`
      files_ref_store: put the packed files lock directly in this struct
      files_transaction_cleanup(): new helper function
      ref_transaction_commit(): check for valid `transaction->state`
      ref_transaction_prepare(): new optional step for reference updates
      ref_update_reject_duplicates(): expose function to whole refs module
      ref_update_reject_duplicates(): use `size_t` rather than `int`
      ref_update_reject_duplicates(): add a sanity check
      should_pack_ref(): new function, extracted from `files_pack_refs()`
      get_packed_ref_cache(): assume "packed-refs" won't change while locked
      read_packed_refs(): do more of the work of reading packed refs
      read_packed_refs(): report unexpected fopen() failures
      refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`
      create_ref_entry(): remove `check_name` option
      cache_ref_iterator_begin(): avoid priming unneeded directories
      lock_packed_refs(): fix cache validity check
      for_each_bisect_ref(): don't trim refnames

Miguel Torroja (3):
      git-p4: git-p4 tests with p4 triggers
      git-p4: parse marshal output "p4 -G" in p4 changes
      git-p4: filter for {'code':'info'} in p4CmdList

Mike Hommey (1):
      fast-import: increase the default pack depth to 50

Nguyễn Thái Ngọc Duy (16):
      environment.c: fix potential segfault by get_git_common_dir()
      refs.c: make submodule ref store hashmap generic
      refs: add REFS_STORE_ALL_CAPS
      refs: introduce get_worktree_ref_store()
      worktree.c: kill parse_ref() in favor of refs_resolve_ref_unsafe()
      refs: kill set_worktree_head_symref()
      split-index: add and use unshare_split_index()
      use xfopen() in more places
      clone: use xfopen() instead of fopen()
      config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD
      wrapper.c: add and use warn_on_fopen_errors()
      wrapper.c: add and use fopen_or_warn()
      wrapper.c: make warn_on_inaccessible() static
      print errno when reporting a system call error
      rerere.c: move error_errno() closer to the source system call
      log: fix memory leak in open_next_file()

Patrick Steinhardt (1):
      git-stash: fix pushing stash with pathspec from subdir

Peter Krefting (1):
      l10n: sv.po: Update Swedish translation (3206t0f0u)

Phillip Wood (10):
      rebase -i: fix reflog message
      rebase -i: silence stash apply
      rebase -i: add missing newline to end of message
      rebase -i: add test for reflog message
      rebase: add regression tests for console output
      rebase: add more regression tests for console output
      add -i: move unquote_path() to Git.pm
      Git::unquote_path(): handle '\a'
      Git::unquote_path(): throw an exception on bad path
      t9700: add tests for Git::unquote_path()

Prathamesh Chavan (1):
      dir: create function count_slashes()

Ralf Thielow (2):
      l10n: de.po: fix typo
      l10n: de.po: update German translation

Ramsay Jones (5):
      t7400: add !CYGWIN prerequisite to 'add with \\ in path'
      archive-tar: fix a sparse 'constant too large' warning
      usage: add NORETURN to BUG() function definitions
      git_fopen: fix a sparse 'not declared' warning
      config.mak.uname: set FREAD_READS_DIRECTORIES for cygwin

René Scharfe (32):
      archive-zip: add tests for big ZIP archives
      archive-zip: use strbuf for ZIP directory
      archive-zip: write ZIP dir entry directly to strbuf
      archive-zip: support archives bigger than 4GB
      archive-zip: support files bigger than 4GB
      archive-zip: set version field for big files correctly
      t5004: require 64-bit support for big ZIP tests
      checkout: check return value of resolve_refdup before using hash
      am: check return value of resolve_refdup before using hash
      p0004: simplify calls of test-lazy-init-name-hash
      p0004: avoid using pipes
      p0004: use test_perf
      p0004: don't abort if multi-threaded is too slow
      p0004: don't error out if test repo is too small
      mingw: simplify PATH handling
      pretty: recalculate duplicate short hashes
      strbuf: let strbuf_addftime handle %z and %Z itself
      sha1_name: cache readdir(3) results in find_short_object_filename()
      p4205: add perf test script for pretty log formats
      sha1_file: let for_each_file_in_obj_subdir() handle subdir names
      sha1_file: guard against invalid loose subdirectory numbers
      apply: check git diffs for missing old filenames
      apply: check git diffs for invalid file modes
      apply: check git diffs for mutually exclusive header lines
      coccinelle: polish FREE_AND_NULL rules
      apply: use starts_with() in gitdiff_verify_name()
      apply: use strcmp(3) for comparing strings in gitdiff_verify_name()
      urlmatch: use hex2chr() in append_normalized_escapes()
      progress: show overall rate in last update
      wt-status: use separate variable for result of shorten_unambiguous_ref
      use DIV_ROUND_UP
      Makefile: allow combining UBSan with other sanitizers

Rikard Falkeborn (7):
      completion: add completions for git config commit
      completion: add git config gc completions
      completion: add git config core completions
      completion: add git config am.threeWay completion
      completion: add git config advice completions
      completion: add git config credential completions
      completion: add git config credentialCache.ignoreSIGHUP

SZEDER Gábor (9):
      test-lib: abort when can't remove trash directory
      docs/config.txt: fix indefinite article in core.fileMode description
      revision.h: turn rev_info.early_output back into an unsigned int
      revision.c: stricter parsing of '--no-{min,max}-parents'
      revision.c: stricter parsing of '--early-output'
      revision.c: use skip_prefix() in handle_revision_opt()
      revision.c: use skip_prefix() in handle_revision_pseudo_opt()
      docs/pretty-formats: stress that %- removes all preceding line-feeds
      blame: fix memory corruption scrambling revision name in error message

Sahil Dua (2):
      branch test: fix invalid config key access
      t3200: add test for single parameter passed to -m option

Samuel Lijin (6):
      t7300: clean -d should skip dirs with ignored files
      t7061: status --ignored should search untracked dirs
      dir: recurse into untracked dirs for ignored files
      dir: hide untracked contents of untracked dirs
      dir: expose cmp_name() and check_contains()
      clean: teach clean -d to preserve ignored paths

Stefan Beller (30):
      entry.c: submodule recursing: respect force flag correctly
      submodule.c: uninitialized submodules are ignored in recursive commands
      submodule.c: submodule_move_head works with broken submodules
      builtin/reset: add --recurse-submodules switch
      submodule_move_head: reuse child_process structure for futher commands
      submodule: avoid auto-discovery in new working tree manipulator code
      submodule: properly recurse for read-tree and checkout
      diff: recurse into nested submodules for inline diff
      diff: enable indent heuristic by default
      t5545: enhance test coverage when no http server is installed
      t5531: fix test description
      submodule recursing: do not write a config variable twice
      submodule test invocation: only pass additional arguments
      reset/checkout/read-tree: unify config callback for submodule recursion
      submodule loading: separate code path for .gitmodules and config overlay
      Introduce 'submodule.recurse' option for worktree manipulators
      builtin/grep.c: respect 'submodule.recurse' option
      builtin/push.c: respect 'submodule.recurse' option
      builtin/fetch.c: respect 'submodule.recurse' option
      Documentation/git-rm: correct submodule description
      t4005: modernize style and drop hard coded sha1
      submodules: overhaul documentation
      builtin/fetch: factor submodule recurse parsing out to submodule config
      builtin/fetch: parse recurse-submodules-default at default options parsing
      pull: optionally rebase submodules (remote submodule changes only)
      builtin/fetch cleanup: always set default value for submodule recursing
      merge-recursive: use DIFF_XDL_SET macro
      hashmap.h: compare function has access to a data field
      patch-ids.c: use hashmap correctly
      hashmap: migrate documentation from Documentation/technical into header

Stephen Kent (1):
      status: add color config slots for branch info in "--short --branch"

Sven Strickroth (1):
      doc: use https links to Wikipedia to avoid http redirects

Sylvestre Ledru (1):
      l10n: fr.po Fix some french typos

Thomas Gummerer (1):
      completion: add git stash push

Torsten Bögershausen (2):
      t0027: tests are not expensive; remove t0025
      cygwin: allow pushing to UNC paths

Trần Ngọc Quân (2):
      l10n: vi.po(3206t): Update Vietnamese translation
      l10n: vi.po (3213t): Updated 9 new strings

Tyler Brazier (1):
      pull: ff --rebase --autostash works in dirty repo

Ville Skyttä (1):
      Spelling fixes

brian m. carlson (61):
      fetch-pack: convert to struct object_id
      Clean up outstanding object_id transforms.
      Convert struct cache_tree to use struct object_id
      builtin/name-rev: convert to struct object_id
      builtin/prune: convert to struct object_id
      bundle: convert to struct object_id
      branch: convert to struct object_id
      builtin/blame: convert static function to struct object_id
      builtin/rev-parse: convert to struct object_id
      fast-import: convert internal structs to struct object_id
      fast-import: convert to struct object_id
      submodule: convert merge_submodule to use struct object_id
      notes-cache: convert to struct object_id
      parse-options-cb: convert to struct object_id
      reflog_expire: convert to struct object_id
      builtin/verify-commit: convert to struct object_id
      tag: convert parse_tag_buffer to struct object_id
      http-push: convert some static functions to struct object_id
      notes-utils: convert internals to struct object_id
      revision: convert prepare_show_merge to struct object_id
      shallow: convert shallow registration functions to object_id
      sequencer: convert some functions to struct object_id
      builtin/tag: convert to struct object_id
      Convert remaining callers of lookup_commit_reference* to object_id
      Convert lookup_commit* to struct object_id
      pack: convert struct pack_idx_entry to struct object_id
      builtin/unpack-objects: convert to struct object_id
      Convert remaining callers of lookup_blob to object_id
      Convert lookup_blob to struct object_id
      tree: convert read_tree_1 to use struct object_id internally
      builtin/reflog: convert tree_is_complete to take struct object_id
      Convert lookup_tree to struct object_id
      log-tree: convert to struct object_id
      Convert lookup_tag to struct object_id
      Convert the verify_pack callback to struct object_id
      Convert struct ref_array_item to struct object_id
      ref-filter: convert some static functions to struct object_id
      refs: convert struct ref_update to use struct object_id
      refs/files-backend: convert many internals to struct object_id
      http-push: convert process_ls_object and descendants to object_id
      revision: rename add_pending_sha1 to add_pending_oid
      revision: convert remaining parse_object callers to object_id
      upload-pack: convert remaining parse_object callers to object_id
      sha1_name: convert internals of peel_onion to object_id
      builtin/read-tree: convert to struct object_id
      builtin/ls-files: convert overlay_tree_on_cache to object_id
      sequencer: convert fast_forward_to to struct object_id
      merge: convert checkout_fast_forward to struct object_id
      builtin/ls-tree: convert to struct object_id
      diff-lib: convert do_diff_cache to struct object_id
      sequencer: convert do_recursive_merge to struct object_id
      tree: convert parse_tree_indirect to struct object_id
      object: convert parse_object* to take struct object_id
      builtin/log: honor log.decorate
      notes: convert internal structures to struct object_id
      notes: convert internal parts to struct object_id
      notes: convert for_each_note to struct object_id
      notes: make get_note return pointer to struct object_id
      notes: convert format_display_notes to struct object_id
      builtin/notes: convert to struct object_id
      notes: convert some accessor functions to struct object_id

xiaoqiang zhao (1):
      send-email: --batch-size to work around some SMTP server limit

Ævar Arnfjörð Bjarmason (77):
      tests: change "cd ... && git fetch" to "cd &&\n\tgit fetch"
      clone: add a --no-tags option to clone without tags
      tests: rename a test having to do with shallow submodules
      config.mak.uname: set NO_REGEX=NeedsStartEnd on AIX
      doc: replace a couple of broken gmane links
      tests: fix tests broken under GETTEXT_POISON=YesPlease
      perf: add function to setup a fresh test repo
      perf: add test showing exponential growth in path globbing
      config: match both symlink & realpath versions in IncludeIf.gitdir:*
      Makefile & configure: reword inaccurate comment about PCRE
      grep & rev-list doc: stop promising libpcre for --perl-regexp
      test-lib: rename the LIBPCRE prerequisite to PCRE
      log: add exhaustive tests for pattern style options & config
      log: make --regexp-ignore-case work with --perl-regexp
      grep: add a test asserting that --perl-regexp dies when !PCRE
      grep: add a test for backreferences in PCRE patterns
      grep: change non-ASCII -i test to stop using --debug
      grep: add tests for --threads=N and grep.threads
      grep: amend submodule recursion test for regex engine testing
      grep: add tests for grep pattern types being passed to submodules
      grep: add a test helper function for less verbose -f \0 tests
      grep: prepare for testing binary regexes containing rx metacharacters
      grep: add tests to fix blind spots with \0 patterns
      perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do
      perf: emit progress output when unpacking & building
      sha1dc: update from upstream
      tag: duplicate mention of --contains should mention --no-contains
      perf: add a comparison test of grep regex engines
      perf: add a comparison test of grep regex engines with -F
      perf: add a comparison test of log --grep regex engines
      perf: add a comparison test of log --grep regex engines with -F
      grep: catch a missing enum in switch statement
      grep: remove redundant regflags assignments
      grep: factor test for \0 in grep patterns into a function
      grep: change the internal PCRE macro names to be PCRE1
      grep: change internal *pcre* variable & function names to be *pcre1*
      grep: move is_fixed() earlier to avoid forward declaration
      test-lib: add a PTHREADS prerequisite
      pack-objects & index-pack: add test for --threads warning
      pack-objects: fix buggy warning about threads
      grep: given --threads with NO_PTHREADS=YesPlease, warn
      grep: assert that threading is enabled when calling grep_{lock,unlock}
      grep: don't redundantly compile throwaway patterns under threading
      grep: skip pthreads overhead when using one thread
      log: add -P as a synonym for --perl-regexp
      grep: add support for the PCRE v1 JIT API
      grep: un-break building with PCRE < 8.32
      grep: un-break building with PCRE < 8.20
      wildmatch test: remove redundant duplicate test
      C style: use standard style for "TRANSLATORS" comments
      grep: un-break building with PCRE >= 8.32 without --enable-jit
      grep: add support for PCRE v2
      perf: work around the tested repo having an index.lock
      sha1dc: update from upstream
      git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL
      wildmatch test: cover a blind spot in "/" matching
      coccinelle: add a rule to make "type" code use FREE_AND_NULL()
      coccinelle: make use of the "type" FREE_AND_NULL() rule
      coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
      coccinelle: make use of the "expression" FREE_AND_NULL() rule
      *.[ch] refactoring: make use of the FREE_AND_NULL() macro
      grep: fix erroneously copy/pasted variable in check/assert pattern
      die(): stop hiding errors due to overzealous recursion guard
      wildmatch: remove unused wildopts parameter
      strbuf.h comment: discuss strbuf_addftime() arguments in order
      grep: remove redundant double assignment to 0
      grep: adjust a redundant grep pattern type assignment
      grep: remove redundant "fixed" field re-assignment to 0
      grep: remove redundant and verbose re-assignments to 0
      grep: remove regflags from the public grep_opt API
      grep: remove redundant REG_NEWLINE when compiling fixed regex
      strbuf: change an always NULL/"" strbuf_addftime() param to bool
      sha1dc: update from upstream
      sha1dc: optionally use sha1collisiondetection as a submodule
      RelNotes: mention "log: add -P as a synonym for --perl-regexp"
      RelNotes: mention "log: make --regexp-ignore-case work with --perl-regexp"
      RelNotes: mention "sha1dc: optionally use sha1collisiondetection as a submodule"

Štěpán Němec (1):
      doc: git-reset: fix a trivial typo


^ permalink raw reply	[relevance 11%]

* Re: [PATCH v2 02/15] submodule: don't use submodule_from_name
  2017-08-03 18:57     ` [PATCH v2 02/15] submodule: don't use submodule_from_name Stefan Beller
@ 2017-08-04 21:53       ` Brandon Williams
  2017-08-11 16:59         ` Heiko Voigt
  0 siblings, 1 reply; 200+ results
From: Brandon Williams @ 2017-08-04 21:53 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Junio C Hamano, Jonathan Nieder, Jens Lehmann

On 08/03, Stefan Beller wrote:
> On Thu, Aug 3, 2017 at 11:19 AM, Brandon Williams <bmwill@google.com> wrote:
> > The function 'submodule_from_name()' is being used incorrectly here as a
> > submodule path is being used instead of a submodule name.  Since the
> > correct function to use with a path to a submodule is already being used
> > ('submodule_from_path()') let's remove the call to
> > 'submodule_from_name()'.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> 
> In case a reroll is needed, you could incorperate Jens feedback
> stating that 851e18c385 should have done it.

K I'll add that into the commit message.

> 
> > ---
> >  submodule.c | 2 --
> >  1 file changed, 2 deletions(-)
> >
> > diff --git a/submodule.c b/submodule.c
> > index 5139b9256..19bd13bb2 100644
> > --- a/submodule.c
> > +++ b/submodule.c
> > @@ -1177,8 +1177,6 @@ static int get_next_submodule(struct child_process *cp,
> >                         continue;
> >
> >                 submodule = submodule_from_path(&null_oid, ce->name);
> > -               if (!submodule)
> > -                       submodule = submodule_from_name(&null_oid, ce->name);
> >
> >                 default_argv = "yes";
> >                 if (spf->command_line_option == RECURSE_SUBMODULES_DEFAULT) {
> > --
> > 2.14.0.rc1.383.gd1ce394fe2-goog
> >

-- 
Brandon Williams

^ permalink raw reply	[relevance 16%]

* [GSoC][PATCH 00/13] Update: Week-12
@ 2017-08-07 21:18 Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
                   ` (11 more replies)
  0 siblings, 12 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

SUMMARY OF MY PROJECT:

Git submodule subcommands are currently implemented by using shell script
'git-submodule.sh'. There are several reasons why we'll prefer not to
use the shell script. My project intends to convert the subcommands into
C code, thus making them builtins. This will increase Git's portability
and hence the efficiency of working with the git-submodule commands.
Link to the complete proposal: [1]

Mentors:
Stefan Beller <sbeller@google.com>
Christian Couder <christian.couder@gmail.com>

UPDATES:

Following are the updates about my ongoing project:

* Following patches were updated after the previous reviews:
  submodule subcommands:
  - deinit
  - summary
  - foreach

* summary: the function print_submodule_summary() is split-up
  into two separate functions: generate_submodule_summary()
  and print_summary().

* porting of submodule subcommand 'add' is completed and I have
  started with debugging ported function. Currently, the
  entire function cmd_add() is ported to the function
  module_add() in C. Soon, its first patch will be floated
  here as well once debugging is completed. Its progress can be
  viewed at [2].

* displaypath: Last week, there was some confusion produced
  with the way, the value of displaypath is being generated,
  which led to some discussion, which is available at: [3].

PLAN FOR WEEK-13 (8 August 2017 to 14 August 2017):

* patches: IMO, the patches till deinit are reviewed many times,
  and hence will try to get at least these patches merged.

* add: As this subcommand is widely used in the test suite, there
  are many tests this ported function is failing at. Hence,
  debugging the subcommand would be another task for the next week.

* deinit: A bug was identified by Stefan in the last patch-series.
  its details are available at: [4]
  Currenlty, the bug was handled by adding a NEEDSWORK tagged
  comment as suggest. If possible, I will also start working
  on debugging the issue asap.

A complete build report of these series of patches is available at: [5].
Build #151
Branch: week-12

The work is pushed on Github and is available at: [6].

[1]: https://docs.google.com/document/d/1krxVLooWl--75Pot3dazhfygR3wCUUWZWzTXtK1L-xU/
[2]: https://github.com/pratham-pc/git/commits/sub-add
[3]: https://public-inbox.org/git/CAME+mvXsh53kLJ4se4uKY=SJcvSbHtEZQ6K2CgAPs=1wxUxk1A@mail.gmail.com/
[4]: https://public-inbox.org/git/CAGZ79kbyyR54me_+wQDZRrikqKTp_a98yozVfr8P85QHfyyy=Q@mail.gmail.com/
[5]: https://travis-ci.org/pratham-pc/git/builds/
[6]: https://github.com/pratham-pc/git/commits/week-12

Prathamesh Chavan (13):
  submodule--helper: introduce get_submodule_displaypath()
  submodule--helper: introduce for_each_submodule_list()
  submodule: port set_name_rev() from shell to C
  submodule: port submodule subcommand 'status' from shell to C
  submodule: port submodule subcommand 'sync' from shell to C
  submodule: port submodule subcommand 'deinit' from shell to C
  diff: change scope of the function count_lines()
  submodule: port submodule subcommand 'summary' from shell to C
  submodule foreach: correct '$path' in nested submodules from a
    subdirectory
  submodule foreach: document '$sm_path' instead of '$path'
  submodule foreach: clarify the '$toplevel' variable documentation
  submodule foreach: document variable '$displaypath'
  submodule: port submodule subcommand 'foreach' from shell to C

 Documentation/git-submodule.txt |   15 +-
 builtin/submodule--helper.c     | 1190 ++++++++++++++++++++++++++++++++++++++-
 diff.c                          |    2 +-
 diff.h                          |    1 +
 git-submodule.sh                |  396 +------------
 t/t7407-submodule-foreach.sh    |   38 +-
 6 files changed, 1222 insertions(+), 420 deletions(-)

-- 
2.13.0


^ permalink raw reply	[relevance 17%]

* [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath()
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
@ 2017-08-07 21:18 ` Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function get_submodule_displaypath() to replace the code
occurring in submodule_init() for generating displaypath of the
submodule with a call to it.

This new function will also be used in other parts of the system
in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 6abdad329..7af4de09b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -220,6 +220,27 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+
+	if (prefix && super_prefix) {
+		BUG("cannot have prefix '%s' and superprefix '%s'",
+		    prefix, super_prefix);
+	} else if (prefix) {
+		struct strbuf sb = STRBUF_INIT;
+		char *displaypath = xstrdup(relative_path(path, prefix, &sb));
+		strbuf_release(&sb);
+		return displaypath;
+	} else if (super_prefix) {
+		int len = strlen(super_prefix);
+		const char *format = is_dir_sep(super_prefix[len - 1]) ? "%s%s" : "%s/%s";
+		return xstrfmt(format, super_prefix, path);
+	} else {
+		return xstrdup(path);
+	}
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -339,16 +360,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 	/* Only loads from .gitmodules, no overlay with .git/config */
 	gitmodules_config();
-
-	if (prefix && get_super_prefix())
-		die("BUG: cannot have prefix and superprefix");
-	else if (prefix)
-		displaypath = xstrdup(relative_path(path, prefix, &sb));
-	else if (get_super_prefix()) {
-		strbuf_addf(&sb, "%s%s", get_super_prefix(), path);
-		displaypath = strbuf_detach(&sb, NULL);
-	} else
-		displaypath = xstrdup(path);
+	displaypath = get_submodule_displaypath(path, prefix);
 
 	sub = submodule_from_path(null_sha1, path);
 
@@ -363,7 +375,6 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 * Set active flag for the submodule being initialized
 	 */
 	if (!is_submodule_active(the_repository, path)) {
-		strbuf_reset(&sb);
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list()
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
@ 2017-08-07 21:18 ` Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Introduce function for_each_submodule_list() and
replace a loop in module_init() with a call to it.

The new function will also be used in other parts of the
system in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7af4de09b..e41572f7a 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -14,6 +14,9 @@
 #include "refs.h"
 #include "connect.h"
 
+typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
+				      void *cb_data);
+
 static char *get_default_remote(void)
 {
 	char *dest = NULL, *ret;
@@ -352,17 +355,30 @@ static int module_list(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static void init_submodule(const char *path, const char *prefix, int quiet)
+static void for_each_submodule_list(const struct module_list list,
+				    submodule_list_func_t fn, void *cb_data)
 {
+	int i;
+	for (i = 0; i < list.nr; i++)
+		fn(list.entries[i], cb_data);
+}
+
+struct init_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+};
+#define INIT_CB_INIT { NULL, 0 }
+
+static void init_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct init_cb *info = cb_data;
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	/* Only loads from .gitmodules, no overlay with .git/config */
-	gitmodules_config();
-	displaypath = get_submodule_displaypath(path, prefix);
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
 
-	sub = submodule_from_path(null_sha1, path);
+	sub = submodule_from_path(null_sha1, list_item->name);
 
 	if (!sub)
 		die(_("No url found for submodule path '%s' in .gitmodules"),
@@ -374,7 +390,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 	 *
 	 * Set active flag for the submodule being initialized
 	 */
-	if (!is_submodule_active(the_repository, path)) {
+	if (!is_submodule_active(the_repository, list_item->name)) {
 		strbuf_addf(&sb, "submodule.%s.active", sub->name);
 		git_config_set_gently(sb.buf, "true");
 	}
@@ -416,7 +432,7 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 		if (git_config_set_gently(sb.buf, url))
 			die(_("Failed to register url for submodule path '%s'"),
 			    displaypath);
-		if (!quiet)
+		if (!info->quiet)
 			fprintf(stderr,
 				_("Submodule '%s' (%s) registered for path '%s'\n"),
 				sub->name, url, displaypath);
@@ -445,10 +461,10 @@ static void init_submodule(const char *path, const char *prefix, int quiet)
 
 static int module_init(int argc, const char **argv, const char *prefix)
 {
+	struct init_cb info = INIT_CB_INIT;
 	struct pathspec pathspec;
 	struct module_list list = MODULE_LIST_INIT;
 	int quiet = 0;
-	int i;
 
 	struct option module_init_options[] = {
 		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
@@ -473,8 +489,11 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	if (!argc && git_config_get_value_multi("submodule.active"))
 		module_list_active(&list);
 
-	for (i = 0; i < list.nr; i++)
-		init_submodule(list.entries[i]->name, prefix, quiet);
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+
+	gitmodules_config();
+	for_each_submodule_list(list, init_submodule, &info);
 
 	return 0;
 }
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 01/13] submodule--helper: introduce get_submodule_displaypath() Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 02/13] submodule--helper: introduce for_each_submodule_list() Prathamesh Chavan
@ 2017-08-07 21:18 ` Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Function set_name_rev() is ported from git-submodule to the
submodule--helper builtin. The function get_name_rev() generates the
value of the revision name as required, and the function
print_name_rev() handles the formating and printing of the obtained
revision name.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 63 +++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 16 ++----------
 2 files changed, 65 insertions(+), 14 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index e41572f7a..421eee1e2 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -244,6 +244,68 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+static char *get_name_rev(const char *sub_path, const char* object_id)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char ***d;
+
+	static const char *describe_bare[] = {
+		NULL
+	};
+
+	static const char *describe_tags[] = {
+		"--tags", NULL
+	};
+
+	static const char *describe_contains[] = {
+		"--contains", NULL
+	};
+
+	static const char *describe_all_always[] = {
+		"--all", "--always", NULL
+	};
+
+	static const char **describe_argv[] = {
+		describe_bare, describe_tags, describe_contains,
+		describe_all_always, NULL
+	};
+
+	for (d = describe_argv; *d; d++) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+		prepare_submodule_repo_env(&cp.env_array);
+		cp.dir = sub_path;
+		cp.git_cmd = 1;
+		cp.no_stderr = 1;
+
+		argv_array_push(&cp.args, "describe");
+		argv_array_pushv(&cp.args, *d);
+		argv_array_push(&cp.args, object_id);
+
+		if (!capture_command(&cp, &sb, 0) && sb.len) {
+			strbuf_strip_suffix(&sb, "\n");
+			return strbuf_detach(&sb, NULL);
+		}
+	}
+
+	strbuf_release(&sb);
+	return NULL;
+}
+
+static int print_name_rev(int argc, const char **argv, const char *prefix)
+{
+	char *namerev;
+	if (argc != 3)
+		die("print-name-rev only accepts two arguments: <path> <sha1>");
+
+	namerev = get_name_rev(argv[1], argv[2]);
+	if (namerev && namerev[0])
+		printf(" (%s)", namerev);
+	printf("\n");
+
+	free(namerev);
+	return 0;
+}
+
 struct module_list {
 	const struct cache_entry **entries;
 	int alloc, nr;
@@ -1242,6 +1304,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index e131760ee..e988167e0 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -759,18 +759,6 @@ cmd_update()
 	}
 }
 
-set_name_rev () {
-	revname=$( (
-		sanitize_submodule_env
-		cd "$1" && {
-			git describe "$2" 2>/dev/null ||
-			git describe --tags "$2" 2>/dev/null ||
-			git describe --contains "$2" 2>/dev/null ||
-			git describe --all --always "$2"
-		}
-	) )
-	test -z "$revname" || revname=" ($revname)"
-}
 #
 # Show commit summary for submodules in index or working tree
 #
@@ -1042,14 +1030,14 @@ cmd_status()
 		fi
 		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
 		then
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say " $sha1 $displaypath$revname"
 		else
 			if test -z "$cached"
 			then
 				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
 			fi
-			set_name_rev "$sm_path" "$sha1"
+			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
 			say "+$sha1 $displaypath$revname"
 		fi
 
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' from shell to C
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (2 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 03/13] submodule: port set_name_rev() from shell to C Prathamesh Chavan
@ 2017-08-07 21:18 ` " Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

This aims to make git-submodule 'status' a built-in. Hence, the function
cmd_status() is ported from shell to C. This is done by introducing
three functions: module_status(), submodule_status() and print_status().

The function module_status() acts as the front-end of the subcommand.
It parses subcommand's options and then calls the function
module_list_compute() for computing the list of submodules. Then
this functions calls for_each_submodule_list() looping through the
list obtained.

Then for_each_submodule_list() calls submodule_status() for each of the
submodule in its list. The function submodule_status() is responsible
for generating the status each submodule it is called for, and
then calls print_status().

Finally, the function print_status() handles the printing of submodule's
status.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 156 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  49 +-------------
 2 files changed, 157 insertions(+), 48 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 421eee1e2..1bf7bb2a2 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -560,6 +560,161 @@ static int module_init(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct status_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+	unsigned int cached: 1;
+};
+#define STATUS_CB_INIT { NULL, 0, 0, 0 }
+
+static void print_status(struct status_cb *info, char state, const char *path,
+			 const struct object_id *oid, const char *displaypath)
+{
+	if (info->quiet)
+		return;
+
+	printf("%c%s %s", state, oid_to_hex(oid), displaypath);
+
+	if (state == ' ' || state == '+') {
+		struct argv_array name_rev_args = ARGV_ARRAY_INIT;
+
+		argv_array_pushl(&name_rev_args, "print-name-rev",
+				 path, oid_to_hex(oid), NULL);
+		print_name_rev(name_rev_args.argc, name_rev_args.argv,
+			       info->prefix);
+
+		argv_array_clear(&name_rev_args);
+	} else {
+		printf("\n");
+	}
+}
+
+static int handle_submodule_head_ref(const char *refname,
+				     const struct object_id *oid, int flags,
+				     void *cb_data)
+{
+	struct object_id *output = cb_data;
+	if (oid)
+		oidcpy(output, oid);
+
+	return 0;
+}
+
+static void status_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct status_cb *info = cb_data;
+	char *displaypath;
+	struct argv_array diff_files_args = ARGV_ARRAY_INIT;
+
+	if (!submodule_from_path(null_sha1, list_item->name))
+		die(_("no submodule mapping found in .gitmodules for path '%s'"),
+		      list_item->name);
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (list_item->ce_flags) {
+		print_status(info, 'U', list_item->name,
+			     &null_oid, displaypath);
+		goto cleanup;
+	}
+
+	if (!is_submodule_active(the_repository, list_item->name)) {
+		print_status(info, '-', list_item->name, &list_item->oid,
+			     displaypath);
+		goto cleanup;
+	}
+
+	argv_array_pushl(&diff_files_args, "diff-files",
+			 "--ignore-submodules=dirty", "--quiet", "--",
+			 list_item->name, NULL);
+
+	if (!cmd_diff_files(diff_files_args.argc, diff_files_args.argv,
+			    info->prefix)) {
+		print_status(info, ' ', list_item->name, &list_item->oid,
+			     displaypath);
+	} else {
+		if (!info->cached) {
+			struct object_id oid;
+
+			if (head_ref_submodule(list_item->name,
+					       handle_submodule_head_ref, &oid))
+				die(_("could not resolve HEAD ref inside the"
+				      "submodule '%s'"), list_item->name);
+
+			print_status(info, '+', list_item->name, &oid,
+				     displaypath);
+		} else {
+			print_status(info, '+', list_item->name,
+				     &list_item->oid, displaypath);
+		}
+	}
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "status", "--recursive",
+				 NULL);
+
+		if (info->cached)
+			argv_array_push(&cpr.args, "--cached");
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	argv_array_clear(&diff_files_args);
+	free(displaypath);
+}
+
+static int module_status(int argc, const char **argv, const char *prefix)
+{
+	struct status_cb info = STATUS_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int cached = 0;
+	int recursive = 0;
+
+	struct option module_status_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT_BOOL(0, "cached", &cached, N_("Use commit stored in the index instead of the one stored in the submodule HEAD")),
+		OPT_BOOL(0, "recursive", &recursive, N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule status [--quiet] [--cached] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_status_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+	info.cached = !!cached;
+
+	gitmodules_config();
+	for_each_submodule_list(list, status_submodule, &info);
+
+	return 0;
+}
+
 static int module_name(int argc, const char **argv, const char *prefix)
 {
 	const struct submodule *sub;
@@ -1306,6 +1461,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
+	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index e988167e0..51b057d82 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1005,54 +1005,7 @@ cmd_status()
 		shift
 	done
 
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		if test "$stage" = U
-		then
-			say "U$sha1 $displaypath"
-			continue
-		fi
-		if ! git submodule--helper is-active "$sm_path" ||
-		{
-			! test -d "$sm_path"/.git &&
-			! test -f "$sm_path"/.git
-		}
-		then
-			say "-$sha1 $displaypath"
-			continue;
-		fi
-		if git diff-files --ignore-submodules=dirty --quiet -- "$sm_path"
-		then
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say " $sha1 $displaypath$revname"
-		else
-			if test -z "$cached"
-			then
-				sha1=$(sanitize_submodule_env; cd "$sm_path" && git rev-parse --verify HEAD)
-			fi
-			revname=$(git submodule--helper print-name-rev "$sm_path" "$sha1")
-			say "+$sha1 $displaypath$revname"
-		fi
-
-		if test -n "$recursive"
-		then
-			(
-				prefix="$displaypath/"
-				sanitize_submodule_env
-				wt_prefix=
-				cd "$sm_path" &&
-				eval cmd_status
-			) ||
-			die "$(eval_gettext "Failed to recurse into submodule path '\$sm_path'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} "$@"
 }
 #
 # Sync remote urls for submodules
-- 
2.13.0


^ permalink raw reply	[relevance 20%]

* [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' from shell to C
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (3 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 04/13] submodule: port submodule subcommand 'status' " Prathamesh Chavan
@ 2017-08-07 21:18 ` " Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

Port the submodule subcommand 'sync' from shell to C using the same
mechanism as that used for porting submodule subcommand 'status'.
Hence, here the function cmd_sync() is ported from shell to C.
This is done by introducing three functions: module_sync(),
sync_submodule() and print_default_remote().

The function print_default_remote() is introduced for getting
the default remote as stdout.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 183 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  57 +-------------
 2 files changed, 184 insertions(+), 56 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 1bf7bb2a2..82f1aed87 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -44,6 +44,20 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static int print_default_remote(int argc, const char **argv, const char *prefix)
+{
+	const char *remote;
+
+	if (argc != 1)
+		die(_("submodule--helper print-default-remote takes no arguments"));
+
+	remote = get_default_remote();
+	if (remote)
+		printf("%s\n", remote);
+
+	return 0;
+}
+
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -379,6 +393,25 @@ static void module_list_active(struct module_list *list)
 	*list = active_modules;
 }
 
+static char *get_up_path(const char *path)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	for (i = count_slashes(path); i; i--)
+		strbuf_addstr(&sb, "../");
+
+	/*
+	 * Check if 'path' ends with slash or not
+	 * for having the same output for dir/sub_dir
+	 * and dir/sub_dir/
+	 */
+	if (!is_dir_sep(path[strlen(path) - 1]))
+		strbuf_addstr(&sb, "../");
+
+	return strbuf_detach(&sb, NULL);
+}
+
 static int module_list(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -734,6 +767,154 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct sync_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define SYNC_CB_INIT { NULL, 0, 0 }
+
+static void sync_submodule(const struct cache_entry *list_item, void *cb_data)
+{
+	struct sync_cb *info = cb_data;
+	const struct submodule *sub;
+	char *remote_key = NULL;
+	char *sub_origin_url, *super_config_url, *displaypath;
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *sub_config_path = NULL;
+
+	if (!is_submodule_active(the_repository, list_item->name))
+		return;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (sub && sub->url) {
+		if (starts_with_dot_dot_slash(sub->url) || starts_with_dot_slash(sub->url)) {
+			char *remote_url, *up_path;
+			char *remote = get_default_remote();
+			strbuf_addf(&sb, "remote.%s.url", remote);
+
+			if (git_config_get_string(sb.buf, &remote_url))
+				remote_url = xgetcwd();
+
+			up_path = get_up_path(list_item->name);
+			sub_origin_url = relative_url(remote_url, sub->url, up_path);
+			super_config_url = relative_url(remote_url, sub->url, NULL);
+
+			free(remote);
+			free(up_path);
+			free(remote_url);
+		} else {
+			sub_origin_url = xstrdup(sub->url);
+			super_config_url = xstrdup(sub->url);
+		}
+	} else {
+		sub_origin_url = "";
+		super_config_url = "";
+	}
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	if (!info->quiet)
+		printf(_("Synchronizing submodule url for '%s'\n"),
+			 displaypath);
+
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "submodule.%s.url", sub->name);
+	if (git_config_set_gently(sb.buf, super_config_url))
+		die(_("failed to register url for submodule path '%s'"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.dir = list_item->name;
+	argv_array_pushl(&cp.args, "submodule--helper",
+			 "print-default-remote", NULL);
+
+	strbuf_reset(&sb);
+	if (capture_command(&cp, &sb, 0))
+		die(_("failed to get the default remote for submodule '%s'"),
+		      list_item->name);
+
+	strbuf_strip_suffix(&sb, "\n");
+	remote_key = xstrfmt("remote.%s.url", sb.buf);
+
+	strbuf_reset(&sb);
+	submodule_to_gitdir(&sb, list_item->name);
+	strbuf_addstr(&sb, "/config");
+
+	if (git_config_set_in_file_gently(sb.buf, remote_key, sub_origin_url))
+		die(_("failed to update remote for submodule '%s'"),
+		      list_item->name);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "sync", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (run_command(&cpr))
+			die(_("failed to recurse into submodule '%s'"),
+			      list_item->name);
+	}
+
+cleanup:
+	strbuf_release(&sb);
+	free(remote_key);
+	free(super_config_url);
+	free(displaypath);
+	free(sub_config_path);
+	free(sub_origin_url);
+}
+
+static int module_sync(int argc, const char **argv, const char *prefix)
+{
+	struct sync_cb info = SYNC_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_sync_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of synchronizing submodule url")),
+		OPT_BOOL(0, "recursive", &recursive,
+			N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper sync [--quiet] [--recursive] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_sync_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, sync_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1462,6 +1643,8 @@ static struct cmd_struct commands[] = {
 	{"print-name-rev", print_name_rev, 0},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
+	{"print-default-remote", print_default_remote, 0},
+	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 51b057d82..5acf6f404 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1037,63 +1037,8 @@ cmd_sync()
 			;;
 		esac
 	done
-	cd_to_toplevel
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
 
-		# skip inactive submodules
-		if ! git submodule--helper is-active "$sm_path"
-		then
-			continue
-		fi
-
-		name=$(git submodule--helper name "$sm_path")
-		url=$(git config -f .gitmodules --get submodule."$name".url)
-
-		# Possibly a url relative to parent
-		case "$url" in
-		./*|../*)
-			# rewrite foo/bar as ../.. to find path from
-			# submodule work tree to superproject work tree
-			up_path="$(printf '%s\n' "$sm_path" | sed "s/[^/][^/]*/../g")" &&
-			# guarantee a trailing /
-			up_path=${up_path%/}/ &&
-			# path from submodule work tree to submodule origin repo
-			sub_origin_url=$(git submodule--helper resolve-relative-url "$url" "$up_path") &&
-			# path from superproject work tree to submodule origin repo
-			super_config_url=$(git submodule--helper resolve-relative-url "$url") || exit
-			;;
-		*)
-			sub_origin_url="$url"
-			super_config_url="$url"
-			;;
-		esac
-
-		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-		say "$(eval_gettext "Synchronizing submodule url for '\$displaypath'")"
-		git config submodule."$name".url "$super_config_url"
-
-		if test -e "$sm_path"/.git
-		then
-		(
-			sanitize_submodule_env
-			cd "$sm_path"
-			remote=$(get_default_remote)
-			git config remote."$remote".url "$sub_origin_url"
-
-			if test -n "$recursive"
-			then
-				prefix="$prefix$sm_path/"
-				eval cmd_sync
-			fi
-		)
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 cmd_absorbgitdirs()
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' from shell to C
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (4 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 05/13] submodule: port submodule subcommand 'sync' " Prathamesh Chavan
@ 2017-08-07 21:18 ` " Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The same mechanism is used even for porting this submodule
subcommand, as used in the ported subcommands till now.
The function cmd_deinit in split up after porting into three
functions: module_deinit(), for_each_submodule_list() and
deinit_submodule().

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 builtin/submodule--helper.c | 148 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  55 +---------------
 2 files changed, 149 insertions(+), 54 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 82f1aed87..02787e4a4 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -915,6 +915,153 @@ static int module_sync(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct deinit_cb {
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int force: 1;
+	unsigned int all: 1;
+};
+#define DEINIT_CB_INIT { NULL, 0, 0, 0 }
+
+static void deinit_submodule(const struct cache_entry *list_item,
+			     void *cb_data)
+{
+	struct deinit_cb *info = cb_data;
+	const struct submodule *sub;
+	char *displaypath = NULL;
+	struct child_process cp_config = CHILD_PROCESS_INIT;
+	struct strbuf sb_config = STRBUF_INIT;
+	char *sub_git_dir = xstrfmt("%s/.git", list_item->name);
+	mode_t mode = 0777;
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub || !sub->name)
+		goto cleanup;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	/* remove the submodule work tree (unless the user already did it) */
+	if (is_directory(list_item->name)) {
+		struct stat st;
+		/*
+		 * protect submodules containing a .git directory
+		 * NEEDSWORK: automatically call absorbgitdirs before
+		 * warning/die.
+		 */
+		if (is_directory(sub_git_dir))
+			die(_("Submodule work tree '%s' contains a .git "
+			      "directory use 'rm -rf' if you really want "
+			      "to remove it including all of its history"),
+			      displaypath);
+
+		if (!info->force) {
+			struct child_process cp_rm = CHILD_PROCESS_INIT;
+			cp_rm.git_cmd = 1;
+			argv_array_pushl(&cp_rm.args, "rm", "-qn",
+					 list_item->name, NULL);
+
+			if (run_command(&cp_rm))
+				die(_("Submodule work tree '%s' contains local "
+				      "modifications; use '-f' to discard them"),
+				      displaypath);
+		}
+
+		if (!lstat(list_item->name, &st)) {
+			struct strbuf sb_rm = STRBUF_INIT;
+			const char *format;
+
+			strbuf_addstr(&sb_rm, list_item->name);
+
+			if (!remove_dir_recursively(&sb_rm, 0))
+				format = _("Cleared directory '%s'\n");
+			else
+				format = _("Could not remove submodule work tree '%s'\n");
+
+			if (!info->quiet)
+				printf(format, displaypath);
+
+			mode = st.st_mode;
+
+			strbuf_release(&sb_rm);
+		}
+	}
+
+	if (mkdir(list_item->name, mode))
+		die_errno(_("could not create empty submodule directory %s"),
+		      displaypath);
+
+	cp_config.git_cmd = 1;
+	argv_array_pushl(&cp_config.args, "config", "--get-regexp", NULL);
+	argv_array_pushf(&cp_config.args, "submodule.%s\\.", sub->name);
+
+	/* remove the .git/config entries (unless the user already did it) */
+	if (!capture_command(&cp_config, &sb_config, 0) && sb_config.len) {
+		char *sub_key = xstrfmt("submodule.%s", sub->name);
+		/*
+		 * remove the whole section so we have a clean state when
+		 * the user later decides to init this submodule again
+		 */
+		git_config_rename_section_in_file(NULL, sub_key, NULL);
+		if (!info->quiet)
+			printf(_("Submodule '%s' (%s) unregistered for path '%s'\n"),
+				 sub->name, sub->url, displaypath);
+		free(sub_key);
+	}
+
+cleanup:
+	free(displaypath);
+	free(sub_git_dir);
+	strbuf_release(&sb_config);
+}
+
+static int module_deinit(int argc, const char **argv, const char *prefix)
+{
+	struct deinit_cb info = DEINIT_CB_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int force = 0;
+	int all = 0;
+
+	struct option module_deinit_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress submodule status output")),
+		OPT__FORCE(&force, N_("Remove submodule working trees even if they contain local changes")),
+		OPT_BOOL(0, "all", &all, N_("Unregister all submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule deinit [--quiet] [-f | --force] [--all | [--] [<path>...]]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_deinit_options,
+			     git_submodule_helper_usage, 0);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.all = !!all;
+	info.force = !!force;
+
+	if (all && argc) {
+		error("pathspec and --all are incompatible");
+		usage_with_options(git_submodule_helper_usage,
+				   module_deinit_options);
+	}
+
+	if (!argc && !all)
+		die(_("Use '--all' if you really want to deinitialize all submodules"));
+
+	gitmodules_config();
+	for_each_submodule_list(list, deinit_submodule, &info);
+
+	return 0;
+}
+
 static int clone_submodule(const char *path, const char *gitdir, const char *url,
 			   const char *depth, struct string_list *reference,
 			   int quiet, int progress)
@@ -1645,6 +1792,7 @@ static struct cmd_struct commands[] = {
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
+	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index 5acf6f404..b34beb8ec 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -428,60 +428,7 @@ cmd_deinit()
 		shift
 	done
 
-	if test -n "$deinit_all" && test "$#" -ne 0
-	then
-		echo >&2 "$(eval_gettext "pathspec and --all are incompatible")"
-		usage
-	fi
-	if test $# = 0 && test -z "$deinit_all"
-	then
-		die "$(eval_gettext "Use '--all' if you really want to deinitialize all submodules")"
-	fi
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" "$@" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		name=$(git submodule--helper name "$sm_path") || exit
-
-		displaypath=$(git submodule--helper relative-path "$sm_path" "$wt_prefix")
-
-		# Remove the submodule work tree (unless the user already did it)
-		if test -d "$sm_path"
-		then
-			# Protect submodules containing a .git directory
-			if test -d "$sm_path/.git"
-			then
-				die "$(eval_gettext "\
-Submodule work tree '\$displaypath' contains a .git directory
-(use 'rm -rf' if you really want to remove it including all of its history)")"
-			fi
-
-			if test -z "$force"
-			then
-				git rm -qn "$sm_path" ||
-				die "$(eval_gettext "Submodule work tree '\$displaypath' contains local modifications; use '-f' to discard them")"
-			fi
-			rm -rf "$sm_path" &&
-			say "$(eval_gettext "Cleared directory '\$displaypath'")" ||
-			say "$(eval_gettext "Could not remove submodule work tree '\$displaypath'")"
-		fi
-
-		mkdir "$sm_path" || say "$(eval_gettext "Could not create empty submodule directory '\$displaypath'")"
-
-		# Remove the .git/config entries (unless the user already did it)
-		if test -n "$(git config --get-regexp submodule."$name\.")"
-		then
-			# Remove the whole section so we have a clean state when
-			# the user later decides to init this submodule again
-			url=$(git config submodule."$name".url)
-			git config --remove-section submodule."$name" 2>/dev/null &&
-			say "$(eval_gettext "Submodule '\$name' (\$url) unregistered for path '\$displaypath'")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${force:+--force} ${deinit_all:+--all} "$@"
 }
 
 is_tip_reachable () (
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' from shell to C
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (5 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 06/13] submodule: port submodule subcommand 'deinit' " Prathamesh Chavan
@ 2017-08-07 21:18 ` " Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

The submodule subcommand 'summary' is ported in the process of
making git-submodule a builtin. The function cmd_summary() from
git-submodule.sh is ported to functions module_summary(),
compute_summary_module_list(), prepare_submodule_summary() and
generate_submodule_summary(), print_summary().

The first function module_summary() parses the options of submodule
subcommand and also acts as the front-end of this subcommand.
After parsing them, it calls the compute_summary_module_list()

The functions compute_summary_module_list() runs the diff_cmd,
and generates the modules list, as required by the subcommand.
The generation of this module list is done by the using the
callback function submodule_summary_callback(), and stored in the
structure module_cb.

Once the module list is generated, prepare_submodule_summary()
further goes through the list and filters the list, for
eventually calling the generate_submodule_summary() function.

The function generate_submodule_summary() takes care of generating
the summary for each submodule and then calls the function
print_summary() for printing it.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
The major changes the patch underwent through is the splitting of the
function print_submodule_summary() into generate_submodule_summary()
and print_summary()

Apart from this, there are also minor changes which include
removal of variables sha1_abbr_dst and sha1_abbr_src,
the variable remote_key wasn't freed earlier, but now it is.

 builtin/submodule--helper.c | 428 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            | 183 +------------------
 2 files changed, 429 insertions(+), 182 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 02787e4a4..2080f4fb9 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -13,6 +13,9 @@
 #include "remote.h"
 #include "refs.h"
 #include "connect.h"
+#include "revision.h"
+#include "diffcore.h"
+#include "diff.h"
 
 typedef void (*submodule_list_func_t)(const struct cache_entry *list_item,
 				      void *cb_data);
@@ -767,6 +770,430 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct module_cb {
+	unsigned int mod_src;
+	unsigned int mod_dst;
+	struct object_id oid_src;
+	struct object_id oid_dst;
+	char status;
+	const char *sm_path;
+};
+#define MODULE_CB_INIT { 0, 0, NULL, NULL, '\0', NULL }
+
+struct module_cb_list {
+	struct module_cb **entries;
+	int alloc, nr;
+};
+#define MODULE_CB_LIST_INIT { NULL, 0, 0 }
+
+struct summary_cb {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	unsigned int cached: 1;
+	unsigned int for_status: 1;
+	unsigned int quiet: 1;
+	unsigned int files: 1;
+	int summary_limits;
+};
+#define SUMMARY_CB_INIT { 0, NULL, NULL, 0, 0, 0, 0, 0 }
+
+static enum {
+	DIFF_INDEX,
+	DIFF_FILES
+} diff_cmd = DIFF_INDEX;
+
+static int verify_submodule_object_name(const char *sm_path, const char *sha1)
+{
+	struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+	cp_rev_parse.git_cmd = 1;
+	cp_rev_parse.no_stdout = 1;
+	cp_rev_parse.dir = sm_path;
+	prepare_submodule_repo_env(&cp_rev_parse.env_array);
+	argv_array_pushl(&cp_rev_parse.args, "rev-parse", "-q",
+			 "--verify", NULL);
+	argv_array_pushf(&cp_rev_parse.args, "%s^0", sha1);
+
+	if (run_command(&cp_rev_parse))
+		return 1;
+
+	return 0;
+}
+
+static void print_summary(struct summary_cb *info, int errmsg, int total_commits,
+			  int missing_src, int missing_dst, const char *displaypath,
+			  int is_sm_git_dir, struct module_cb *p)
+{
+	if (p->status == 'T') {
+		if (S_ISGITLINK(p->mod_dst))
+			printf(_("* %s %s(blob)->%s(submodule)"),
+				 displaypath, find_unique_abbrev(p->oid_src.hash, 7),
+				 find_unique_abbrev(p->oid_dst.hash, 7));
+		else
+			printf(_("* %s %s(submodule)->%s(blob)"),
+				 displaypath, find_unique_abbrev(p->oid_src.hash, 7),
+				 find_unique_abbrev(p->oid_dst.hash, 7));
+	} else {
+			printf("* %s %s...%s", displaypath, find_unique_abbrev(p->oid_src.hash, 7),
+				 find_unique_abbrev(p->oid_dst.hash, 7));
+	}
+
+	if (total_commits < 0)
+		printf(":\n");
+	else
+		printf(" (%d):\n", total_commits);
+
+	if (errmsg) {
+		/*
+		 * Don't give error msg for modification whose dst is not
+		 * submodule, i.e. deleted or changed to blob
+		 */
+		if (S_ISGITLINK(p->mod_src)) {
+			if (missing_src && missing_dst) {
+				printf(_("  Warn: %s doesn't contain commits %s and %s\n"),
+				 displaypath, oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+			} else if (missing_src) {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, oid_to_hex(&p->oid_src));
+			} else {
+				printf(_("  Warn: %s doesn't contain commit %s\n"),
+				 displaypath, oid_to_hex(&p->oid_dst));
+			}
+		}
+	} else if (is_sm_git_dir) {
+		struct child_process cp_log = CHILD_PROCESS_INIT;
+
+		cp_log.git_cmd = 1;
+		cp_log.dir = p->sm_path;
+		prepare_submodule_repo_env(&cp_log.env_array);
+		argv_array_pushl(&cp_log.args, "log", NULL);
+
+		if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst)) {
+			if (info->summary_limits > 0)
+				argv_array_pushf(&cp_log.args, "-%d", info->summary_limits);
+
+			argv_array_pushl(&cp_log.args, "--pretty=  %m %s",
+					 "--first-parent", NULL);
+			argv_array_pushf(&cp_log.args, "%s...%s", oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+		} else if (S_ISGITLINK(p->mod_dst)) {
+			argv_array_pushl(&cp_log.args, "--pretty=  > %s",
+					 "-1", oid_to_hex(&p->oid_dst), NULL);
+		} else {
+			argv_array_pushl(&cp_log.args, "--pretty=  < %s",
+					 "-1", oid_to_hex(&p->oid_src), NULL);
+		}
+		run_command(&cp_log);
+	}
+	printf("\n");
+}
+
+static void generate_submodule_summary(struct summary_cb *info,
+				       struct module_cb *p)
+{
+	int missing_src = 0;
+	int missing_dst = 0;
+	char *displaypath;
+	int errmsg = 0;
+	int total_commits = -1;
+	char *sm_git_dir = xstrfmt("%s/.git", p->sm_path);
+	int is_sm_git_dir = 0;
+
+	if (!info->cached && !oidcmp(&p->oid_dst, &null_oid)) {
+		if (S_ISGITLINK(p->mod_dst)) {
+			/*
+			 * NEEDSWORK: avoid using separate process with
+			 * the help of the function head_ref_submodule()
+			 */
+			struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_parse = STRBUF_INIT;
+
+			cp_rev_parse.git_cmd = 1;
+			cp_rev_parse.no_stderr = 1;
+			cp_rev_parse.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_parse.env_array);
+
+			argv_array_pushl(&cp_rev_parse.args,
+					 "rev-parse", "HEAD", NULL);
+			if (!capture_command(&cp_rev_parse, &sb_rev_parse, 0)) {
+				strbuf_strip_suffix(&sb_rev_parse, "\n");
+
+				get_oid_hex(sb_rev_parse.buf, &p->oid_dst);
+			}
+			strbuf_release(&sb_rev_parse);
+		} else if (S_ISLNK(p->mod_dst) || S_ISREG(p->mod_dst)) {
+			struct child_process cp_hash_object = CHILD_PROCESS_INIT;
+			struct strbuf sb_hash_object = STRBUF_INIT;
+
+			cp_hash_object.git_cmd = 1;
+			argv_array_pushl(&cp_hash_object.args,
+					 "hash-object", p->sm_path,
+					 NULL);
+			if (!capture_command(&cp_hash_object,
+					     &sb_hash_object, 0)) {
+				strbuf_strip_suffix(&sb_hash_object, "\n");
+
+				get_oid_hex(sb_hash_object.buf, &p->oid_dst);
+			}
+			strbuf_release(&sb_hash_object);
+		} else {
+			if (p->mod_dst)
+				die(_("unexpected mode %d\n"), p->mod_dst);
+		}
+	}
+
+	if (is_git_directory(sm_git_dir))
+		is_sm_git_dir = 1;
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_src))
+		missing_src = verify_submodule_object_name(p->sm_path,
+							   oid_to_hex(&p->oid_src));
+
+	if (is_sm_git_dir && S_ISGITLINK(p->mod_dst))
+		missing_dst = verify_submodule_object_name(p->sm_path,
+							   oid_to_hex(&p->oid_dst));
+
+	displaypath = get_submodule_displaypath(p->sm_path, info->prefix);
+
+	if (!missing_dst && !missing_src) {
+		if (is_sm_git_dir) {
+			struct child_process cp_rev_list = CHILD_PROCESS_INIT;
+			struct strbuf sb_rev_list = STRBUF_INIT;
+			char *range;
+
+			if (S_ISGITLINK(p->mod_src) && S_ISGITLINK(p->mod_dst))
+				range = xstrfmt("%s...%s", oid_to_hex(&p->oid_src), oid_to_hex(&p->oid_dst));
+			else if (S_ISGITLINK(p->mod_src))
+				range = xstrdup(oid_to_hex(&p->oid_src));
+			else
+				range = xstrdup(oid_to_hex(&p->oid_dst));
+
+			cp_rev_list.git_cmd = 1;
+			cp_rev_list.dir = p->sm_path;
+			prepare_submodule_repo_env(&cp_rev_list.env_array);
+
+			argv_array_pushl(&cp_rev_list.args, "rev-list",
+					 "--first-parent", range, "--", NULL);
+			if (!capture_command(&cp_rev_list, &sb_rev_list, 0)) {
+				if (sb_rev_list.len)
+					total_commits = count_lines(sb_rev_list.buf,
+								    sb_rev_list.len);
+				else
+					total_commits = 0;
+			}
+
+			free(range);
+			strbuf_release(&sb_rev_list);
+		}
+	} else {
+		errmsg = 1;
+	}
+
+	print_summary(info, errmsg, total_commits, missing_src, missing_dst,
+		      displaypath, is_sm_git_dir, p);
+
+	free(displaypath);
+	free(sm_git_dir);
+}
+
+static void prepare_submodule_summary(struct summary_cb *info,
+				      struct module_cb_list *list)
+{
+	int i;
+	for (i = 0; i < list->nr; i++) {
+		struct module_cb *p = list->entries[i];
+		struct child_process cp_rev_parse = CHILD_PROCESS_INIT;
+
+		if (p->status == 'D' || p->status == 'T') {
+			generate_submodule_summary(info, p);
+			continue;
+		}
+
+		if (info->for_status) {
+			char *config_key;
+			const char *ignore_config = "none";
+			const char *value;
+			const struct submodule *sub = submodule_from_path(null_sha1, p->sm_path);
+
+			if (sub && p->status != 'A') {
+				config_key = xstrfmt("submodule.%s.ignore",
+						     sub->name);
+				if (!git_config_get_string_const(config_key, &value))
+					ignore_config = value;
+				else if (sub->ignore)
+					ignore_config = sub->ignore;
+
+				free(config_key);
+				if (!strcmp(ignore_config, "all"))
+					continue;
+			}
+		}
+
+		/* Also show added or modified modules which are checked out */
+		cp_rev_parse.dir = p->sm_path;
+		cp_rev_parse.git_cmd = 1;
+		cp_rev_parse.no_stderr = 1;
+		cp_rev_parse.no_stdout = 1;
+
+		argv_array_pushl(&cp_rev_parse.args, "rev-parse",
+				 "--git-dir", NULL);
+
+		if (!run_command(&cp_rev_parse))
+			generate_submodule_summary(info, p);
+	}
+}
+
+static void submodule_summary_callback(struct diff_queue_struct *q,
+				       struct diff_options *options,
+				       void *data)
+{
+	int i;
+	struct module_cb_list *list = data;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		struct module_cb *temp;
+
+		if (!S_ISGITLINK(p->one->mode) && !S_ISGITLINK(p->two->mode))
+			continue;
+		temp = (struct module_cb*)malloc(sizeof(struct module_cb));
+		temp->mod_src = p->one->mode;
+		temp->mod_dst = p->two->mode;
+		temp->oid_src = p->one->oid;
+		temp->oid_dst = p->two->oid;
+		temp->status = p->status;
+		temp->sm_path = xstrdup(p->one->path);
+
+		ALLOC_GROW(list->entries, list->nr + 1, list->alloc);
+		list->entries[list->nr++] = temp;
+	}
+}
+
+static int compute_summary_module_list(char *head, struct summary_cb *info)
+{
+	struct argv_array diff_args = ARGV_ARRAY_INIT;
+	struct rev_info rev;
+	struct module_cb_list list = MODULE_CB_LIST_INIT;
+
+	argv_array_push(&diff_args, diff_cmd ? "diff-files" : "diff-index");
+	if (info->cached)
+		argv_array_push(&diff_args, "--cached");
+	argv_array_pushl(&diff_args, "--ignore-submodules=dirty", "--raw",
+			 NULL);
+	if (head)
+		argv_array_push(&diff_args, head);
+	argv_array_push(&diff_args, "--");
+	if (info->argc)
+		argv_array_pushv(&diff_args, info->argv);
+
+	git_config(git_diff_basic_config, NULL);
+	init_revisions(&rev, info->prefix);
+	gitmodules_config();
+	rev.abbrev = 0;
+	precompose_argv(diff_args.argc, diff_args.argv);
+
+	diff_args.argc = setup_revisions(diff_args.argc, diff_args.argv,
+					 &rev, NULL);
+	rev.diffopt.output_format = DIFF_FORMAT_NO_OUTPUT | DIFF_FORMAT_CALLBACK;
+	rev.diffopt.format_callback = submodule_summary_callback;
+	rev.diffopt.format_callback_data = &list;
+
+	if (!info->cached) {
+		if (!diff_cmd)
+			setup_work_tree();
+		if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
+			perror("read_cache_preload");
+			return -1;
+		}
+	} else if (read_cache() < 0) {
+		perror("read_cache");
+		return -1;
+	}
+
+	if (!diff_cmd)
+		run_diff_index(&rev, info->cached);
+	else
+		run_diff_files(&rev, 0);
+	prepare_submodule_summary(info, &list);
+	return 0;
+}
+
+static int module_summary(int argc, const char **argv, const char *prefix)
+{
+	struct summary_cb info = SUMMARY_CB_INIT;
+	int cached = 0;
+	int for_status = 0;
+	int quiet = 0;
+	int files = 0;
+	int summary_limits = -1;
+	struct child_process cp_rev = CHILD_PROCESS_INIT;
+	struct strbuf sb = STRBUF_INIT;
+	int ret;
+
+	struct option module_summary_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output for initializing a submodule")),
+		OPT_BOOL(0, "cached", &cached, N_("Use the commit stored in the index instead of the submodule HEAD")),
+		OPT_BOOL(0, "files", &files, N_("To compares the commit in the index with that in the submodule HEAD")),
+		OPT_BOOL(0, "for-status", &for_status, N_("Skip submodules with 'all' ignore_config value")),
+		OPT_INTEGER('n', "summary-limits", &summary_limits, N_("Limit the summary size")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper summary [<options>] [--] [<path>]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_summary_options,
+			     git_submodule_helper_usage, 0);
+
+	if (!summary_limits)
+		return 0;
+
+	cp_rev.git_cmd = 1;
+	argv_array_pushl(&cp_rev.args, "rev-parse", "-q", "--verify",
+			 argc ? argv[0] : "HEAD", NULL);
+
+	if (!capture_command(&cp_rev, &sb, 0)) {
+		strbuf_strip_suffix(&sb, "\n");
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else if (!argc || !strcmp(argv[0], "HEAD")) {
+		/* before the first commit: compare with an empty tree */
+		struct stat st;
+		struct object_id oid;
+		if (fstat(0, &st) < 0 || index_fd(oid.hash, 0, &st, 2, prefix, 3))
+			die("Unable to add %s to database", oid.hash);
+		strbuf_addstr(&sb, oid_to_hex(&oid));
+		if (argc) {
+			argv++;
+			argc--;
+		}
+	} else {
+		strbuf_addstr(&sb, "HEAD");
+	}
+
+	if (files) {
+		if (cached)
+			die(_("The --cached option cannot be used with the --files option"));
+		diff_cmd++;
+	}
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.cached = !!cached;
+	info.for_status = !!for_status;
+	info.quiet = quiet;
+	info.files = files;
+	info.summary_limits = summary_limits;
+
+	ret = compute_summary_module_list(diff_cmd ? NULL : sb.buf, &info);
+	strbuf_release(&sb);
+	return ret;
+}
+
 struct sync_cb {
 	const char *prefix;
 	unsigned int quiet: 1;
@@ -1793,6 +2220,7 @@ static struct cmd_struct commands[] = {
 	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, SUPPORT_SUPER_PREFIX},
+	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
diff --git a/git-submodule.sh b/git-submodule.sh
index b34beb8ec..ec57d6528 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -51,31 +51,6 @@ die_if_unmatched ()
 	fi
 }
 
-#
-# Print a submodule configuration setting
-#
-# $1 = submodule name
-# $2 = option name
-# $3 = default value
-#
-# Checks in the usual git-config places first (for overrides),
-# otherwise it falls back on .gitmodules.  This allows you to
-# distribute project-wide defaults in .gitmodules, while still
-# customizing individual repositories if necessary.  If the option is
-# not in .gitmodules either, print a default value.
-#
-get_submodule_config () {
-	name="$1"
-	option="$2"
-	default="$3"
-	value=$(git config submodule."$name"."$option")
-	if test -z "$value"
-	then
-		value=$(git config -f .gitmodules submodule."$name"."$option")
-	fi
-	printf '%s' "${value:-$default}"
-}
-
 isnumber()
 {
 	n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
@@ -755,163 +730,7 @@ cmd_summary() {
 		shift
 	done
 
-	test $summary_limit = 0 && return
-
-	if rev=$(git rev-parse -q --verify --default HEAD ${1+"$1"})
-	then
-		head=$rev
-		test $# = 0 || shift
-	elif test -z "$1" || test "$1" = "HEAD"
-	then
-		# before the first commit: compare with an empty tree
-		head=$(git hash-object -w -t tree --stdin </dev/null)
-		test -z "$1" || shift
-	else
-		head="HEAD"
-	fi
-
-	if [ -n "$files" ]
-	then
-		test -n "$cached" &&
-		die "$(gettext "The --cached option cannot be used with the --files option")"
-		diff_cmd=diff-files
-		head=
-	fi
-
-	cd_to_toplevel
-	eval "set $(git rev-parse --sq --prefix "$wt_prefix" -- "$@")"
-	# Get modified modules cared by user
-	modules=$(git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- "$@" |
-		sane_egrep '^:([0-7]* )?160000' |
-		while read -r mod_src mod_dst sha1_src sha1_dst status sm_path
-		do
-			# Always show modules deleted or type-changed (blob<->module)
-			if test "$status" = D || test "$status" = T
-			then
-				printf '%s\n' "$sm_path"
-				continue
-			fi
-			# Respect the ignore setting for --for-status.
-			if test -n "$for_status"
-			then
-				name=$(git submodule--helper name "$sm_path")
-				ignore_config=$(get_submodule_config "$name" ignore none)
-				test $status != A && test $ignore_config = all && continue
-			fi
-			# Also show added or modified modules which are checked out
-			GIT_DIR="$sm_path/.git" git-rev-parse --git-dir >/dev/null 2>&1 &&
-			printf '%s\n' "$sm_path"
-		done
-	)
-
-	test -z "$modules" && return
-
-	git $diff_cmd $cached --ignore-submodules=dirty --raw $head -- $modules |
-	sane_egrep '^:([0-7]* )?160000' |
-	cut -c2- |
-	while read -r mod_src mod_dst sha1_src sha1_dst status name
-	do
-		if test -z "$cached" &&
-			test $sha1_dst = 0000000000000000000000000000000000000000
-		then
-			case "$mod_dst" in
-			160000)
-				sha1_dst=$(GIT_DIR="$name/.git" git rev-parse HEAD)
-				;;
-			100644 | 100755 | 120000)
-				sha1_dst=$(git hash-object $name)
-				;;
-			000000)
-				;; # removed
-			*)
-				# unexpected type
-				eval_gettextln "unexpected mode \$mod_dst" >&2
-				continue ;;
-			esac
-		fi
-		missing_src=
-		missing_dst=
-
-		test $mod_src = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_src^0 >/dev/null &&
-		missing_src=t
-
-		test $mod_dst = 160000 &&
-		! GIT_DIR="$name/.git" git-rev-parse -q --verify $sha1_dst^0 >/dev/null &&
-		missing_dst=t
-
-		display_name=$(git submodule--helper relative-path "$name" "$wt_prefix")
-
-		total_commits=
-		case "$missing_src,$missing_dst" in
-		t,)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_src")"
-			;;
-		,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commit \$sha1_dst")"
-			;;
-		t,t)
-			errmsg="$(eval_gettext "  Warn: \$display_name doesn't contain commits \$sha1_src and \$sha1_dst")"
-			;;
-		*)
-			errmsg=
-			total_commits=$(
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				range="$sha1_src...$sha1_dst"
-			elif test $mod_src = 160000
-			then
-				range=$sha1_src
-			else
-				range=$sha1_dst
-			fi
-			GIT_DIR="$name/.git" \
-			git rev-list --first-parent $range -- | wc -l
-			)
-			total_commits=" ($(($total_commits + 0)))"
-			;;
-		esac
-
-		sha1_abbr_src=$(echo $sha1_src | cut -c1-7)
-		sha1_abbr_dst=$(echo $sha1_dst | cut -c1-7)
-		if test $status = T
-		then
-			blob="$(gettext "blob")"
-			submodule="$(gettext "submodule")"
-			if test $mod_dst = 160000
-			then
-				echo "* $display_name $sha1_abbr_src($blob)->$sha1_abbr_dst($submodule)$total_commits:"
-			else
-				echo "* $display_name $sha1_abbr_src($submodule)->$sha1_abbr_dst($blob)$total_commits:"
-			fi
-		else
-			echo "* $display_name $sha1_abbr_src...$sha1_abbr_dst$total_commits:"
-		fi
-		if test -n "$errmsg"
-		then
-			# Don't give error msg for modification whose dst is not submodule
-			# i.e. deleted or changed to blob
-			test $mod_dst = 160000 && echo "$errmsg"
-		else
-			if test $mod_src = 160000 && test $mod_dst = 160000
-			then
-				limit=
-				test $summary_limit -gt 0 && limit="-$summary_limit"
-				GIT_DIR="$name/.git" \
-				git log $limit --pretty='format:  %m %s' \
-				--first-parent $sha1_src...$sha1_dst
-			elif test $mod_dst = 160000
-			then
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  > %s' -1 $sha1_dst
-			else
-				GIT_DIR="$name/.git" \
-				git log --pretty='format:  < %s' -1 $sha1_src
-			fi
-			echo
-		fi
-		echo
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${for_status:+--for-status} ${files:+--files} ${cached:+--cached} ${summary_limit:+-n $summary_limit} "$@"
 }
 #
 # List all submodules, prefixed with:
-- 
2.13.0


^ permalink raw reply	[relevance 15%]

* [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (6 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 08/13] submodule: port submodule subcommand 'summary' " Prathamesh Chavan
@ 2017-08-07 21:18 ` Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

When running 'git submodule foreach' from a subdirectory of your
repository, nested submodules get a bogus value for $sm_path:
For a submodule 'sub' that contains a nested submodule 'nested',
running 'git -C dir submodule foreach echo $path' would report
path='../nested' for the nested submodule. The first part '../' is
derived from the logic computing the relative path from $pwd to the
root of the superproject. The second part is the submodule path inside
the submodule. This value is of little use and is hard to document.

There are two different possible solutions that have more value:
(a) The path value is documented as the path from the toplevel of the
    superproject to the mount point of the submodule.
    In this case we would want to have path='sub/nested'.

(b) As Ramsay noticed the documented value is wrong. For the non-nested
    case the path is equal to the relative path from $pwd to the
    submodules working directory. When following this model,
    the expected value would be path='../sub/nested'.

The behavior for (b) was introduced in 091a6eb0fe (submodule: drop the
top-level requirement, 2013-06-16) the intent for $path seemed to be
relative to $cwd to the submodule worktree, but that did not work for
nested submodules, as the intermittent submodules were not included in
the path.

If we were to fix the meaning of the $path using (a) such that "path"
is "the path from the toplevel of the superproject to the mount point
of the submodule", we would break any existing submodule user that runs
foreach from non-root of the superproject as the non-nested submodule
'../sub' would change its path to 'sub'.

If we would fix the meaning of the $path using (b), such that "path"
is "the relative path from $pwd to the submodule", then we would break
any user that uses nested submodules (even from the root directory) as
the 'nested' would become 'sub/nested'.

Both groups can be found in the wild.  The author has no data if one group
outweighs the other by large margin, and offending each one seems equally
bad at first.  However in the authors imagination it is better to go with
(a) as running from a sub directory sounds like it is carried out
by a human rather than by some automation task.  With a human on
the keyboard the feedback loop is short and the changed behavior can be
adapted to quickly unlike some automation that can break silently.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh             |  1 -
 t/t7407-submodule-foreach.sh | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index ec57d6528..4b7da2fc1 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -320,7 +320,6 @@ cmd_foreach()
 				prefix="$prefix$sm_path/"
 				sanitize_submodule_env
 				cd "$sm_path" &&
-				sm_path=$(git submodule--helper relative-path "$sm_path" "$wt_prefix") &&
 				# we make $path available to scripts ...
 				path=$sm_path &&
 				if test $# -eq 1
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 6ba5daf42..0663622a4 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,9 +82,9 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-../sub1-$sub1sha1
+$pwd/clone-foo1-sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-../sub3-$sub3sha1
+$pwd/clone-foo3-sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
@@ -196,6 +196,38 @@ test_expect_success 'test messages from "foreach --recursive" from subdirectory'
 	) &&
 	test_i18ncmp expect actual
 '
+sub1sha1=$(cd clone2/sub1 && git rev-parse HEAD)
+sub2sha1=$(cd clone2/sub2 && git rev-parse HEAD)
+sub3sha1=$(cd clone2/sub3 && git rev-parse HEAD)
+nested1sha1=$(cd clone2/nested1 && git rev-parse HEAD)
+nested2sha1=$(cd clone2/nested1/nested2 && git rev-parse HEAD)
+nested3sha1=$(cd clone2/nested1/nested2/nested3 && git rev-parse HEAD)
+submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEAD)
+
+cat >expect <<EOF
+Entering '../nested1'
+$pwd/clone2-nested1-nested1-$nested1sha1
+Entering '../nested1/nested2'
+$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+Entering '../nested1/nested2/nested3'
+$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+Entering '../nested1/nested2/nested3/submodule'
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+Entering '../sub1'
+$pwd/clone2-foo1-sub1-$sub1sha1
+Entering '../sub2'
+$pwd/clone2-foo2-sub2-$sub2sha1
+Entering '../sub3'
+$pwd/clone2-foo3-sub3-$sub3sha1
+EOF
+
+test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
+	(
+		cd clone2/untracked &&
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+	) &&
+	test_i18ncmp expect actual
+'
 
 cat > expect <<EOF
 nested1-nested1
-- 
2.13.0


^ permalink raw reply	[relevance 22%]

* [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path'
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (7 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 09/13] submodule foreach: correct '$path' in nested submodules from a subdirectory Prathamesh Chavan
@ 2017-08-07 21:18 ` Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

As using a variable '$path' may be harmful to users due to
capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
use $path variable in eval_gettext string, 2012-04-17). Adjust
the documentation to advocate for using $sm_path,  which contains
the same value. We still make the 'path' variable available and
document it as a deprecated synonym of 'sm_path'.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index ff612001d..a23baef62 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,12 +183,14 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $path, $sha1 and
+	The command has access to the variables $name, $sm_path, $sha1 and
 	$toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
-	$path is the name of the submodule directory relative to the
-	superproject, $sha1 is the commit as recorded in the superproject,
-	and $toplevel is the absolute path to the top-level of the superproject.
+	$sm_path is the path of the submodule as recorded in the superproject,
+	$sha1 is the commit as recorded in the superproject, and
+	$toplevel is the absolute path to the top-level of the superproject.
+	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
+	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given `--quiet`, foreach prints the name
 	of each submodule before evaluating the command.
-- 
2.13.0


^ permalink raw reply	[relevance 23%]

* [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (8 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 10/13] submodule foreach: document '$sm_path' instead of '$path' Prathamesh Chavan
@ 2017-08-07 21:18 ` Prathamesh Chavan
  2017-08-07 21:18 ` [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
  2017-08-07 21:19 ` [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

It does not contain the topmost superproject as the author assumed,
but the direct superproject, such that $toplevel/$sm_path is the
actual absolute path of the submodule.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index a23baef62..8e7930ebc 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -188,7 +188,8 @@ foreach [--recursive] <command>::
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
 	$sha1 is the commit as recorded in the superproject, and
-	$toplevel is the absolute path to the top-level of the superproject.
+	$toplevel is the absolute path to its superproject, such that
+	$toplevel/$sm_path is the absolute path of the submodule.
 	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
 	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
-- 
2.13.0


^ permalink raw reply	[relevance 24%]

* [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath'
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (9 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 11/13] submodule foreach: clarify the '$toplevel' variable documentation Prathamesh Chavan
@ 2017-08-07 21:18 ` Prathamesh Chavan
  2017-08-07 21:19 ` [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:18 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

It was observed that the variable '$displaypath' was accessible but
undocumented. Hence, document it.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
 Documentation/git-submodule.txt |  6 ++++--
 t/t7407-submodule-foreach.sh    | 22 +++++++++++-----------
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 8e7930ebc..0cca702cb 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,10 +183,12 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $sm_path, $sha1 and
-	$toplevel:
+	The command has access to the variables $name, $sm_path, $displaypath,
+	$sha1 and $toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
+	$displaypath contains the relative path from the current working
+	directory to the submodules root directory,
 	$sha1 is the commit as recorded in the superproject, and
 	$toplevel is the absolute path to its superproject, such that
 	$toplevel/$sm_path is the absolute path of the submodule.
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 0663622a4..6ad57e061 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,16 +82,16 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-sub1-$sub1sha1
+$pwd/clone-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-sub3-$sub3sha1
+$pwd/clone-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
 	mkdir clone/sub &&
 	(
 		cd clone/sub &&
-		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
@@ -206,25 +206,25 @@ submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEA
 
 cat >expect <<EOF
 Entering '../nested1'
-$pwd/clone2-nested1-nested1-$nested1sha1
+$pwd/clone2-nested1-nested1-../nested1-$nested1sha1
 Entering '../nested1/nested2'
-$pwd/clone2/nested1-nested2-nested2-$nested2sha1
+$pwd/clone2/nested1-nested2-nested2-../nested1/nested2-$nested2sha1
 Entering '../nested1/nested2/nested3'
-$pwd/clone2/nested1/nested2-nested3-nested3-$nested3sha1
+$pwd/clone2/nested1/nested2-nested3-nested3-../nested1/nested2/nested3-$nested3sha1
 Entering '../nested1/nested2/nested3/submodule'
-$pwd/clone2/nested1/nested2/nested3-submodule-submodule-$submodulesha1
+$pwd/clone2/nested1/nested2/nested3-submodule-submodule-../nested1/nested2/nested3/submodule-$submodulesha1
 Entering '../sub1'
-$pwd/clone2-foo1-sub1-$sub1sha1
+$pwd/clone2-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub2'
-$pwd/clone2-foo2-sub2-$sub2sha1
+$pwd/clone2-foo2-sub2-../sub2-$sub2sha1
 Entering '../sub3'
-$pwd/clone2-foo3-sub3-$sub3sha1
+$pwd/clone2-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
 	(
 		cd clone2/untracked &&
-		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach --recursive "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
-- 
2.13.0


^ permalink raw reply	[relevance 21%]

* [GSoC][PATCH 13/13] submodule: port submodule subcommand 'foreach' from shell to C
  2017-08-07 21:18 [GSoC][PATCH 00/13] Update: Week-12 Prathamesh Chavan
                   ` (10 preceding siblings ...)
  2017-08-07 21:18 ` [GSoC][PATCH 12/13] submodule foreach: document variable '$displaypath' Prathamesh Chavan
@ 2017-08-07 21:19 ` Prathamesh Chavan
  11 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-07 21:19 UTC (permalink / raw)
  To: git; +Cc: sbeller, christian.couder, Prathamesh Chavan

This aims to make git-submodule foreach a builtin. This is the very
first step taken in this direction. Hence, 'foreach' is ported to
submodule--helper, and submodule--helper is called from git-submodule.sh.
The code is split up to have one function to obtain all the list of
submodules. This function acts as the front-end of git-submodule foreach
subcommand. It calls the function for_each_submodule_list, which basically
loops through the list and calls function fn, which in this case is
runcommand_in_submodule. This third function is a calling function that
takes care of running the command in that submodule, and recursively
perform the same when --recursive is flagged.

The first function module_foreach first parses the options present in
argv, and then with the help of module_list_compute, generates the list of
submodules present in the current working tree.

The second function for_each_submodule_list traverses through the
list, and calls function fn (which in case of submodule subcommand
foreach is runcommand_in_submodule) is called for each entry.

The third function runcommand_in_submodule, generates a submodule struct sub
for $name, value and then later prepends name=sub->name; and other
value assignment to the env argv_array structure of a child_process.
Also the <command> of submodule-foreach is push to args argv_array
structure and finally, using run_command the commands are executed
using a shell.

The third function also takes care of the recursive flag, by creating
a separate child_process structure and prepending "--super-prefix displaypath",
to the args argv_array structure. Other required arguments and the
input <command> of submodule-foreach is also appended to this argv_array.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
---
In this new version, the following changes have been made:
* A comment was added to clarify why the env variables were made
  available only for the case of argc == 1.

 builtin/submodule--helper.c | 142 ++++++++++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  39 +-----------
 2 files changed, 143 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2080f4fb9..0717ecf80 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -770,6 +770,147 @@ static int module_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+struct cb_foreach {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	unsigned int quiet: 1;
+	unsigned int recursive: 1;
+};
+#define CB_FOREACH_INIT { 0, NULL, NULL, 0, 0 }
+
+static void runcommand_in_submodule(const struct cache_entry *list_item,
+				    void *cb_data)
+{
+	struct cb_foreach *info = cb_data;
+	const struct submodule *sub;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *displaypath;
+
+	displaypath = get_submodule_displaypath(list_item->name, info->prefix);
+
+	sub = submodule_from_path(null_sha1, list_item->name);
+
+	if (!sub)
+		die(_("No url found for submodule path '%s' in .gitmodules"),
+		      displaypath);
+
+	if (!is_submodule_populated_gently(list_item->name, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+
+	/*
+	 * For the purpose of executing <command> in the submodule,
+	 * separate shell is used for the purpose of running the
+	 * child process.
+	 */
+	cp.use_shell = 1;
+	cp.dir = list_item->name;
+
+	/*
+	 * NEEDSWORK: the command currently has access to the variables $name,
+	 * $sm_path, $displaypath, $sha1 and $toplevel only when the command
+	 * contains a single argument. This is done for maintianing a faithful
+	 * translation from shell script.
+	 */
+	if (info->argc == 1) {
+		char *toplevel = xgetcwd();
+
+		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
+		argv_array_pushf(&cp.env_array, "sm_path=%s", list_item->name);
+		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
+		argv_array_pushf(&cp.env_array, "sha1=%s",
+				 oid_to_hex(&list_item->oid));
+		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
+
+		/*
+		 * Since the path variable was accessible from the script
+		 * before porting, it is also made available after porting.
+		 * The environment variable "PATH" has a very special purpose
+		 * on windows. And since environment variables are
+		 * case-insensitive in windows, it interferes with the
+		 * existing PATH variable. Hence, to avoid that, we expose
+		 * path via the args argv_array and not via env_array.
+		 */
+		argv_array_pushf(&cp.args, "path=%s; %s",
+				 list_item->name, info->argv[0]);
+		free(toplevel);
+	} else {
+		argv_array_pushv(&cp.args, info->argv);
+	}
+
+	if (!info->quiet)
+		printf(_("Entering '%s'\n"), displaypath);
+
+	if (info->argv[0] && run_command(&cp))
+		die(_("run_command returned non-zero status for %s\n."),
+		      displaypath);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = list_item->name;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", displaypath,
+				 "submodule--helper", "foreach", "--recursive",
+				 NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		argv_array_pushv(&cpr.args, info->argv);
+
+		if (run_command(&cpr))
+			die(_("run_command returned non-zero status while"
+			      "recursing in the nested submodules of %s\n."),
+			      displaypath);
+	}
+
+cleanup:
+	free(displaypath);
+}
+
+static int module_foreach(int argc, const char **argv, const char *prefix)
+{
+	struct cb_foreach info;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	int quiet = 0;
+	int recursive = 0;
+
+	struct option module_foreach_options[] = {
+		OPT__QUIET(&quiet, N_("Suppress output of entering each submodule command")),
+		OPT_BOOL(0, "recursive", &recursive,
+			 N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_foreach_options,
+			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
+		BUG("module_list_compute should not choke on empty pathspec");
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+	info.quiet = !!quiet;
+	info.recursive = !!recursive;
+
+	gitmodules_config();
+	for_each_submodule_list(list, runcommand_in_submodule, &info);
+
+	return 0;
+}
+
 struct module_cb {
 	unsigned int mod_src;
 	unsigned int mod_dst;
@@ -2215,6 +2356,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"print-name-rev", print_name_rev, 0},
+	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 4b7da2fc1..ccd30fdc7 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -298,44 +298,7 @@ cmd_foreach()
 		shift
 	done
 
-	toplevel=$(pwd)
-
-	# dup stdin so that it can be restored when running the external
-	# command in the subshell (and a recursive call to this function)
-	exec 3<&0
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		if test -e "$sm_path"/.git
-		then
-			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-			say "$(eval_gettext "Entering '\$displaypath'")"
-			name=$(git submodule--helper name "$sm_path")
-			(
-				prefix="$prefix$sm_path/"
-				sanitize_submodule_env
-				cd "$sm_path" &&
-				# we make $path available to scripts ...
-				path=$sm_path &&
-				if test $# -eq 1
-				then
-					eval "$1"
-				else
-					"$@"
-				fi &&
-				if test -n "$recursive"
-				then
-					cmd_foreach "--recursive" "$@"
-				fi
-			) <&3 3<&- ||
-			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 #
-- 
2.13.0


^ permalink raw reply	[relevance 19%]

* Re: [PATCH v2 2/2 / RFC] branch: quote branch/ref names to improve readability
      [irrelevant]   ` <20170808171136.31168-2-kaarticsivaraam91196@gmail.com>
@ 2017-08-08 18:55     ` Stefan Beller
  2017-08-08 19:43       ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-08-08 18:55 UTC (permalink / raw)
  To: Kaartic Sivaraam; +Cc: git

On Tue, Aug 8, 2017 at 10:11 AM, Kaartic Sivaraam
<kaarticsivaraam91196@gmail.com> wrote:
> Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
> ---
>  branch.c | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)

I like this patch.

In submodule.c we quote a lot of things (branches, submodules, paths), so
this is another step to make the output as a whole more consistent.
(Though wondering for non-submodule users, if they perceive it as
inconsistency as other parts of the code may not follow the rigorous quoting)

>
> diff --git a/branch.c b/branch.c
> index ad5a2299b..a40721f3c 100644
> --- a/branch.c
> +++ b/branch.c
> @@ -90,24 +90,24 @@ int install_branch_config(int flag, const char *local, const char *origin, const
>                 if (shortname) {
>                         if (origin)
>                                 printf_ln(rebasing ?
> -                                         _("Branch %s set up to track remote branch %s from %s by rebasing.") :
> -                                         _("Branch %s set up to track remote branch %s from %s."),
> +                                         _("Branch '%s' set up to track remote branch '%s' from '%s' by rebasing.") :
> +                                         _("Branch '%s' set up to track remote branch '%s' from '%s'."),
>                                           local, shortname, origin);
>                         else
>                                 printf_ln(rebasing ?
> -                                         _("Branch %s set up to track local branch %s by rebasing.") :
> -                                         _("Branch %s set up to track local branch %s."),
> +                                         _("Branch '%s' set up to track local branch '%s' by rebasing.") :
> +                                         _("Branch '%s' set up to track local branch '%s'."),
>                                           local, shortname);
>                 } else {
>                         if (origin)
>                                 printf_ln(rebasing ?
> -                                         _("Branch %s set up to track remote ref %s by rebasing.") :
> -                                         _("Branch %s set up to track remote ref %s."),
> +                                         _("Branch '%s' set up to track remote ref '%s' by rebasing.") :
> +                                         _("Branch '%s' set up to track remote ref '%s'."),
>                                           local, remote);
>                         else
>                                 printf_ln(rebasing ?
> -                                         _("Branch %s set up to track local ref %s by rebasing.") :
> -                                         _("Branch %s set up to track local ref %s."),
> +                                         _("Branch '%s' set up to track local ref '%s' by rebasing.") :
> +                                         _("Branch '%s' set up to track local ref '%s'."),
>                                           local, remote);
>                 }
>         }
> --
> 2.14.0.rc1.434.g6eded367a
>

^ permalink raw reply	[relevance 18%]

* Re: [PATCH v2 2/2 / RFC] branch: quote branch/ref names to improve readability
  2017-08-08 18:55     ` [PATCH v2 2/2 / RFC] branch: quote branch/ref names to improve readability Stefan Beller
@ 2017-08-08 19:43       ` Junio C Hamano
  2017-08-08 20:08         ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-08-08 19:43 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Kaartic Sivaraam, git\

Stefan Beller <sbeller@google.com> writes:

> (Though wondering for non-submodule users, if they perceive it as
> inconsistency as other parts of the code may not follow the rigorous quoting)

Do you mean that we may instead want to remove the excessive quoting
of branch names and stuff from submodule.c code, because they are
newer ones that broke the consistency existed before them (i.e. not
quoting)?

That certainly is tempting, but I personally find it easier to read
a message that marks parts that holds "external data" differently
from the message's text, so I think this patch 2/2 goes in the right
direction.

^ permalink raw reply	[relevance 15%]

* Re: [PATCH v2 2/2 / RFC] branch: quote branch/ref names to improve readability
  2017-08-08 19:43       ` Junio C Hamano
@ 2017-08-08 20:08         ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-08 20:08 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Kaartic Sivaraam, git

On Tue, Aug 8, 2017 at 12:43 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> (Though wondering for non-submodule users, if they perceive it as
>> inconsistency as other parts of the code may not follow the rigorous quoting)
>
> Do you mean that we may instead want to remove the excessive quoting
> of branch names and stuff from submodule.c code, because they are
> newer ones that broke the consistency existed before them (i.e. not
> quoting)?

No, I do not. I was just wondering if a non-submodule user
may see differences between different commands now.

For example "checkout -b" already quotes 'external data', which
would be inline with this proposal, but there may be others.
My question was rather an encouragement to check the code base
if there are other occurrences left that do not quote.

In an ideal code base we could just grep for any %s that has no
surrounding quotes, but of course it is not as easy in the real world:
* some outputs use %s construction for non-human consumption
  in e.g. the diff machinery
* sometimes we play sentence lego, stringing words together
  which also is using %s unquoted correctly.

> That certainly is tempting, but I personally find it easier to read
> a message that marks parts that holds "external data" differently
> from the message's text, so I think this patch 2/2 goes in the right
> direction.

Yes. I like the direction this patch is going.

A note on 'external data':
For branchs, paths, submodule names a single quote seems
to be best (my opinion), whereas in e.g. git-status:

    Submodule 'sm1' 0000000...1beffeb (new submodule)

parens seem to do a better job as they describe the state,
not reproducing external data. (This is also the place
where I was reminded of potential sentence lego)

^ permalink raw reply	[relevance 18%]

* Re: [PATCH v1 1/1] dir: teach status to show ignored directories
      [irrelevant] ` <20170810184936.239542-2-jamill@microsoft.com>
@ 2017-08-10 20:03   ` Stefan Beller
  2017-08-11 17:48     ` Jameson Miller
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-08-10 20:03 UTC (permalink / raw)
  To: Jameson Miller; +Cc: git, Samuel Lijin, Brandon Williams, Junio C Hamano, Jeff King, Jameson Miller

On Thu, Aug 10, 2017 at 11:49 AM, Jameson Miller
<jameson.miller81@gmail.com> wrote:

Welcome to the Git mailing list. :)

> Teach Git to optionally show ignored directories when showing all
> untracked files. The git status command exposes the options to report
> ignored and/or untracked files. However, when reporting all untracked
> files (--untracked-files=all), all individual ignored files are reported
> as well. It is not currently possible to get the reporting behavior of
> the --ignored flag, while also reporting all untracked files.

Trying to understand this based off the documentation for
--untracked=all and --ignored, I realize that the documentation
for --ignored seems to be lacking as I do not understand what the
--ignored behavior is in combination with --untracked=[all, normal, no]

> This
> change exposes a flag to report all untracked files while not showing
> individual files in ignored directories.

By the description up to here, it sounds as if you want to introduce
mode for --untracked, e.g. "normal-adjusted-for-ignored" (it's a bad
suggestion)? However the patch seems to add an orthogonal flag,
such that

  status --no-ignored --untracked=no --show-ignored-directory

would also be possible. What is a reasonable expectation for
the output of such?

> Motivation:
> Our application (Visual Studio) needs all untracked files listed
> individually, but does not need all ignored files listed individually.

For parsing output, I would strongly recommend --porcelain[=2],
but that is a tangent.

> Reporting all ignored files can affect the time it takes for status
> to run. For a representative repository, here are some measurements
> showing a large perf improvement for this scenario:
>
> | Command | Reported ignored entries | Time (s) |
> | ------- | ------------------------ | -------- |
> | 1       | 0                        | 1.3      |
> | 2       | 1024                     | 4.2      |
> | 3       | 174904                   | 7.5      |
> | 4       | 1046                     | 1.6      |
>
> Commands:
>  1) status
>  2) status --ignored
>  3) status --ignored --untracked-files=all
>  4) status --ignored --untracked-files=all --show-ignored-directory

(2) is --untracked-files=normal I'd presume, such that this flag
can be understood as a tweak to "normal" based on the similar size
between 2 and 4? (The timing improvement from 2 to 4 is huge though).

> This changes exposes a --show-ignored-directory flag to the git status

s/changes/change/

> command. This flag is utilized when running git status with the
> --ignored and --untracked-files options to not list ignored individual
> ignored files contained in directories that match an ignore pattern.
>
> Part of the perf improvement comes from the tweak to
> read_directory_recursive to stop scanning the file system after it
> encounters the first file. When a directory is ignored, all it needs to
> determine is if the directory is empty or not. The logic currently keeps
> scanning the file system until it finds an untracked file. However, as
> the directory is ignored, all the contained contents are also marked
> excluded. For ignored directories that contain a large number of files,
> this can take some time.

I think it is possible to ignore a directory and still track files in it, what
are the implications of this flag on a tracked (and changed) file in an
ignored dir?

What happens to empty directories that match an ignore pattern?


> @@ -1362,6 +1363,8 @@ int cmd_status(int argc, const char **argv, const char *prefix)
>                   N_("ignore changes to submodules, optional when: all, dirty, untracked. (Default: all)"),
>                   PARSE_OPT_OPTARG, NULL, (intptr_t)"all" },
>                 OPT_COLUMN(0, "column", &s.colopts, N_("list untracked files in columns")),
> +               OPT_BOOL(0, "show-ignored-directory", &show_ignored_directory,

Is it possible to directly read into  s.show_ignored_directory here?

> +test_expect_success 'Verify behavior of status on folders with ignored files' '
> +       test_when_finished "git clean -fdx" &&
> +       git status --porcelain=v2 --ignored --untracked-files=all --show-ignored-directory >output &&
> +       test_i18ncmp expect output
> +'
> +
> +# Test status bahavior on folder with tracked and ignored files

behavior

> +cat >expect <<\EOF
> +? expect
> +? output
> +! dir/tracked_ignored/ignored_1.ign
> +! dir/tracked_ignored/ignored_2.ign
> +! tracked_ignored/ignored_1.ign
> +! tracked_ignored/ignored_2.ign
> +EOF

I think our latest 'best style' is to include these heredocs into the test.

^ permalink raw reply	[relevance 8%]

* Re: [PATCH 02/15] submodule: don't use submodule_from_name
  2017-07-31 20:43         ` Stefan Beller
@ 2017-08-11 16:53           ` Heiko Voigt
  0 siblings, 0 replies; 200+ results
From: Heiko Voigt @ 2017-08-11 16:53 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Jens Lehmann, Junio C Hamano, Brandon Williams, git, Jonathan Nieder

Hi,

sorry for the late reply, just stumpled upon this.

On Mon, Jul 31, 2017 at 01:43:04PM -0700, Stefan Beller wrote:
> On Sun, Jul 30, 2017 at 6:43 AM, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> > Am 26.07.2017 um 23:06 schrieb Junio C Hamano:
> >>
> >> Stefan Beller <sbeller@google.com> writes:
> >>
> >>> Rereading the archives, there was quite some discussion on the design
> >>> of these patches, but these lines of code did not get any attention
> >>>
> >>>      https://public-inbox.org/git/4CDB3063.5010801@web.de/
> >>>
> >>> I cc'd Jens in the hope of him having a good memory why he
> >>> wrote the code that way. :)
> >>
> >>
> >> Thanks for digging.  I wouldn't be surprised if this were a fallback
> >> to help a broken entry in .gitmodules that lack .path variable, but
> >> we shouldn't be sweeping the problem under the rug like that.
> >
> >
> > Sorry to disappoint you ;-) I added this in 7dce19d374 because
> > submodule by path lookup back then only parsed the checked out
> > .gitmodules file.
> 
> This is still the case AFAICT, as we never ask for a specific .gitmodules
> file identified by sha1 of the commit.

This was actually part of my original approach[1] but it seems I never got
around to implement that last part for which I originally started the
submodule config API: Proper recursive fetch.

I still have a patch for moved submodules lying around which pass a commit id
for a gitmodules file. That particular patch is quite simple and finished but
I was planning to include that in the finished fetch series. So I can have a
look if I can quickly update that to the current state, so we can at least have
one proper user of the submodule config API.

> > So looking for it by name was a good guess to
> > fetch a new submodule that wasn't present in the current HEAD's
> > .gitmodules, as the path is used as the default name in "git
> > submodule add".

I will have a look whether we can easily replace this hack with the proper
lookup now. Lets see how many low hanging fruits we have lying around
for recursive fetch. The full blown implementation including cloning of
new submodules might still take some time...

Cheers Heiko

[1] https://public-inbox.org/git/f5baa2acc09531a16f4f693eebbe60706bb8ed1e.1361751905.git.hvoigt@hvoigt.net/

^ permalink raw reply	[relevance 25%]

* Re: [PATCH v2 02/15] submodule: don't use submodule_from_name
  2017-08-04 21:53       ` Brandon Williams
@ 2017-08-11 16:59         ` Heiko Voigt
  0 siblings, 0 replies; 200+ results
From: Heiko Voigt @ 2017-08-11 16:59 UTC (permalink / raw)
  To: Brandon Williams; +Cc: Stefan Beller, git, Junio C Hamano, Jonathan Nieder, Jens Lehmann

On Fri, Aug 04, 2017 at 02:53:11PM -0700, Brandon Williams wrote:
> On 08/03, Stefan Beller wrote:
> > On Thu, Aug 3, 2017 at 11:19 AM, Brandon Williams <bmwill@google.com> wrote:
> > > The function 'submodule_from_name()' is being used incorrectly here as a
> > > submodule path is being used instead of a submodule name.  Since the
> > > correct function to use with a path to a submodule is already being used
> > > ('submodule_from_path()') let's remove the call to
> > > 'submodule_from_name()'.
> > >
> > > Signed-off-by: Brandon Williams <bmwill@google.com>
> > 
> > In case a reroll is needed, you could incorperate Jens feedback
> > stating that 851e18c385 should have done it.
> 
> K I'll add that into the commit message.

Well, thats not 100% correct... IMO, it should have been a follow up patch
which I never got to implement. See my other reply to the v1 of this
patch I just sent out.

As stated there I will have a look into where it makes sense to pass a
commit id and behave more correctly.

Cheers Heiko

^ permalink raw reply	[relevance 7%]

* Re: [PATCH v1 1/1] dir: teach status to show ignored directories
  2017-08-10 20:03   ` [PATCH v1 1/1] dir: teach status to show ignored directories Stefan Beller
@ 2017-08-11 17:48     ` Jameson Miller
  2017-08-14 21:05       ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Jameson Miller @ 2017-08-11 17:48 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Samuel Lijin, Brandon Williams, Junio C Hamano, Jeff King, Jameson Miller



On 08/10/2017 04:03 PM, Stefan Beller wrote:
> On Thu, Aug 10, 2017 at 11:49 AM, Jameson Miller
> <jameson.miller81@gmail.com> wrote:
>
> Welcome to the Git mailing list. :)

Thank you for the welcome and the review! I will include the
suggested code changes in the next patch version.

>> Teach Git to optionally show ignored directories when showing all
>> untracked files. The git status command exposes the options to report
>> ignored and/or untracked files. However, when reporting all untracked
>> files (--untracked-files=all), all individual ignored files are reported
>> as well. It is not currently possible to get the reporting behavior of
>> the --ignored flag, while also reporting all untracked files.
> Trying to understand this based off the documentation for
> --untracked=all and --ignored, I realize that the documentation
> for --ignored seems to be lacking as I do not understand what the
> --ignored behavior is in combination with --untracked=[all, normal, no]
>

The set of files listed by "--ignored" changes when different
values are given to "--untracked-files".  If would be nice to
be able to make the ignored output independent of the untracked
settings.  This patch attempts to do that while maintaining
backward compatibility with the existing behavior.

When "--ignored" is used by itself ("--untracked-files=normal"),
ignored directories (both directories that explicitly match a
directory ignore pattern -and- directories containing only ignored
files) are listed. Individual files within are *not* listed.

When "--ignored" is used with "--untracked-files=all", git always
lists the individual files within the ignored directories and DOES
NOT collapse the output to just the containing (explicitly or
implicitly ignored) directory.

This can cause a massive performance problem when there are
a lot of ignored files in a well-defined set of ignored directories.
For example, on Windows, Visual Studio creates a bin/ and
obj/ directory inside your project where it writes all .obj
files.  Normal usage is to explicitly ignore those 2 directory
names in your .gitignores (rather than or in addition to *.obj).
We just want to see the "bin/" and "obj/" paths
regardless of which "--untracked" flag is passed in.

We want to see the paths that explicitly match an ignore pattern.
This means that if a directory only contains ignored files, but the
directory itself is not explicitly ignored, then we want to see the
individual files. This is slightly different than the current behavior
of "--ignored".

I am open to suggestions on how to present the options to control
this behavior.

>> This
>> change exposes a flag to report all untracked files while not showing
>> individual files in ignored directories.
> By the description up to here, it sounds as if you want to introduce
> mode for --untracked, e.g. "normal-adjusted-for-ignored" (it's a bad
> suggestion)? However the patch seems to add an orthogonal flag,
> such that
>
>    status --no-ignored --untracked=no --show-ignored-directory
>
> would also be possible. What is a reasonable expectation for
> the output of such?

The current patch does add another flag. This flag only has meaning if
the "--ignored" and "--untracked=all" flags are also specified. Another
option I had considered is to let the "--ignored" flag take an argument.
Then, we could express this new behavior through (for example) a
"--ignored=exact" flag to reflect the fact that this new option
returns paths that match the ignore pattern, and does not
collapse directories that contain only ignored files.

>> Motivation:
>> Our application (Visual Studio) needs all untracked files listed
>> individually, but does not need all ignored files listed individually.
> For parsing output, I would strongly recommend --porcelain[=2],
> but that is a tangent.
>
>> Reporting all ignored files can affect the time it takes for status
>> to run. For a representative repository, here are some measurements
>> showing a large perf improvement for this scenario:
>>
>> | Command | Reported ignored entries | Time (s) |
>> | ------- | ------------------------ | -------- |
>> | 1       | 0                        | 1.3      |
>> | 2       | 1024                     | 4.2      |
>> | 3       | 174904                   | 7.5      |
>> | 4       | 1046                     | 1.6      |
>>
>> Commands:
>>   1) status
>>   2) status --ignored
>>   3) status --ignored --untracked-files=all
>>   4) status --ignored --untracked-files=all --show-ignored-directory
>> (2) is --untracked-files=normal I'd presume, such that this flag
>> can be understood as a tweak to "normal" based on the similar size
>> between 2 and 4? (The timing improvement from 2 to 4 is huge though).
(2) is --untracked-files=normal. Although the count of ignored
files similar between 2 and 4, I consider this flag more of a
tweak on 3, as we want the untracked files reported with
the "--untracked=all" flag. The counts between 2 and 4 are
similar in this case because most of the ignored files are
contained in ignored directories.

Our application calls status including the following flags:

--porcelain=v2 --ignored --untracked-files=all --ignore-submodules=none

This means we have bad performance compared to just "git status"
when there is a large number of files in ignored directories
With this new behavior, our application would move from case 3 to
case 4 for this repository.

You also point out the timing difference between case 2 and 4. I
think there is an optimization we can make when running "git
status --ignored" logic that will improve the the timing
here. Currently, the logic in dir.c is iterating over all files
contained in an ignored directory to see if it is empty or
not. This is not necessary - we should be able to stop after
finding any file. I plan to follow up on this in a different
patch.

>> This changes exposes a --show-ignored-directory flag to the git status
> s/changes/change/
>
>> command. This flag is utilized when running git status with the
>> --ignored and --untracked-files options to not list ignored individual
>> ignored files contained in directories that match an ignore pattern.
>>
>> Part of the perf improvement comes from the tweak to
>> read_directory_recursive to stop scanning the file system after it
>> encounters the first file. When a directory is ignored, all it needs to
>> determine is if the directory is empty or not. The logic currently keeps
>> scanning the file system until it finds an untracked file. However, as
>> the directory is ignored, all the contained contents are also marked
>> excluded. For ignored directories that contain a large number of files,
>> this can take some time.
> I think it is possible to ignore a directory and still track files in it, what
> are the implications of this flag on a tracked (and changed) file in an
> ignored dir?
It is possible to have tracked files in an ignored directory. The
behavior with this patch is to show individual files in the
ignored directory. This patch includes a test covering this
scenario. I think this behavior is reasonable for this case - I
am interested if there are other opinions on this.
>
> What happens to empty directories that match an ignore pattern?
>
The "git status" command would not show empty directories. From
the lower level implementation in dir.c, I think it *should*
depend on the DIR_HIDE_EMPTY_DIRECTORIES flag, but does not
currently. I will fix this with the next patch version.
>> @@ -1362,6 +1363,8 @@ int cmd_status(int argc, const char **argv, const char *prefix)
>>                    N_("ignore changes to submodules, optional when: all, dirty, untracked. (Default: all)"),
>>                    PARSE_OPT_OPTARG, NULL, (intptr_t)"all" },
>>                  OPT_COLUMN(0, "column", &s.colopts, N_("list untracked files in columns")),
>> +               OPT_BOOL(0, "show-ignored-directory", &show_ignored_directory,
> Is it possible to directly read into  s.show_ignored_directory here?
>
>> +test_expect_success 'Verify behavior of status on folders with ignored files' '
>> +       test_when_finished "git clean -fdx" &&
>> +       git status --porcelain=v2 --ignored --untracked-files=all --show-ignored-directory >output &&
>> +       test_i18ncmp expect output
>> +'
>> +
>> +# Test status bahavior on folder with tracked and ignored files
> behavior
>
>> +cat >expect <<\EOF
>> +? expect
>> +? output
>> +! dir/tracked_ignored/ignored_1.ign
>> +! dir/tracked_ignored/ignored_2.ign
>> +! tracked_ignored/ignored_1.ign
>> +! tracked_ignored/ignored_2.ign
>> +EOF
> I think our latest 'best style' is to include these heredocs into the test.


^ permalink raw reply	[relevance 10%]

* Re: [RFC PATCH 2/3] diff: respect MIN_BLOCK_LENGTH for last block
      [irrelevant] ` <ca94745f8d72a1d472462e9cb25ef3d2e1285f86.1502491372.git.jonathantanmy@google.com>
@ 2017-08-14 17:17   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-14 17:17 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: git

On Fri, Aug 11, 2017 at 3:49 PM, Jonathan Tan <jonathantanmy@google.com> wrote:
> Currently, MIN_BLOCK_LENGTH is only checked when diff encounters a line
> that does not belong to the current block. In particular, this means
> that MIN_BLOCK_LENGTH is not checked after all lines are encountered.
>
> Perform that check.

Thanks for spotting! This fix looks straightforward correct.
(Also thanks for factoring out the adjustment, I am tempted to
start a bike shedding discussion about its name, though. :P)

> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  diff.c                     | 29 ++++++++++++++++++++++-------
>  t/t4015-diff-whitespace.sh | 35 +++++++++++++++++++++++++++++++++++
>  2 files changed, 57 insertions(+), 7 deletions(-)
>
> diff --git a/diff.c b/diff.c
> index 4965ffbc4..95620b130 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -858,6 +858,26 @@ static int shrink_potential_moved_blocks(struct moved_entry **pmb,
>         return rp + 1;
>  }
>
> +/*
> + * If o->color_moved is COLOR_MOVED_PLAIN, this function does nothing.
> + *
> + * Otherwise, if the last block has fewer lines than
> + * COLOR_MOVED_MIN_BLOCK_LENGTH, unset DIFF_SYMBOL_MOVED_LINE on all lines in
> + * that block.
> + *
> + * The last block consists of the (n - block_length)'th line up to but not
> + * including the nth line.
> + */
> +static void adjust_last_block(struct diff_options *o, int n, int block_length)
> +{
> +       int i;
> +       if (block_length >= COLOR_MOVED_MIN_BLOCK_LENGTH ||
> +           o->color_moved == COLOR_MOVED_PLAIN)
> +               return;
> +       for (i = 1; i < block_length + 1; i++)
> +               o->emitted_symbols->buf[n - i].flags &= ~DIFF_SYMBOL_MOVED_LINE;
> +}
> +
>  /* Find blocks of moved code, delegate actual coloring decision to helper */
>  static void mark_color_as_moved(struct diff_options *o,
>                                 struct hashmap *add_lines,
> @@ -893,13 +913,7 @@ static void mark_color_as_moved(struct diff_options *o,
>                 }
>
>                 if (!match) {
> -                       if (block_length < COLOR_MOVED_MIN_BLOCK_LENGTH &&
> -                           o->color_moved != COLOR_MOVED_PLAIN) {
> -                               for (i = 1; i < block_length + 1; i++) {
> -                                       l = &o->emitted_symbols->buf[n - i];
> -                                       l->flags &= ~DIFF_SYMBOL_MOVED_LINE;
> -                               }
> -                       }
> +                       adjust_last_block(o, n, block_length);
>                         pmb_nr = 0;
>                         block_length = 0;
>                         continue;
> @@ -941,6 +955,7 @@ static void mark_color_as_moved(struct diff_options *o,
>                 if (flipped_block)
>                         l->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;
>         }
> +       adjust_last_block(o, n, block_length);
>
>         free(pmb);
>  }
> diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
> index c3b697411..6f7758e5c 100755
> --- a/t/t4015-diff-whitespace.sh
> +++ b/t/t4015-diff-whitespace.sh
> @@ -1382,6 +1382,41 @@ EOF
>         test_cmp expected actual
>  '
>
> +test_expect_success '--color-moved block at end of diff output respects MIN_BLOCK_LENGTH' '
> +       git reset --hard &&
> +       >bar &&
> +       cat <<-\EOF >foo &&
> +       irrelevant_line
> +       line1
> +       EOF
> +       git add foo bar &&
> +       git commit -m x &&
> +
> +       cat <<-\EOF >bar &&
> +       line1
> +       EOF
> +       cat <<-\EOF >foo &&
> +       irrelevant_line
> +       EOF
> +
> +       git diff HEAD --color-moved=zebra --no-renames | grep -v "index" | test_decode_color >actual &&
> +       cat >expected <<-\EOF &&
> +       <BOLD>diff --git a/bar b/bar<RESET>
> +       <BOLD>--- a/bar<RESET>
> +       <BOLD>+++ b/bar<RESET>
> +       <CYAN>@@ -0,0 +1 @@<RESET>
> +       <GREEN>+<RESET><GREEN>line1<RESET>
> +       <BOLD>diff --git a/foo b/foo<RESET>
> +       <BOLD>--- a/foo<RESET>
> +       <BOLD>+++ b/foo<RESET>
> +       <CYAN>@@ -1,2 +1 @@<RESET>
> +        irrelevant_line<RESET>
> +       <RED>-line1<RESET>
> +       EOF
> +
> +       test_cmp expected actual
> +'
> +
>  test_expect_success 'move detection with submodules' '
>         test_create_repo bananas &&
>         echo ripe >bananas/recipe &&
> --
> 2.14.0.434.g98096fd7a8-goog
>

^ permalink raw reply	[relevance 8%]

* Re: [PATCH v1 1/1] dir: teach status to show ignored directories
  2017-08-11 17:48     ` Jameson Miller
@ 2017-08-14 21:05       ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-14 21:05 UTC (permalink / raw)
  To: Jameson Miller; +Cc: git, Samuel Lijin, Brandon Williams, Junio C Hamano, Jeff King, Jameson Miller

On Fri, Aug 11, 2017 at 10:48 AM, Jameson Miller
<jameson.miller81@gmail.com> wrote:
> The set of files listed by "--ignored" changes when different
> values are given to "--untracked-files".  If would be nice to
> be able to make the ignored output independent of the untracked
> settings.  This patch attempts to do that while maintaining
> backward compatibility with the existing behavior.


So after rereading this, maybe we could have
'--ignored[=as-untracked, new-mode]' with as-untracked as default?

This would satisfy your goal, while still maintaining backwards
compatibility.

> We want to see the paths that explicitly match an ignore pattern.
> This means that if a directory only contains ignored files, but the
> directory itself is not explicitly ignored, then we want to see the
> individual files. This is slightly different than the current behavior
> of "--ignored".
>
> I am open to suggestions on how to present the options to control
> this behavior.

I would strongly favor not introducing yet another flag to tweak the
behavior of another command line option. (The UX of Git is already
bloated, IMHO. But that is just my general stance on things)

So we'd have two proposals:

(1) The current patch proposes --show-ignored-directory that modifies
    the behavior of other flags
(2) extend an already existing option to take another mode (which I
    propose)

Strong arguments for (1) would be if the flag is orthogonal and apply
to the other flags in a uniform way, i.e.:
    status --no-ignored --untracked=no --show-ignored-directory
would make sense and produce an output that a user doesn't
find confusing.

Arguments for (2), are
* less documentation for users (see below)
* if the two flags are truly orthogonal, test effort is decreased
* no exposure of an API that may yield strange results when the user
  gives strange input.

>>> This
>>> change exposes a flag to report all untracked files while not showing
>>> individual files in ignored directories.
>>
>> By the description up to here, it sounds as if you want to introduce
>> mode for --untracked, e.g. "normal-adjusted-for-ignored" (it's a bad
>> suggestion)? However the patch seems to add an orthogonal flag,
>> such that
>>
>>    status --no-ignored --untracked=no --show-ignored-directory
>>
>> would also be possible. What is a reasonable expectation for
>> the output of such?
>
>
> The current patch does add another flag. This flag only has meaning if
> the "--ignored" and "--untracked=all" flags are also specified. Another
> option I had considered is to let the "--ignored" flag take an argument.

I would think that is a better solution, as then untracked and ignored
files (and their parameters) will be orthogonal in the long run.
Things like

  --untracked=all --ignored=none
  --untracked=none --ignored=only-dirs
  --untracked=reasonable-default --ignored=other-reasonable-default

will be possible and eventually the default for ignored files may be
independent of untracked settings.

> Then, we could express this new behavior through (for example) a
> "--ignored=exact" flag to reflect the fact that this new option
> returns paths that match the ignore pattern, and does not
> collapse directories that contain only ignored files.

Yes, and we would not need to worry about how the new
flag interacts with other flags. I would prefer that solution as it
seems the API will be cleaner that way.

Less things depend on each other, e.g. as a user I do not need to
know about the 'other' flag, be it ignored or untracked to solve my
problem. If we'd go with this third flag, we'd need to (a) document it,
but (b) also have to add a sentence to untracked and ignored flag
    "In the corner case of also having the third flag, we change
    behavior once again"
which I'd dislike from a users perspective.

>>> Commands:
>>>   1) status
>>>   2) status --ignored
>>>   3) status --ignored --untracked-files=all
>>>   4) status --ignored --untracked-files=all --show-ignored-directory
>>> (2) is --untracked-files=normal I'd presume, such that this flag
>>> can be understood as a tweak to "normal" based on the similar size
>>> between 2 and 4? (The timing improvement from 2 to 4 is huge though).
>
> (2) is --untracked-files=normal. Although the count of ignored
> files similar between 2 and 4, I consider this flag more of a
> tweak on 3, as we want the untracked files reported with
> the "--untracked=all" flag. The counts between 2 and 4 are
> similar in this case because most of the ignored files are
> contained in ignored directories.
>
> Our application calls status including the following flags:
>
> --porcelain=v2 --ignored --untracked-files=all --ignore-submodules=none
>
> This means we have bad performance compared to just "git status"
> when there is a large number of files in ignored directories
> With this new behavior, our application would move from case 3 to
> case 4 for this repository.
>
> You also point out the timing difference between case 2 and 4. I
> think there is an optimization we can make when running "git
> status --ignored" logic that will improve the the timing
> here. Currently, the logic in dir.c is iterating over all files
> contained in an ignored directory to see if it is empty or
> not. This is not necessary - we should be able to stop after
> finding any file. I plan to follow up on this in a different
> patch.

Thanks!

^ permalink raw reply	[relevance 7%]

* [PATCH] push: do not add submodule odb as an alternate when recursing on demand
      [irrelevant] <CAGZ79kZouNBxOKr7X8j6wqebp3Wh3cDqhYR-t_PxaF7AwQ0Wzg@mail.gmail.com>
@ 2017-08-15 22:43 ` Stefan Beller
  2017-08-15 23:10   ` Jonathan Nieder
                     ` (2 more replies)
  0 siblings, 3 replies; 200+ results
From: Stefan Beller @ 2017-08-15 22:43 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jrnieder

"git push --recurse-submodules=on-demand" adds each submodule as an
alternate with add_submodule_odb before checking whether the
submodule has anything to push and pushing it if so.

However, it never accesses any objects from the submodule.  In the
parent process it uses the submodule's ref database to see if there
is anything to push.  The actual push (which does rely on objects)
occurs in a child process.

The same was true when this call was originally added in
v1.7.11-rc0~111^2 (push: teach --recurse-submodules the on-demand
option, 2012-03-29).  Most likely it was added by analogy with
fetch --recurse-submodules=on-demand, which did use the submodule's
object database.

Use is_submodule_populated_gently instead, which is simpler and
cheaper.

Signed-off-by: Stefan Beller <sbeller@google.com>
---

 Originally I intended to send this out as part of a larger series,
 but the series is getting too large series, sending all things in smaller
 units!
 
 Thanks,
 Stefan

 submodule.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/submodule.c b/submodule.c
index 111a3007fc..e20216bc0f 100644
--- a/submodule.c
+++ b/submodule.c
@@ -966,7 +966,9 @@ static int push_submodule(const char *path,
 			  const struct string_list *push_options,
 			  int dry_run)
 {
-	if (add_submodule_odb(path))
+	int code;
+
+	if (!is_submodule_populated_gently(path, &code))
 		return 1;
 
 	if (for_each_remote_ref_submodule(path, has_remote, NULL) > 0) {
-- 
2.14.0.rc0.3.g6c2e499285


^ permalink raw reply	[relevance 33%]

* [GSoC] Update: Week-13
@ 2017-08-15 23:04 Prathamesh Chavan
  0 siblings, 0 replies; 200+ results
From: Prathamesh Chavan @ 2017-08-15 23:04 UTC (permalink / raw)
  To: git; +Cc: Christian Couder, Stefan Beller

SUMMARY OF MY PROJECT:

Git submodule subcommands are currently implemented by using shell script
'git-submodule.sh'. There are several reasons why we'll prefer not to
use the shell script. My project intends to convert the subcommands into
C code, thus making them builtins. This will increase Git's portability
and hence the efficiency of working with the git-submodule commands.
Link to the complete proposal: [1]

Mentors:
Stefan Beller <sbeller@google.com>
Christian Couder <christian.couder@gmail.com>

UPDATES:

Following are the updates about my ongoing project:

* add: most of the time of the week was spent in debugging the
  ported submodule subcommand functions. But, even after so,
  currently the ported functions fail in total 15 tests from
  the test suit. The WIP patches have been updated regularly,
  and currently the patch is under discussion with the mentors
  as well. The current status of the patch is pushed on github as
  well, and can be viewed at:[2]

Since the rest of the patches were almost the same as that in the
previous update(except for the 'summary' patch, which was last
updated after Christian's review), the haven't been uploaded
again to avoid unnecessary floating patches.
The previous updated series is available at: [3]
But given that, next week I plan to float a separate patch
series, containing the initial patches from the subcommand porting
series (till deinit), which IMO, are ready for maintainer's
review.

Also, this week's update was one-day late, since I was traveling
on the previous day. But a prior idea about this was given to the
mentors.

PLAN FOR WEEK-14 (15 August 2017 to 21 August 2017):

* patches: Float a separate series, till deinit, and ask the maintainer
  for its review.

* add: The main aim of the next week is to resolve the issues with
  the current patch, and get all the tests pass.

* update: it is the last remaining subcommand to be ported. I aim to
  atleast start with this in the following week.

The work till week-13 is pushed on Github and is available at: [4].

[1]: https://docs.google.com/document/d/1krxVLooWl--75Pot3dazhfygR3wCUUWZWzTXtK1L-xU/
[2]: https://github.com/pratham-pc/git/commits/sub-add
[3]: https://public-inbox.org/git/20170807211900.15001-1-pc44800@gmail.com/
[4]: https://github.com/pratham-pc/git/commits/week-13

^ permalink raw reply	[relevance 16%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-15 22:43 ` [PATCH] push: do not add submodule odb as an alternate when recursing on demand Stefan Beller
@ 2017-08-15 23:10   ` Jonathan Nieder
  2017-08-15 23:14   ` Jonathan Nieder
  2017-08-15 23:23   ` Junio C Hamano
  2 siblings, 0 replies; 200+ results
From: Jonathan Nieder @ 2017-08-15 23:10 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, gitster

Stefan Beller wrote:

> "git push --recurse-submodules=on-demand" adds each submodule as an
> alternate with add_submodule_odb before checking whether the
> submodule has anything to push and pushing it if so.
>
> However, it never accesses any objects from the submodule.  In the
> parent process it uses the submodule's ref database to see if there
> is anything to push.  The actual push (which does rely on objects)
> occurs in a child process.
>
> The same was true when this call was originally added in
> v1.7.11-rc0~111^2 (push: teach --recurse-submodules the on-demand
> option, 2012-03-29).  Most likely it was added by analogy with
> fetch --recurse-submodules=on-demand, which did use the submodule's
> object database.
>
> Use is_submodule_populated_gently instead, which is simpler and
> cheaper.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  submodule.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

Thanks for picking up this loose end.

> diff --git a/submodule.c b/submodule.c
> index 111a3007fc..e20216bc0f 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -966,7 +966,9 @@ static int push_submodule(const char *path,
>  			  const struct string_list *push_options,
>  			  int dry_run)
>  {
> -	if (add_submodule_odb(path))
> +	int code;
> +
> +	if (!is_submodule_populated_gently(path, &code))
>  		return 1;
>  
>  	if (for_each_remote_ref_submodule(path, has_remote, NULL) > 0) {

^ permalink raw reply	[relevance 16%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-15 22:43 ` [PATCH] push: do not add submodule odb as an alternate when recursing on demand Stefan Beller
  2017-08-15 23:10   ` Jonathan Nieder
@ 2017-08-15 23:14   ` Jonathan Nieder
  2017-08-15 23:27     ` Stefan Beller
  2017-08-15 23:23   ` Junio C Hamano
  2 siblings, 1 reply; 200+ results
From: Jonathan Nieder @ 2017-08-15 23:14 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, gitster

Stefan Beller wrote:

> Use is_submodule_populated_gently instead, which is simpler and
> cheaper.
[...]
> --- a/submodule.c
> +++ b/submodule.c
> @@ -966,7 +966,9 @@ static int push_submodule(const char *path,
>  			  const struct string_list *push_options,
>  			  int dry_run)
>  {
> -	if (add_submodule_odb(path))
> +	int code;
> +
> +	if (!is_submodule_populated_gently(path, &code))
>  		return 1;

Ah, I forgot about this detail.  I don't think it should block this
patch (so my Reviewed-by still stands), but I wonder why this needs to
be gentle.  add_submodule_odb is gentle so that is the conservative
thing to do, but that doesn't mean it is the *right* thing to do.

If this passed NULL instead of &code as the second argument, would
anything break?

Could there be a comment explaining what kind of error we are
expecting and why it is okay to continue when that error is
encountered without any error handling?

Thanks,
Jonathan

^ permalink raw reply	[relevance 16%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-15 22:43 ` [PATCH] push: do not add submodule odb as an alternate when recursing on demand Stefan Beller
  2017-08-15 23:10   ` Jonathan Nieder
  2017-08-15 23:14   ` Jonathan Nieder
@ 2017-08-15 23:23   ` Junio C Hamano
  2017-08-15 23:31     ` Stefan Beller
  2 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-08-15 23:23 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, jrnieder

Stefan Beller <sbeller@google.com> writes:

> "git push --recurse-submodules=on-demand" adds each submodule as an
> alternate with add_submodule_odb before checking whether the
> submodule has anything to push and pushing it if so.
>
> However, it never accesses any objects from the submodule.
> ...
> Use is_submodule_populated_gently instead, which is simpler and
> cheaper.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>
>  Originally I intended to send this out as part of a larger series,
>  but the series is getting too large series, sending all things in smaller
>  units!

This vaguely reminds me that you sent something imilar perhaps for a
different codepath.

Is "is it populated" a good thing to check here, though?  IIRC,
add-submodule-odb allows you to add the object database of an
inactivated submodule, so this seems to change the behaviour.  I do
not know if the behaviour change is a good thing (i.e. bugfix) or
not (i.e. regression) offhand, though.

Thanks.

> diff --git a/submodule.c b/submodule.c
> index 111a3007fc..e20216bc0f 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -966,7 +966,9 @@ static int push_submodule(const char *path,
>  			  const struct string_list *push_options,
>  			  int dry_run)
>  {
> -	if (add_submodule_odb(path))
> +	int code;
> +
> +	if (!is_submodule_populated_gently(path, &code))
>  		return 1;
>  
>  	if (for_each_remote_ref_submodule(path, has_remote, NULL) > 0) {

^ permalink raw reply	[relevance 24%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-15 23:14   ` Jonathan Nieder
@ 2017-08-15 23:27     ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-15 23:27 UTC (permalink / raw)
  To: Jonathan Nieder; +Cc: git, Junio C Hamano

On Tue, Aug 15, 2017 at 4:14 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Stefan Beller wrote:
>
>> Use is_submodule_populated_gently instead, which is simpler and
>> cheaper.
> [...]
>> --- a/submodule.c
>> +++ b/submodule.c
>> @@ -966,7 +966,9 @@ static int push_submodule(const char *path,
>>                         const struct string_list *push_options,
>>                         int dry_run)
>>  {
>> -     if (add_submodule_odb(path))
>> +     int code;
>> +
>> +     if (!is_submodule_populated_gently(path, &code))
>>               return 1;
>
> Ah, I forgot about this detail.  I don't think it should block this
> patch (so my Reviewed-by still stands), but I wonder why this needs to
> be gentle.  add_submodule_odb is gentle so that is the conservative
> thing to do, but that doesn't mean it is the *right* thing to do.
>
> If this passed NULL instead of &code as the second argument, would
> anything break?

push_submodule, which is called by push_unpushed_submodules
just issues warnings on submodule push error, (which happen all
before the main push,) such that any submodule error does not prevent
the main push.

We could tighten that, but I would suggest another patch for that.

> Could there be a comment explaining what kind of error we are
> expecting and why it is okay to continue when that error is
> encountered without any error handling?

The same as in add_submodule_odb, digging through the
logs. I up to now we did not care about the submodule succeeding
as much, it was just an aid for the main repo push.

>
> Thanks,
> Jonathan

^ permalink raw reply	[relevance 25%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-15 23:23   ` Junio C Hamano
@ 2017-08-15 23:31     ` Stefan Beller
  2017-08-16  0:11       ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-08-15 23:31 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jonathan Nieder

On Tue, Aug 15, 2017 at 4:23 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> "git push --recurse-submodules=on-demand" adds each submodule as an
>> alternate with add_submodule_odb before checking whether the
>> submodule has anything to push and pushing it if so.
>>
>> However, it never accesses any objects from the submodule.
>> ...
>> Use is_submodule_populated_gently instead, which is simpler and
>> cheaper.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>
>>  Originally I intended to send this out as part of a larger series,
>>  but the series is getting too large series, sending all things in smaller
>>  units!
>
> This vaguely reminds me that you sent something imilar perhaps for a
> different codepath.

https://public-inbox.org/git/xmqqh8xzq6td.fsf@gitster.mtv.corp.google.com/

> Is "is it populated" a good thing to check here, though?  IIRC,
> add-submodule-odb allows you to add the object database of an
> inactivated submodule, so this seems to change the behaviour.  I do
> not know if the behaviour change is a good thing (i.e. bugfix) or
> not (i.e. regression) offhand, though.

Good point, we should be able to push non-populated, even inactive(?)
submodules. For that we strictly need add_submodule_odb here
(or the repo object of the submodule, eventually).

So let's retract this patch for now.

Thanks.

^ permalink raw reply	[relevance 24%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-15 23:31     ` Stefan Beller
@ 2017-08-16  0:11       ` Junio C Hamano
  2017-08-16  1:05         ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2017-08-16  0:11 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git\, Jonathan Nieder

Stefan Beller <sbeller@google.com> writes:

>> Is "is it populated" a good thing to check here, though?  IIRC,
>> add-submodule-odb allows you to add the object database of an
>> inactivated submodule, so this seems to change the behaviour.  I do
>> not know if the behaviour change is a good thing (i.e. bugfix) or
>> not (i.e. regression) offhand, though.
>
> Good point, we should be able to push non-populated, even inactive(?)
> submodules. For that we strictly need add_submodule_odb here
> (or the repo object of the submodule, eventually).
>
> So let's retract this patch for now.

Not so fast.  

I am not convinced "push --recursive" should touch a submodule that
was once cloned from the upstream and then deactivated, so using
add-submodule-odb to decide if the push should go through may be a
bug that we may want to fix, in which case the diff of the patch in
question may be good as-is.  We need to sell it as a bugfix to the
users, who may complain about behaviour change (if there is one).

On the other hand, even if it were desirable for such a deactivated
submodule to be pushed, as your log message explained, there is no
reason to contaminate the in-core object hash by calling the
add-submodule-odb helper, when the only thing we care about is "do
we have the refs and object store for this submodule? we do not care
if it is activated or not".  Perhaps there is a more appropriate
helper in submodule.c that answers that question that we should be
using instead of add-submodule-odb, and if there is not yet such a
helper, perhaps this indicates that we need to add such a helper,
which essentially is the early half of what add-submodule-odb does,
i.e. ask git_path_submodule() for the object store and check if that
directory exists.

Thanks.

^ permalink raw reply	[relevance 25%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-16  0:11       ` Junio C Hamano
@ 2017-08-16  1:05         ` Stefan Beller
  2017-08-16  2:08           ` Jonathan Nieder
  2017-08-16 16:35           ` Heiko Voigt
  0 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2017-08-16  1:05 UTC (permalink / raw)
  To: Junio C Hamano, Jonathan Tan; +Cc: git, Jonathan Nieder

On Tue, Aug 15, 2017 at 5:11 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>> Is "is it populated" a good thing to check here, though?  IIRC,
>>> add-submodule-odb allows you to add the object database of an
>>> inactivated submodule, so this seems to change the behaviour.  I do
>>> not know if the behaviour change is a good thing (i.e. bugfix) or
>>> not (i.e. regression) offhand, though.
>>
>> Good point, we should be able to push non-populated, even inactive(?)
>> submodules. For that we strictly need add_submodule_odb here
>> (or the repo object of the submodule, eventually).
>>
>> So let's retract this patch for now.
>
> Not so fast.

Ok, I took another look at the code.

While we may desire that un-populated submodules can be pushed
(due to checking out another revision where the submodule
doesn't exist, before pushing), this is not supported currently, because
the call to run the push in the submodule assumes there is a
"<path>/.git" on which the child process can operate.
So for now we HAVE to have the submodule populated.

  In the future we may have the a lighter version just checking the
  object store of the submodule. Maybe this use case in the submodule
  can be covered by the .remote/.imported mechanism as well, such
  that we'd know if we have any local objects.

> I am not convinced "push --recursive" should touch a submodule that
> was once cloned from the upstream and then deactivated, so using
> add-submodule-odb to decide if the push should go through may be a
> bug that we may want to fix, in which case the diff of the patch in
> question may be good as-is.  We need to sell it as a bugfix to the
> users, who may complain about behaviour change (if there is one).

correct. Coupled with the observation above, we want really need
both "is active and populated" (your paragraph suggest not-active
submodules don't need pushing, the populated bit comes from the
observation above)

> On the other hand, even if it were desirable for such a deactivated
> submodule to be pushed, as your log message explained, there is no
> reason to contaminate the in-core object hash by calling the
> add-submodule-odb helper, when the only thing we care about is "do
> we have the refs and object store for this submodule? we do not care
> if it is activated or not".  Perhaps there is a more appropriate
> helper in submodule.c that answers that question that we should be
> using instead of add-submodule-odb, and if there is not yet such a
> helper, perhaps this indicates that we need to add such a helper,
> which essentially is the early half of what add-submodule-odb does,
> i.e. ask git_path_submodule() for the object store and check if that
> directory exists.

I think due to the constraint of also needing a worktree the is_populated
helper is the correct choice here.


>
> Thanks.

^ permalink raw reply	[relevance 26%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-16  1:05         ` Stefan Beller
@ 2017-08-16  2:08           ` Jonathan Nieder
  2017-08-16  5:52             ` Stefan Beller
  2017-08-16 16:35           ` Heiko Voigt
  1 sibling, 1 reply; 200+ results
From: Jonathan Nieder @ 2017-08-16  2:08 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Junio C Hamano, Jonathan Tan, git

Hi,

Stefan Beller wrote:
> On Tue, Aug 15, 2017 at 5:11 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>> Junio C Hamano wrote:

>>>> Is "is it populated" a good thing to check here, though?  IIRC,
>>>> add-submodule-odb allows you to add the object database of an
>>>> inactivated submodule, so this seems to change the behaviour.  I do
>>>> not know if the behaviour change is a good thing (i.e. bugfix) or
>>>> not (i.e. regression) offhand, though.
>>>
>>> Good point, we should be able to push non-populated, even inactive(?)
>>> submodules. For that we strictly need add_submodule_odb here
>>> (or the repo object of the submodule, eventually).
>>>
>>> So let's retract this patch for now.
>>
>> Not so fast.
>
> Ok, I took another look at the code.
>
> While we may desire that un-populated submodules can be pushed
> (due to checking out another revision where the submodule
> doesn't exist, before pushing), this is not supported currently, because
> the call to run the push in the submodule assumes there is a
> "<path>/.git" on which the child process can operate.
> So for now we HAVE to have the submodule populated.

It was not immediately obvious to me that this is just "for now".

I would be really confused if I had deactivated a submodule and
"git push --recurse-submodules" pushed from it anyway.  If the
submodule is active but not populated, then the question becomes "Why
wasn't it populated?"

If this is a bare repository, then nothing is populated, and pushing
from an active-but-unpopulated submodule sounds like a plausible wish.
But in a non-bare repository, I'm having trouble imagining the use
case that brings this situation about.

And where people have been needing this so far has been non-bare
repositories.  In that context, the check "is active and populated"
does not seem unusual or provisional.  Are you hinting that replacing
the check with "is active" would make it work well in bare
repositories?  I think I agree, though you'd have to be careful about
the case where the submodule is active but hasn't been fetched to
$GIT_DIR/modules yet.

Thanks,
Jonathan

^ permalink raw reply	[relevance 25%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-16  2:08           ` Jonathan Nieder
@ 2017-08-16  5:52             ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-16  5:52 UTC (permalink / raw)
  To: Jonathan Nieder; +Cc: Junio C Hamano, Jonathan Tan, git

On Tue, Aug 15, 2017 at 7:08 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi,
>
> Stefan Beller wrote:
>> On Tue, Aug 15, 2017 at 5:11 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Stefan Beller <sbeller@google.com> writes:
>>>> Junio C Hamano wrote:
>
>>>>> Is "is it populated" a good thing to check here, though?  IIRC,
>>>>> add-submodule-odb allows you to add the object database of an
>>>>> inactivated submodule, so this seems to change the behaviour.  I do
>>>>> not know if the behaviour change is a good thing (i.e. bugfix) or
>>>>> not (i.e. regression) offhand, though.
>>>>
>>>> Good point, we should be able to push non-populated, even inactive(?)
>>>> submodules. For that we strictly need add_submodule_odb here
>>>> (or the repo object of the submodule, eventually).
>>>>
>>>> So let's retract this patch for now.
>>>
>>> Not so fast.
>>
>> Ok, I took another look at the code.
>>
>> While we may desire that un-populated submodules can be pushed
>> (due to checking out another revision where the submodule
>> doesn't exist, before pushing), this is not supported currently, because
>> the call to run the push in the submodule assumes there is a
>> "<path>/.git" on which the child process can operate.
>> So for now we HAVE to have the submodule populated.
>
> It was not immediately obvious to me that this is just "for now".
>
> I would be really confused if I had deactivated a submodule and
> "git push --recurse-submodules" pushed from it anyway.

agreed.

>  If the
> submodule is active but not populated, then the question becomes "Why
> wasn't it populated?"

because you checked out a different revision where the sub is not
part of the working tree, for example. As long as you have locally
interesting "stuff" you want to transfer it. (IMHO even when the sub
is not active)

> If this is a bare repository, then nothing is populated,

Currently bare repos do not support having submodules
AT ALL, i.e. there is no modules/<name> directory
where the submodules would have any valuable information.
instead we *only* have the .gitmodules files at e.g. HEAD.

> and pushing
> from an active-but-unpopulated submodule sounds like a plausible wish.

it seems we're on the same page here.

> But in a non-bare repository, I'm having trouble imagining the use
> case that brings this situation about.

git checkout revision-with-sub
# hack.. hack.. hack..
"git commit --recurse-submodules -m bugfix"
# ^ not yet implemented
git checkout revision-without-sub
# Oh! I forgot to push the bugfix
git push --recurse-submodule <bugfix>

> And where people have been needing this so far has been non-bare
> repositories.  In that context, the check "is active and populated"
> does not seem unusual or provisional.  Are you hinting that replacing
> the check with "is active" would make it work well in bare
> repositories?

No I was saying:
* currently the code only works when "is populated"
* currently "is active" seems not to be considered

* in the future the code shall work even when not "is populated",
  (but submodule git dir is there at the very least)
* we may want to discuss if we want to care about "is active"
  additionally

>  I think I agree, though you'd have to be careful about
> the case where the submodule is active but hasn't been fetched to
> $GIT_DIR/modules yet.

When not fetched yet, there is no modules/<name> git dir and
we are sure there is no locally interesting information, so we
want to omit the push. (see above "has git dir" is the minimum
requirement, as opposed to current "is populated".

> Thanks,
> Jonathan

^ permalink raw reply	[relevance 24%]

* Re: [PATCH] push: do not add submodule odb as an alternate when recursing on demand
  2017-08-16  1:05         ` Stefan Beller
  2017-08-16  2:08           ` Jonathan Nieder
@ 2017-08-16 16:35           ` Heiko Voigt
  1 sibling, 0 replies; 200+ results
From: Heiko Voigt @ 2017-08-16 16:35 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Junio C Hamano, Jonathan Tan, git, Jonathan Nieder

Hi,

was about to write that we are maybe overly cautious here. Because the
current way a submodule ends up in the list to be pushed is through:

    find_unpushed_submodules()

that itself collects all changed submodules when submodule_needs_pushing() is
true. In there we have this:

    if (!submodule_has_commits(path, commits))
        /*
         * NOTE: We do consider it safe to return "no" here. The
         * correct answer would be "We do not know" instead of
         * "No push needed", but it is quite hard to change
         * the submodule pointer without having the submodule
         * around. If a user did however change the submodules
         * without having the submodule around, this indicates
         * an expert who knows what they are doing or a
         * maintainer integrating work from other people. In
         * both cases it should be safe to skip this check.
         */
        return 0;

So if the check, whether a submodule has commits, fails for any reason it will
not end up in the list to be pushed.

As a side note: inside submodule_has_commits() there is an add_submodule_odb()
followed by a process to really make sure that the commits are in the
submodule.

So IMO at this point we can be sure that the *database* exists and this extra
check could be dropped if we said that a caller to push_submodule() should make
sure that the submodule exists. The current ones are doing it already (if I did
not miss anything).

On Tue, Aug 15, 2017 at 06:05:25PM -0700, Stefan Beller wrote:
> On Tue, Aug 15, 2017 at 5:11 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Stefan Beller <sbeller@google.com> writes:
> >
> >>> Is "is it populated" a good thing to check here, though?  IIRC,
> >>> add-submodule-odb allows you to add the object database of an
> >>> inactivated submodule, so this seems to change the behaviour.  I do
> >>> not know if the behaviour change is a good thing (i.e. bugfix) or
> >>> not (i.e. regression) offhand, though.
> >>
> >> Good point, we should be able to push non-populated, even inactive(?)
> >> submodules. For that we strictly need add_submodule_odb here
> >> (or the repo object of the submodule, eventually).
> >>
> >> So let's retract this patch for now.
> >
> > Not so fast.
> 
> Ok, I took another look at the code.
> 
> While we may desire that un-populated submodules can be pushed
> (due to checking out another revision where the submodule
> doesn't exist, before pushing), this is not supported currently, because
> the call to run the push in the submodule assumes there is a
> "<path>/.git" on which the child process can operate.
> So for now we HAVE to have the submodule populated.

That is a good point though. In the current form of push_submodule() we need to
have a populated submodule. So IMO to check whether the submodule is actually
*populated* instead of adding the odb is correct and a possible bug fix.

Cheers Heiko

^ permalink raw reply	[relevance 24%]

* Re: Submodule regression in 2.14?
      [irrelevant] <4283F0B0-BC1C-4ED1-8126-7E512D84484B@gmail.com>
@ 2017-08-16 18:51 ` Stefan Beller
  2017-08-16 18:53   ` Stefan Beller
  2017-08-17 21:21   ` Lars Schneider
  0 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2017-08-16 18:51 UTC (permalink / raw)
  To: Lars Schneider; +Cc: git, Junio C Hamano

On Wed, Aug 16, 2017 at 11:35 AM, Lars Schneider
<larsxschneider@gmail.com> wrote:
> Hi,
>
> I think we discovered a regression in Git 2.14.1 today.
> It looks like as if "git submodule update --init --recursive" removes
> the "skip submodules" config.
>
> Consider the following steps:
>
>     git clone https://server/repo.git
>     cd repo
>     git config --local submodule.some/other/repo.update none
>     git submodule update --init --recursive
>     git pull --recurse-submodules
>
> With Git 2.14 the last "git pull" will clone the "some/other/repo"
> submodule. This did not happen with Git 2.13.
>
> Bug or feature? I don't have anymore time for Git today. I am happy to
> provide a proper test case tomorrow, though.

$ git log --oneline v2.13.0..v2.14.1 -- git-submodule.sh
532139940c add: warn when adding an embedded repository
(I am confident this is not the suspect, let's keep searching.
Not a lot happened in submodule land apparently)

Looking through all commits v2.13..v2.14 doesn't have me
suspect any of them.

Any chance the "did not happen with 2.13" was not
freshly cloned but tested on an existing repo? If so a hot
candidate for suspicion is a93dcb0a56 (Merge branch
'bw/submodule-is-active', 2017-03-30), IMHO, just
gut feeling, though.

Oh, wait.
$ git log --oneline v2.13.0..v2.14.1 -- builtin/pull.c
c9c63ee558 Merge branch 'sb/pull-rebase-submodule'
a6d7eb2c7a pull: optionally rebase submodules (remote submodule changes only)
could also be a culprit. Do you have pull.rebase set?

>
> Cheers,
> Lars

^ permalink raw reply	[relevance 26%]

* Re: Submodule regression in 2.14?
  2017-08-16 18:51 ` Submodule regression in 2.14? Stefan Beller
@ 2017-08-16 18:53   ` Stefan Beller
  2017-08-17 21:21   ` Lars Schneider
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2017-08-16 18:53 UTC (permalink / raw)
  To: Lars Schneider; +Cc: git, Junio C Hamano

On Wed, Aug 16, 2017 at 11:51 AM, Stefan Beller <sbeller@google.com> wrote:

> Any chance the "did not happen with 2.13" was not
> freshly cloned but tested on an existing repo? If so a hot
> candidate for suspicion is a93dcb0a56 (Merge branch
> 'bw/submodule-is-active', 2017-03-30), IMHO, just
> gut feeling, though.

which makes it a feature, I should add.

^ permalink raw reply	[relevance 24%]

* [PATCH] submodule.sh: remove unused variable
@ 2017-08-16 22:50 Stefan Beller
  2017-08-16 22:59 ` Jonathan Nieder
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2017-08-16 22:50 UTC (permalink / raw)
  To: jrnieder; +Cc: git, Stefan Beller

This could have been part of 48308681b0 (git submodule update: have a
dedicated helper for cloning, 2016-02-29).

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index e131760eec..9dcec7b356 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -611,7 +611,6 @@ cmd_update()
 		die_if_unmatched "$mode" "$sha1"
 
 		name=$(git submodule--helper name "$sm_path") || exit
-		url=$(git config submodule."$name".url)
 		if ! test -z "$update"
 		then
 			update_module=$update
-- 
2.14.0.rc0.3.g6c2e499285


^ permalink raw reply	[relevance 35%]

* Re: [PATCH] submodule.sh: remove unused variable
  2017-08-16 22:50 [PATCH] submodule.sh: remove unused variable Stefan Beller
@ 2017-08-16 22:59 ` Jonathan Nieder
  0 siblings, 0 replies; 200+ results
From: Jonathan Nieder @ 2017-08-16 22:59 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git

Stefan Beller wrote:

> This could have been part of 48308681b0 (git submodule update: have a
> dedicated helper for cloning, 2016-02-29).
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  git-submodule.sh | 1 -
>  1 file changed, 1 deletion(-)

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

Thanks for cleaning up.

> diff --git a/git-submodule.sh b/git-submodule.sh
> index e131760eec..9dcec7b356 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -611,7 +611,6 @@ cmd_update()
>  		die_if_unmatched "$mode" "$sha1"
>  
>  		name=$(git submodule--helper name "$sm_path") || exit
> -		url=$(git config submodule."$name".url)
>  		if ! test -z "$update"
>  		then
>  			update_module=$update
> -- 
> 2.14.0.rc0.3.g6c2e499285
> 

^ permalink raw reply	[relevance 16%]

* Re: [RFC PATCH 1/2] implement fetching of moved submodules
      [irrelevant] <20170817105349.GC52233@book.hvoigt.net>
@ 2017-08-17 17:20 ` Stefan Beller
  2017-08-18 16:06   ` Heiko Voigt
      [irrelevant] ` <20170817110013.GD52233@book.hvoigt.net>
  1 sibling, 1 reply; 200+ results
From: Stefan Beller @ 2017-08-17 17:20 UTC (permalink / raw)
  To: Heiko Voigt; +Cc: git, Jonathan Nieder, Jens Lehmann, Brandon Williams

On Thu, Aug 17, 2017 at 3:53 AM, Heiko Voigt <hvoigt@hvoigt.net> wrote:
> We store the changed submodules paths to calculate which submodule needs
> fetching. This does not work for moved submodules since their paths do
> not stay the same in case of a moved submodules. In case of new
> submodules we do not have a path in the current checkout, since they
> just appeared in this fetch.
>
> It is more general to collect the submodule names for changes instead of
> their paths to include the above cases.
>
> With the change described above we implement 'on-demand' fetching of
> changes in moved submodules.

This sounds as if this would also enable fetching new submodules
eventually?

> Note: This does only work when repositories have a .gitmodules file. In
> other words: It breaks if we do not get a name for a repository.
> IIRC, consensus was that this is a requirement to get nice submodule
> handling these days?

I think that should have been the consensus since ~1.7.8 (since the
submodules git dir can live inside the superprojects
<gitdir>/module/<name>).

A gitlink entry without corresponding .gitmodules entry is just a gitlink.
If we happen to have a repository at that path of the gitlink, we can
be nice and pretend like it is a functional submodule, but it really is
not. It's just another repo inside the superproject that happen to live
at the path of a gitlink.

> Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
> ---
>
> I updated the leftover code from my series implementing recursive fetch
> for moved submodules[1] to the current master.
>
> This breaks t5531 and t5545 because they do not use a .gitmodules file.
>
> I also have some code leftover that does fallback on paths in case no
> submodule names can be found. But I do not really like it. The question
> here is how far do we support not using .gitmodules. Is it e.g.
> reasonable to say: "For --recurse-submodules=on-demand you need a
> .gitmodules file?"

I would not intentionally break users here, but any new functionality can
safely assume (a) we have a proper .gitmodules entry or (b) it is not a
submodule, so do nothing/be extra careful.

For example in recursive diff sort of makes sense to also handle
non-submodule gitlinks, but fetch is harder to tell.

(just last night I was rereading
https://public-inbox.org/git/CAJo=hJvnAPNAdDcAAwAvU9C4RVeQdoS3Ev9WTguHx4fD0V_nOg@mail.gmail.com/
which I think is a super cute application of gitlinks. If you happen
to checkout such
a tree, you don't want to fetch all of the fake submodules)