* [PATCH 1/8] submodule-config: keep update strategy around
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
2016-02-03 23:09 ` Junio C Hamano
2016-02-02 17:51 ` [PATCH 2/8] submodule-config: drop check against NULL Stefan Beller
` (6 subsequent siblings)
7 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
We need the submodule update strategies in a later patch.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
submodule-config.c | 11 +++++++++++
submodule-config.h | 1 +
2 files changed, 12 insertions(+)
diff --git a/submodule-config.c b/submodule-config.c
index afe0ea8..4239b0e 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -194,6 +194,7 @@ static struct submodule *lookup_or_create_by_name(struct submodule_cache *cache,
submodule->path = NULL;
submodule->url = NULL;
+ submodule->update = NULL;
submodule->fetch_recurse = RECURSE_SUBMODULES_NONE;
submodule->ignore = NULL;
@@ -311,6 +312,16 @@ static int parse_config(const char *var, const char *value, void *data)
free((void *) submodule->url);
submodule->url = xstrdup(value);
}
+ } else if (!strcmp(item.buf, "update")) {
+ if (!value)
+ ret = config_error_nonbool(var);
+ else if (!me->overwrite && submodule->update != NULL)
+ warn_multiple_config(me->commit_sha1, submodule->name,
+ "update");
+ else {
+ free((void *) submodule->update);
+ submodule->update = xstrdup(value);
+ }
}
strbuf_release(&name);
diff --git a/submodule-config.h b/submodule-config.h
index 9061e4e..f9e2a29 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -14,6 +14,7 @@ struct submodule {
const char *url;
int fetch_recurse;
const char *ignore;
+ const char *update;
/* the sha1 blob id of the responsible .gitmodules file */
unsigned char gitmodules_sha1[20];
};
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 1/8] submodule-config: keep update strategy around
2016-02-02 17:51 ` [PATCH 1/8] submodule-config: keep update strategy around Stefan Beller
@ 2016-02-03 23:09 ` Junio C Hamano
2016-02-04 3:15 ` Jeff King
0 siblings, 1 reply; 28+ messages in thread
From: Junio C Hamano @ 2016-02-03 23:09 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, jrnieder, Jens.Lehmann
Stefan Beller <sbeller@google.com> writes:
> We need the submodule update strategies in a later patch.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> submodule-config.c | 11 +++++++++++
> submodule-config.h | 1 +
> 2 files changed, 12 insertions(+)
>
> diff --git a/submodule-config.c b/submodule-config.c
> index afe0ea8..4239b0e 100644
> --- a/submodule-config.c
> +++ b/submodule-config.c
> @@ -194,6 +194,7 @@ static struct submodule *lookup_or_create_by_name(struct submodule_cache *cache,
>
> submodule->path = NULL;
> submodule->url = NULL;
> + submodule->update = NULL;
> submodule->fetch_recurse = RECURSE_SUBMODULES_NONE;
> submodule->ignore = NULL;
>
> @@ -311,6 +312,16 @@ static int parse_config(const char *var, const char *value, void *data)
> free((void *) submodule->url);
> submodule->url = xstrdup(value);
> }
> + } else if (!strcmp(item.buf, "update")) {
> + if (!value)
> + ret = config_error_nonbool(var);
> + else if (!me->overwrite && submodule->update != NULL)
> + warn_multiple_config(me->commit_sha1, submodule->name,
> + "update");
> + else {
> + free((void *) submodule->update);
> + submodule->update = xstrdup(value);
This is a tangent, but whenever I see us needing to cast the
constness away to call free(), it makes me wonder how much value we
are getting by marking the field as a pointer to "const" in the
first place. With this patch alone, we cannot really see it,
because we cannot see how it is used.
> + }
> }
>
> strbuf_release(&name);
> diff --git a/submodule-config.h b/submodule-config.h
> index 9061e4e..f9e2a29 100644
> --- a/submodule-config.h
> +++ b/submodule-config.h
> @@ -14,6 +14,7 @@ struct submodule {
> const char *url;
> int fetch_recurse;
> const char *ignore;
> + const char *update;
> /* the sha1 blob id of the responsible .gitmodules file */
> unsigned char gitmodules_sha1[20];
> };
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 1/8] submodule-config: keep update strategy around
2016-02-03 23:09 ` Junio C Hamano
@ 2016-02-04 3:15 ` Jeff King
0 siblings, 0 replies; 28+ messages in thread
From: Jeff King @ 2016-02-04 3:15 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Stefan Beller, git, jrnieder, Jens.Lehmann
On Wed, Feb 03, 2016 at 03:09:06PM -0800, Junio C Hamano wrote:
> > + } else if (!strcmp(item.buf, "update")) {
> > + if (!value)
> > + ret = config_error_nonbool(var);
> > + else if (!me->overwrite && submodule->update != NULL)
> > + warn_multiple_config(me->commit_sha1, submodule->name,
> > + "update");
> > + else {
> > + free((void *) submodule->update);
> > + submodule->update = xstrdup(value);
>
> This is a tangent, but whenever I see us needing to cast the
> constness away to call free(), it makes me wonder how much value we
> are getting by marking the field as a pointer to "const" in the
> first place. With this patch alone, we cannot really see it,
> because we cannot see how it is used.
I suspect we may have just inherited this from other config code. We
typically use a const pointer to declare config strings, because
git_config_string() takes a pointer to a const pointer for its
destination. And we do that because some strings need BSS defaults, and
we cannot assign a string literal to a non-const pointer without a cast
(and if we did, it is inviting somebody to accidentally free() the
string literal).
So _somebody_ generally has to cast to cover all situations. But here we
are not using git_config_string() at all, so I think we could get away
with a non-const pointer.
As a tangent to your tangent, another problem with git_config_string()
and const pointers is that we never free the "old" value for a string,
and we leak it. It's usually OK in practice because people do not have
unbounded repetition of their config keys. We don't free because it
would be an error to do so for a default-assigned string literal. But
for any string variable which defaults to NULL, it would be fine.
IOW, git_config_string (and probably git_config_pathname) could be
implemented like this:
int git_config_string(char **dest, const char *var, const char *value)
{
if (!value)
return config_error_nonbool(var);
free(*dest);
*dest = xstrdup(value);
return 0;
}
and then:
char *git_foo;
...
return git_config_string(&git_foo, var, value);
would do the right thing. But I'm not sure how much that would ripple
through the code-base. Any string who wants:
const char *git_foo = "default";
would have to use another function (and keep the leak), or convert its
uses to treat NULL the same as "default". I have a suspicion that _most_
sites would prefer it the non-const way, and it would be a net win. But
I think you'd have to replace the function, then convert all of the
variables used (which the compiler will helpfully point out to you
because of the type mismatch), and then see how many are broken (which
the compiler will also tell you).
-Peff
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 2/8] submodule-config: drop check against NULL
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
2016-02-02 17:51 ` [PATCH 1/8] submodule-config: keep update strategy around Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
2016-02-03 23:09 ` Junio C Hamano
2016-02-02 17:51 ` [PATCH 3/8] submodule-config: remove name_and_item_from_var Stefan Beller
` (5 subsequent siblings)
7 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
Adhere to the common coding style of Git and not check explicitly
for NULL throughout the file. There are still other occurrences in the
code base but that is usually inside of conditions with side effects.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
submodule-config.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/submodule-config.c b/submodule-config.c
index 4239b0e..6d01941 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -265,7 +265,7 @@ static int parse_config(const char *var, const char *value, void *data)
if (!strcmp(item.buf, "path")) {
if (!value)
ret = config_error_nonbool(var);
- else if (!me->overwrite && submodule->path != NULL)
+ else if (!me->overwrite && submodule->path)
warn_multiple_config(me->commit_sha1, submodule->name,
"path");
else {
@@ -289,7 +289,7 @@ static int parse_config(const char *var, const char *value, void *data)
} else if (!strcmp(item.buf, "ignore")) {
if (!value)
ret = config_error_nonbool(var);
- else if (!me->overwrite && submodule->ignore != NULL)
+ else if (!me->overwrite && submodule->ignore)
warn_multiple_config(me->commit_sha1, submodule->name,
"ignore");
else if (strcmp(value, "untracked") &&
@@ -305,7 +305,7 @@ static int parse_config(const char *var, const char *value, void *data)
} else if (!strcmp(item.buf, "url")) {
if (!value) {
ret = config_error_nonbool(var);
- } else if (!me->overwrite && submodule->url != NULL) {
+ } else if (!me->overwrite && submodule->url) {
warn_multiple_config(me->commit_sha1, submodule->name,
"url");
} else {
@@ -315,7 +315,7 @@ static int parse_config(const char *var, const char *value, void *data)
} else if (!strcmp(item.buf, "update")) {
if (!value)
ret = config_error_nonbool(var);
- else if (!me->overwrite && submodule->update != NULL)
+ else if (!me->overwrite && submodule->update)
warn_multiple_config(me->commit_sha1, submodule->name,
"update");
else {
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 2/8] submodule-config: drop check against NULL
2016-02-02 17:51 ` [PATCH 2/8] submodule-config: drop check against NULL Stefan Beller
@ 2016-02-03 23:09 ` Junio C Hamano
0 siblings, 0 replies; 28+ messages in thread
From: Junio C Hamano @ 2016-02-03 23:09 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, jrnieder, Jens.Lehmann
Stefan Beller <sbeller@google.com> writes:
> Adhere to the common coding style of Git and not check explicitly
> for NULL throughout the file. There are still other occurrences in the
> code base but that is usually inside of conditions with side effects.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> submodule-config.c | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/submodule-config.c b/submodule-config.c
> index 4239b0e..6d01941 100644
> --- a/submodule-config.c
> +++ b/submodule-config.c
> @@ -265,7 +265,7 @@ static int parse_config(const char *var, const char *value, void *data)
> if (!strcmp(item.buf, "path")) {
> if (!value)
> ret = config_error_nonbool(var);
> - else if (!me->overwrite && submodule->path != NULL)
> + else if (!me->overwrite && submodule->path)
> warn_multiple_config(me->commit_sha1, submodule->name,
> "path");
Good.
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 3/8] submodule-config: remove name_and_item_from_var
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
2016-02-02 17:51 ` [PATCH 1/8] submodule-config: keep update strategy around Stefan Beller
2016-02-02 17:51 ` [PATCH 2/8] submodule-config: drop check against NULL Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
2016-02-03 23:09 ` Junio C Hamano
2016-02-02 17:51 ` [PATCH 4/8] submodule-config: introduce parse_generic_submodule_config Stefan Beller
` (4 subsequent siblings)
7 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
`name_and_item_from_var` does not provide the proper abstraction
we need here in a later patch.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
submodule-config.c | 48 ++++++++++++++++--------------------------------
1 file changed, 16 insertions(+), 32 deletions(-)
diff --git a/submodule-config.c b/submodule-config.c
index 6d01941..b826841 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -161,31 +161,17 @@ static struct submodule *cache_lookup_name(struct submodule_cache *cache,
return NULL;
}
-static int name_and_item_from_var(const char *var, struct strbuf *name,
- struct strbuf *item)
-{
- const char *subsection, *key;
- int subsection_len, parse;
- parse = parse_config_key(var, "submodule", &subsection,
- &subsection_len, &key);
- if (parse < 0 || !subsection)
- return 0;
-
- strbuf_add(name, subsection, subsection_len);
- strbuf_addstr(item, key);
-
- return 1;
-}
-
static struct submodule *lookup_or_create_by_name(struct submodule_cache *cache,
- const unsigned char *gitmodules_sha1, const char *name)
+ const unsigned char *gitmodules_sha1,
+ const char *name_ptr, int name_len)
{
struct submodule *submodule;
struct strbuf name_buf = STRBUF_INIT;
+ char *name = xmemdupz(name_ptr, name_len);
submodule = cache_lookup_name(cache, gitmodules_sha1, name);
if (submodule)
- return submodule;
+ goto out;
submodule = xmalloc(sizeof(*submodule));
@@ -201,7 +187,8 @@ static struct submodule *lookup_or_create_by_name(struct submodule_cache *cache,
hashcpy(submodule->gitmodules_sha1, gitmodules_sha1);
cache_add(cache, submodule);
-
+out:
+ free(name);
return submodule;
}
@@ -251,18 +238,18 @@ static int parse_config(const char *var, const char *value, void *data)
{
struct parse_config_parameter *me = data;
struct submodule *submodule;
- struct strbuf name = STRBUF_INIT, item = STRBUF_INIT;
- int ret = 0;
+ int subsection_len, ret = 0;
+ const char *subsection, *key;
- /* this also ensures that we only parse submodule entries */
- if (!name_and_item_from_var(var, &name, &item))
+ if (parse_config_key(var, "submodule", &subsection,
+ &subsection_len, &key) < 0 || !subsection_len)
return 0;
submodule = lookup_or_create_by_name(me->cache,
me->gitmodules_sha1,
- name.buf);
+ subsection, subsection_len);
- if (!strcmp(item.buf, "path")) {
+ if (!strcmp(key, "path")) {
if (!value)
ret = config_error_nonbool(var);
else if (!me->overwrite && submodule->path)
@@ -275,7 +262,7 @@ static int parse_config(const char *var, const char *value, void *data)
submodule->path = xstrdup(value);
cache_put_path(me->cache, submodule);
}
- } else if (!strcmp(item.buf, "fetchrecursesubmodules")) {
+ } else if (!strcmp(key, "fetchrecursesubmodules")) {
/* when parsing worktree configurations we can die early */
int die_on_error = is_null_sha1(me->gitmodules_sha1);
if (!me->overwrite &&
@@ -286,7 +273,7 @@ static int parse_config(const char *var, const char *value, void *data)
submodule->fetch_recurse = parse_fetch_recurse(
var, value,
die_on_error);
- } else if (!strcmp(item.buf, "ignore")) {
+ } else if (!strcmp(key, "ignore")) {
if (!value)
ret = config_error_nonbool(var);
else if (!me->overwrite && submodule->ignore)
@@ -302,7 +289,7 @@ static int parse_config(const char *var, const char *value, void *data)
free((void *) submodule->ignore);
submodule->ignore = xstrdup(value);
}
- } else if (!strcmp(item.buf, "url")) {
+ } else if (!strcmp(key, "url")) {
if (!value) {
ret = config_error_nonbool(var);
} else if (!me->overwrite && submodule->url) {
@@ -312,7 +299,7 @@ static int parse_config(const char *var, const char *value, void *data)
free((void *) submodule->url);
submodule->url = xstrdup(value);
}
- } else if (!strcmp(item.buf, "update")) {
+ } else if (!strcmp(key, "update")) {
if (!value)
ret = config_error_nonbool(var);
else if (!me->overwrite && submodule->update)
@@ -324,9 +311,6 @@ static int parse_config(const char *var, const char *value, void *data)
}
}
- strbuf_release(&name);
- strbuf_release(&item);
-
return ret;
}
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 3/8] submodule-config: remove name_and_item_from_var
2016-02-02 17:51 ` [PATCH 3/8] submodule-config: remove name_and_item_from_var Stefan Beller
@ 2016-02-03 23:09 ` Junio C Hamano
0 siblings, 0 replies; 28+ messages in thread
From: Junio C Hamano @ 2016-02-03 23:09 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, jrnieder, Jens.Lehmann
Stefan Beller <sbeller@google.com> writes:
> `name_and_item_from_var` does not provide the proper abstraction
> we need here in a later patch.
... so the idea is to first open-code the calling site in this
patch, and bring a different abstraction that is better suited to
this machinery in a later step? Makes sense, especially it only
have a single callsite--let's continue reading...
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> submodule-config.c | 48 ++++++++++++++++--------------------------------
> 1 file changed, 16 insertions(+), 32 deletions(-)
>
> diff --git a/submodule-config.c b/submodule-config.c
> index 6d01941..b826841 100644
> --- a/submodule-config.c
> +++ b/submodule-config.c
> @@ -161,31 +161,17 @@ static struct submodule *cache_lookup_name(struct submodule_cache *cache,
> return NULL;
> }
>
> -static int name_and_item_from_var(const char *var, struct strbuf *name,
> - struct strbuf *item)
> -{
> - const char *subsection, *key;
> - int subsection_len, parse;
> - parse = parse_config_key(var, "submodule", &subsection,
> - &subsection_len, &key);
> - if (parse < 0 || !subsection)
> - return 0;
> -
> - strbuf_add(name, subsection, subsection_len);
> - strbuf_addstr(item, key);
> -
> - return 1;
> -}
> -
> static struct submodule *lookup_or_create_by_name(struct submodule_cache *cache,
> - const unsigned char *gitmodules_sha1, const char *name)
> + const unsigned char *gitmodules_sha1,
> + const char *name_ptr, int name_len)
> {
> struct submodule *submodule;
> struct strbuf name_buf = STRBUF_INIT;
> + char *name = xmemdupz(name_ptr, name_len);
This used to be fed the whole name, but now the caller uses
parse_config_key() and we get a <ptr,len> pair, hence a pair of new
variable allocation and release in this function. OK.
> submodule = cache_lookup_name(cache, gitmodules_sha1, name);
> if (submodule)
> - return submodule;
> + goto out;
>
> submodule = xmalloc(sizeof(*submodule));
>
> @@ -201,7 +187,8 @@ static struct submodule *lookup_or_create_by_name(struct submodule_cache *cache,
> hashcpy(submodule->gitmodules_sha1, gitmodules_sha1);
>
> cache_add(cache, submodule);
> -
> +out:
> + free(name);
> return submodule;
> }
>
> @@ -251,18 +238,18 @@ static int parse_config(const char *var, const char *value, void *data)
> {
> struct parse_config_parameter *me = data;
> struct submodule *submodule;
> - struct strbuf name = STRBUF_INIT, item = STRBUF_INIT;
> - int ret = 0;
> + int subsection_len, ret = 0;
> + const char *subsection, *key;
>
> - /* this also ensures that we only parse submodule entries */
> - if (!name_and_item_from_var(var, &name, &item))
> + if (parse_config_key(var, "submodule", &subsection,
> + &subsection_len, &key) < 0 || !subsection_len)
> return 0;
The strbuf name became subsection ptr,len pair [*1*].
> submodule = lookup_or_create_by_name(me->cache,
> me->gitmodules_sha1,
> - name.buf);
> + subsection, subsection_len);
... and that is used here.
>
> - if (!strcmp(item.buf, "path")) {
> + if (!strcmp(key, "path")) {
... and the "item" that used to hold the third-level variable name
is now a variable "key". Looks correct.
[Footnote]
*1* I notice that the error detection has been slightly changed.
The original code used to detect a two-level variable name by
checking if the subsection pointer is NULL. Now you check the
length.
I didn't check the implementation of parse_config_key(), but it
feels to me that the original would be "more" correct. Even
though we may not accept an empty second level name (or do we
already?), it would be a good idea to get the code prepared to
differentiate between the lack of subsection and having a
subsection that happens to be an empty string, i.e.
[section ""]
variable = value
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 4/8] submodule-config: introduce parse_generic_submodule_config
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
` (2 preceding siblings ...)
2016-02-02 17:51 ` [PATCH 3/8] submodule-config: remove name_and_item_from_var Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
2016-02-03 23:09 ` Junio C Hamano
2016-02-02 17:51 ` [PATCH 5/8] fetching submodules: respect `submodule.fetchJobs` config option Stefan Beller
` (3 subsequent siblings)
7 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
This rewrites parse_config to distinguish between configs specific to
one submodule and configs which apply generically to all submodules.
We do not have generic submodule configs yet, but the next patch will
introduce "submodule.fetchJobs".
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
submodule-config.c | 41 ++++++++++++++++++++++++++++++++---------
1 file changed, 32 insertions(+), 9 deletions(-)
diff --git a/submodule-config.c b/submodule-config.c
index b826841..29e21b2 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -234,17 +234,22 @@ struct parse_config_parameter {
int overwrite;
};
-static int parse_config(const char *var, const char *value, void *data)
+static int parse_generic_submodule_config(const char *key,
+ const char *var,
+ const char *value,
+ struct parse_config_parameter *me)
{
- struct parse_config_parameter *me = data;
- struct submodule *submodule;
- int subsection_len, ret = 0;
- const char *subsection, *key;
-
- if (parse_config_key(var, "submodule", &subsection,
- &subsection_len, &key) < 0 || !subsection_len)
- return 0;
+ return 0;
+}
+static int parse_specific_submodule_config(const char *subsection, int subsection_len,
+ const char *key,
+ const char *var,
+ const char *value,
+ struct parse_config_parameter *me)
+{
+ int ret = 0;
+ struct submodule *submodule;
submodule = lookup_or_create_by_name(me->cache,
me->gitmodules_sha1,
subsection, subsection_len);
@@ -314,6 +319,24 @@ static int parse_config(const char *var, const char *value, void *data)
return ret;
}
+static int parse_config(const char *var, const char *value, void *data)
+{
+ struct parse_config_parameter *me = data;
+ int subsection_len;
+ const char *subsection, *key;
+
+ if (parse_config_key(var, "submodule", &subsection,
+ &subsection_len, &key) < 0)
+ return 0;
+
+ if (!subsection_len)
+ return parse_generic_submodule_config(key, var, value, me);
+ else
+ return parse_specific_submodule_config(subsection,
+ subsection_len, key,
+ var, value, me);
+}
+
static int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,
unsigned char *gitmodules_sha1)
{
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 4/8] submodule-config: introduce parse_generic_submodule_config
2016-02-02 17:51 ` [PATCH 4/8] submodule-config: introduce parse_generic_submodule_config Stefan Beller
@ 2016-02-03 23:09 ` Junio C Hamano
2016-02-03 23:26 ` Stefan Beller
0 siblings, 1 reply; 28+ messages in thread
From: Junio C Hamano @ 2016-02-03 23:09 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, jrnieder, Jens.Lehmann
Stefan Beller <sbeller@google.com> writes:
> This rewrites parse_config to distinguish between configs specific to
> one submodule and configs which apply generically to all submodules.
> We do not have generic submodule configs yet, but the next patch will
> introduce "submodule.fetchJobs".
OK.
> +static int parse_config(const char *var, const char *value, void *data)
> +{
> + struct parse_config_parameter *me = data;
> + int subsection_len;
> + const char *subsection, *key;
> +
> + if (parse_config_key(var, "submodule", &subsection,
> + &subsection_len, &key) < 0)
> + return 0;
> +
> + if (!subsection_len)
> + return parse_generic_submodule_config(key, var, value, me);
The same comment as in the footnote for the review on [3/8] applies
here.
> + else
> + return parse_specific_submodule_config(subsection,
> + subsection_len, key,
> + var, value, me);
> +}
> +
> static int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,
> unsigned char *gitmodules_sha1)
> {
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 4/8] submodule-config: introduce parse_generic_submodule_config
2016-02-03 23:09 ` Junio C Hamano
@ 2016-02-03 23:26 ` Stefan Beller
2016-02-04 0:51 ` Junio C Hamano
0 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-03 23:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git@vger.kernel.org, Jonathan Nieder, Jens Lehmann
On Wed, Feb 3, 2016 at 3:09 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> This rewrites parse_config to distinguish between configs specific to
>> one submodule and configs which apply generically to all submodules.
>> We do not have generic submodule configs yet, but the next patch will
>> introduce "submodule.fetchJobs".
>
> OK.
>
>> +static int parse_config(const char *var, const char *value, void *data)
>> +{
>> + struct parse_config_parameter *me = data;
>> + int subsection_len;
>> + const char *subsection, *key;
>> +
>> + if (parse_config_key(var, "submodule", &subsection,
>> + &subsection_len, &key) < 0)
>> + return 0;
>> +
>> + if (!subsection_len)
>> + return parse_generic_submodule_config(key, var, value, me);
>
> The same comment as in the footnote for the review on [3/8] applies
> here.
Ok, so we want to have
[section ""]
and
[section]
be different, such that we could have a submodule with an "" name,
but still be able to differentiate to the common section which should be
applied to all submodules?
It sounds like a corner case to me, but if we want to guarantee such a behavior,
let me see if test cases for that need some special love in this series.
Thanks,
Stefan
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 4/8] submodule-config: introduce parse_generic_submodule_config
2016-02-03 23:26 ` Stefan Beller
@ 2016-02-04 0:51 ` Junio C Hamano
0 siblings, 0 replies; 28+ messages in thread
From: Junio C Hamano @ 2016-02-04 0:51 UTC (permalink / raw)
To: Stefan Beller; +Cc: git@vger.kernel.org, Jonathan Nieder, Jens Lehmann
Stefan Beller <sbeller@google.com> writes:
>> The same comment as in the footnote for the review on [3/8] applies
>> here.
>
> Ok, so we want to have
>
> [section ""]
>
> and
>
> [section]
>
> be different, such that we could have a submodule with an "" name,
> but still be able to differentiate to the common section which should be
> applied to all submodules?
It is not about submodule per-se, but what is the right way to use
the parse_config_key() function. If the caller is interested in
seeing if the name is two-level or three-level, NULL-ness of the
subsection pointer is the right thing to check, not the length of
the subsection (set to 0 or positive number if exists--and otherwise
it is set to 0).
People copy & paste existing uses of API functions, and even if for
the specific use of your code (i.e. submodule) it is illegal to have
an empty second level, we shouldn't leaving a bad practice in the
code to be imitated.
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 5/8] fetching submodules: respect `submodule.fetchJobs` config option
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
` (3 preceding siblings ...)
2016-02-02 17:51 ` [PATCH 4/8] submodule-config: introduce parse_generic_submodule_config Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
2016-02-03 23:09 ` Junio C Hamano
2016-02-02 17:51 ` [PATCH 6/8] git submodule update: have a dedicated helper for cloning Stefan Beller
` (2 subsequent siblings)
7 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
This allows to configure fetching and updating in parallel
without having the command line option.
This moved the responsibility to determine how many parallel processes
to start from builtin/fetch to submodule.c as we need a way to communicate
"The user did not specify the number of parallel processes in the command
line options" in the builtin fetch. The submodule code takes care of
the precedence (CLI > config > default)
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Documentation/config.txt | 7 +++++++
builtin/fetch.c | 2 +-
submodule-config.c | 15 +++++++++++++++
submodule-config.h | 2 ++
submodule.c | 5 +++++
t/t5526-fetch-submodules.sh | 14 ++++++++++++++
6 files changed, 44 insertions(+), 1 deletion(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 2d06b11..eda3276 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2646,6 +2646,13 @@ submodule.<name>.ignore::
"--ignore-submodules" option. The 'git submodule' commands are not
affected by this setting.
+submodule.fetchJobs::
+ This is used to determine how many submodules will be
+ fetched/cloned at the same time. Specifying a positive integer
+ allows up to that number of submodules being fetched in parallel.
+ This is used in fetch and clone operations only. A value of 0 will
+ give some reasonable configuration. It defaults to 1.
+
tag.sort::
This variable controls the sort ordering of tags when displayed by
linkgit:git-tag[1]. Without the "--sort=<value>" option provided, the
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 586840d..5aa1c2d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -37,7 +37,7 @@ static int prune = -1; /* unspecified */
static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity;
static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
static int tags = TAGS_DEFAULT, unshallow, update_shallow;
-static int max_children = 1;
+static int max_children = -1;
static const char *depth;
static const char *upload_pack;
static struct strbuf default_rla = STRBUF_INIT;
diff --git a/submodule-config.c b/submodule-config.c
index 29e21b2..a32259e 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -32,6 +32,7 @@ enum lookup_type {
static struct submodule_cache cache;
static int is_cache_init;
+static int parallel_jobs = -1;
static int config_path_cmp(const struct submodule_entry *a,
const struct submodule_entry *b,
@@ -239,6 +240,15 @@ static int parse_generic_submodule_config(const char *key,
const char *value,
struct parse_config_parameter *me)
{
+ if (!strcmp(key, "fetchjobs")) {
+ parallel_jobs = strtol(value, NULL, 10);
+ if (parallel_jobs < 0) {
+ warning("submodule.fetchJobs not allowed to be negative.");
+ parallel_jobs = 1;
+ return 1;
+ }
+ }
+
return 0;
}
@@ -482,3 +492,8 @@ void submodule_free(void)
cache_free(&cache);
is_cache_init = 0;
}
+
+int config_parallel_submodules(void)
+{
+ return parallel_jobs;
+}
diff --git a/submodule-config.h b/submodule-config.h
index f9e2a29..d9bbf9a 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -27,4 +27,6 @@ const struct submodule *submodule_from_path(const unsigned char *commit_sha1,
const char *path);
void submodule_free(void);
+int config_parallel_submodules(void);
+
#endif /* SUBMODULE_CONFIG_H */
diff --git a/submodule.c b/submodule.c
index b83939c..eb7d54b 100644
--- a/submodule.c
+++ b/submodule.c
@@ -751,6 +751,11 @@ int fetch_populated_submodules(const struct argv_array *options,
argv_array_push(&spf.args, "--recurse-submodules-default");
/* default value, "--submodule-prefix" and its value are added later */
+ if (max_parallel_jobs < 0)
+ max_parallel_jobs = config_parallel_submodules();
+ if (max_parallel_jobs < 0)
+ max_parallel_jobs = 1;
+
calculate_changed_submodule_paths();
run_processes_parallel(max_parallel_jobs,
get_next_submodule,
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 1241146..954d0e4 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -471,4 +471,18 @@ test_expect_success "don't fetch submodule when newly recorded commits are alrea
test_i18ncmp expect.err actual.err
'
+test_expect_success 'fetching submodules respects parallel settings' '
+ git config fetch.recurseSubmodules true &&
+ (
+ cd downstream &&
+ GIT_TRACE=$(pwd)/trace.out git fetch --jobs 7 &&
+ grep "7 tasks" trace.out &&
+ git config submodule.fetchJobs 8 &&
+ GIT_TRACE=$(pwd)/trace.out git fetch &&
+ grep "8 tasks" trace.out &&
+ GIT_TRACE=$(pwd)/trace.out git fetch --jobs 9 &&
+ grep "9 tasks" trace.out
+ )
+'
+
test_done
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 5/8] fetching submodules: respect `submodule.fetchJobs` config option
2016-02-02 17:51 ` [PATCH 5/8] fetching submodules: respect `submodule.fetchJobs` config option Stefan Beller
@ 2016-02-03 23:09 ` Junio C Hamano
0 siblings, 0 replies; 28+ messages in thread
From: Junio C Hamano @ 2016-02-03 23:09 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, jrnieder, Jens.Lehmann
Stefan Beller <sbeller@google.com> writes:
> This allows to configure fetching and updating in parallel
> without having the command line option.
>
> This moved the responsibility to determine how many parallel processes
> to start from builtin/fetch to submodule.c as we need a way to communicate
> "The user did not specify the number of parallel processes in the command
> line options" in the builtin fetch. The submodule code takes care of
> the precedence (CLI > config > default)
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> Documentation/config.txt | 7 +++++++
> builtin/fetch.c | 2 +-
> submodule-config.c | 15 +++++++++++++++
> submodule-config.h | 2 ++
> submodule.c | 5 +++++
> t/t5526-fetch-submodules.sh | 14 ++++++++++++++
> 6 files changed, 44 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 2d06b11..eda3276 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -2646,6 +2646,13 @@ submodule.<name>.ignore::
> "--ignore-submodules" option. The 'git submodule' commands are not
> affected by this setting.
>
> +submodule.fetchJobs::
> + This is used to determine how many submodules will be
> + fetched/cloned at the same time. Specifying a positive integer
> + allows up to that number of submodules being fetched in parallel.
s/being //? Either would be fine, but shorter might be better? I
dunno.
s/is used to determine/specifies/? That may shorten the whole
thing.
Specifies how many submodules are fetched/cloned at the same
time. A positive integer allows up to that number of ...
> + This is used in fetch and clone operations only. A value of 0 will
> + give some reasonable configuration. It defaults to 1.
s/used/respected/, perhaps?
s/reasonable configuration/reasonable default/, perhaps?
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index 586840d..5aa1c2d 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -37,7 +37,7 @@ static int prune = -1; /* unspecified */
> static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity;
> static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
> static int tags = TAGS_DEFAULT, unshallow, update_shallow;
> -static int max_children = 1;
> +static int max_children = -1;
> static const char *depth;
> static const char *upload_pack;
> static struct strbuf default_rla = STRBUF_INIT;
> diff --git a/submodule-config.c b/submodule-config.c
> index 29e21b2..a32259e 100644
> --- a/submodule-config.c
> +++ b/submodule-config.c
> @@ -32,6 +32,7 @@ enum lookup_type {
>
> static struct submodule_cache cache;
> static int is_cache_init;
> +static int parallel_jobs = -1;
>
> static int config_path_cmp(const struct submodule_entry *a,
> const struct submodule_entry *b,
> @@ -239,6 +240,15 @@ static int parse_generic_submodule_config(const char *key,
> const char *value,
> struct parse_config_parameter *me)
> {
> + if (!strcmp(key, "fetchjobs")) {
> + parallel_jobs = strtol(value, NULL, 10);
We do not notice trailing 'x' in "[submodule] fetchjobs = 10x";
intended? I am wondering this is a good excuse to invent
git_parse_long() and git_config_long() that do not exist yet.
In general, I'd personally prefer to reduce the number of calls to
strto[u]l that does not check trailing garbage, not increase it.
If we reject negatives anyway, we might be able to just use
git_parse_ulong() without adding new "signed" variants.
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 6/8] git submodule update: have a dedicated helper for cloning
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
` (4 preceding siblings ...)
2016-02-02 17:51 ` [PATCH 5/8] fetching submodules: respect `submodule.fetchJobs` config option Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
2016-02-03 23:24 ` Junio C Hamano
2016-02-02 17:51 ` [PATCH 7/8] submodule update: expose parallelism to the user Stefan Beller
2016-02-02 17:51 ` [PATCH 8/8] clone: allow an explicit argument for parallel submodule clones Stefan Beller
7 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
This introduces a new helper function in git submodule--helper
which takes care of cloning all submodules, which we want to
parallelize eventually.
Some tests (such as empty URL, update_mode=none) are required in the
helper to make the decision for cloning. These checks have been
moved into the C function as well (no need to repeat them in the
shell script).
As we can only access the stderr channel from within the parallel
processing engine, we need to reroute the error message for
specified but initialized submodules to stderr. As it is an error
message, this should have gone to stderr in the first place, so it
is a bug fix along the way.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
builtin/submodule--helper.c | 229 ++++++++++++++++++++++++++++++++++++++++++++
git-submodule.sh | 45 +++------
t/t7400-submodule-basic.sh | 4 +-
3 files changed, 242 insertions(+), 36 deletions(-)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f4c3eff..9441f20 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -255,6 +255,234 @@ static int module_clone(int argc, const char **argv, const char *prefix)
return 0;
}
+static int git_submodule_config(const char *var, const char *value, void *cb)
+{
+ return parse_submodule_config_option(var, value);
+}
+
+struct submodule_update_clone {
+ /* states */
+ int count;
+ int print_unmatched;
+ /* configuration */
+ int quiet;
+ const char *reference;
+ const char *depth;
+ const char *update;
+ const char *recursive_prefix;
+ const char *prefix;
+ struct module_list list;
+ struct string_list projectlines;
+ struct pathspec pathspec;
+};
+#define SUBMODULE_UPDATE_CLONE_INIT {0, 0, 0, NULL, NULL, NULL, NULL, NULL, MODULE_LIST_INIT, STRING_LIST_INIT_DUP}
+
+static void fill_clone_command(struct child_process *cp, int quiet,
+ const char *prefix, const char *path,
+ const char *name, const char *url,
+ const char *reference, const char *depth)
+{
+ cp->git_cmd = 1;
+ cp->no_stdin = 1;
+ cp->stdout_to_stderr = 1;
+ cp->err = -1;
+ argv_array_push(&cp->args, "submodule--helper");
+ argv_array_push(&cp->args, "clone");
+ if (quiet)
+ argv_array_push(&cp->args, "--quiet");
+
+ if (prefix)
+ argv_array_pushl(&cp->args, "--prefix", prefix, NULL);
+
+ argv_array_pushl(&cp->args, "--path", path, NULL);
+ argv_array_pushl(&cp->args, "--name", name, NULL);
+ argv_array_pushl(&cp->args, "--url", url, NULL);
+ if (reference)
+ argv_array_push(&cp->args, reference);
+ if (depth)
+ argv_array_push(&cp->args, depth);
+}
+
+static int update_clone_get_next_task(struct child_process *cp,
+ struct strbuf *err,
+ void *pp_cb,
+ void **pp_task_cb)
+{
+ struct submodule_update_clone *pp = pp_cb;
+
+ for (; pp->count < pp->list.nr; pp->count++) {
+ const struct submodule *sub = NULL;
+ const char *displaypath = NULL;
+ const struct cache_entry *ce = pp->list.entries[pp->count];
+ struct strbuf sb = STRBUF_INIT;
+ const char *update_module = NULL;
+ char *url = NULL;
+ int needs_cloning = 0;
+
+ if (ce_stage(ce)) {
+ if (pp->recursive_prefix)
+ strbuf_addf(err, "Skipping unmerged submodule %s/%s\n",
+ pp->recursive_prefix, ce->name);
+ else
+ strbuf_addf(err, "Skipping unmerged submodule %s\n",
+ ce->name);
+ continue;
+ }
+
+ sub = submodule_from_path(null_sha1, ce->name);
+ if (!sub) {
+ strbuf_addf(err, "BUG: internal error managing submodules. "
+ "The cache could not locate '%s'", ce->name);
+ pp->print_unmatched = 1;
+ continue;
+ }
+
+ if (pp->recursive_prefix)
+ displaypath = relative_path(pp->recursive_prefix, ce->name, &sb);
+ else
+ displaypath = ce->name;
+
+ if (pp->update)
+ update_module = pp->update;
+ if (!update_module)
+ update_module = sub->update;
+ if (!update_module)
+ update_module = "checkout";
+ if (!strcmp(update_module, "none")) {
+ strbuf_addf(err, "Skipping submodule '%s'\n", displaypath);
+ continue;
+ }
+
+ /*
+ * Looking up the url in .git/config.
+ * We must not fall back to .gitmodules as we only want to process
+ * configured submodules.
+ */
+ strbuf_reset(&sb);
+ strbuf_addf(&sb, "submodule.%s.url", sub->name);
+ git_config_get_string(sb.buf, &url);
+ if (!url) {
+ /*
+ * Only mention uninitialized submodules when its
+ * path have been specified
+ */
+ if (pp->pathspec.nr)
+ strbuf_addf(err, _("Submodule path '%s' not initialized\n"
+ "Maybe you want to use 'update --init'?"), displaypath);
+ continue;
+ }
+
+ strbuf_reset(&sb);
+ strbuf_addf(&sb, "%s/.git", ce->name);
+ needs_cloning = !file_exists(sb.buf);
+
+ strbuf_reset(&sb);
+ strbuf_addf(&sb, "%06o %s %d %d\t%s\n", ce->ce_mode,
+ sha1_to_hex(ce->sha1), ce_stage(ce),
+ needs_cloning, ce->name);
+ string_list_append(&pp->projectlines, sb.buf);
+
+ if (needs_cloning) {
+ fill_clone_command(cp, pp->quiet, pp->prefix, ce->name,
+ sub->name, url, pp->reference, pp->depth);
+ pp->count++;
+ free(url);
+ return 1;
+ } else
+ free(url);
+ }
+ return 0;
+}
+
+static int update_clone_start_failure(struct child_process *cp,
+ struct strbuf *err,
+ void *pp_cb,
+ void *pp_task_cb)
+{
+ struct submodule_update_clone *pp = pp_cb;
+
+ strbuf_addf(err, "error when starting a child process");
+ pp->print_unmatched = 1;
+
+ return 1;
+}
+
+static int update_clone_task_finished(int result,
+ struct child_process *cp,
+ struct strbuf *err,
+ void *pp_cb,
+ void *pp_task_cb)
+{
+ struct submodule_update_clone *pp = pp_cb;
+
+ if (!result) {
+ return 0;
+ } else {
+ strbuf_addf(err, "error in one child process");
+ pp->print_unmatched = 1;
+ return 1;
+ }
+}
+
+static int update_clone(int argc, const char **argv, const char *prefix)
+{
+ struct string_list_item *item;
+ struct submodule_update_clone pp = SUBMODULE_UPDATE_CLONE_INIT;
+
+ struct option module_list_options[] = {
+ OPT_STRING(0, "prefix", &prefix,
+ N_("path"),
+ N_("path into the working tree")),
+ OPT_STRING(0, "recursive_prefix", &pp.recursive_prefix,
+ N_("path"),
+ N_("path into the working tree, across nested "
+ "submodule boundaries")),
+ OPT_STRING(0, "update", &pp.update,
+ N_("string"),
+ N_("update command for submodules")),
+ OPT_STRING(0, "reference", &pp.reference, "<repository>",
+ N_("Use the local reference repository "
+ "instead of a full clone")),
+ OPT_STRING(0, "depth", &pp.depth, "<depth>",
+ N_("Create a shallow clone truncated to the "
+ "specified number of revisions")),
+ OPT__QUIET(&pp.quiet, N_("do't print cloning progress")),
+ OPT_END()
+ };
+
+ const char *const git_submodule_helper_usage[] = {
+ N_("git submodule--helper list [--prefix=<path>] [<path>...]"),
+ NULL
+ };
+ pp.prefix = prefix;
+
+ argc = parse_options(argc, argv, prefix, module_list_options,
+ git_submodule_helper_usage, 0);
+
+ if (module_list_compute(argc, argv, prefix, &pp.pathspec, &pp.list) < 0) {
+ printf("#unmatched\n");
+ return 1;
+ }
+
+ gitmodules_config();
+ /* Overlay the parsed .gitmodules file with .git/config */
+ git_config(git_submodule_config, NULL);
+ run_processes_parallel(1, update_clone_get_next_task,
+ update_clone_start_failure,
+ update_clone_task_finished,
+ &pp);
+
+ if (pp.print_unmatched) {
+ printf("#unmatched\n");
+ return 1;
+ }
+
+ for_each_string_list_item(item, &pp.projectlines)
+ utf8_fprintf(stdout, "%s", item->string);
+
+ return 0;
+}
+
struct cmd_struct {
const char *cmd;
int (*fn)(int, const char **, const char *);
@@ -264,6 +492,7 @@ static struct cmd_struct commands[] = {
{"list", module_list},
{"name", module_name},
{"clone", module_clone},
+ {"update-clone", update_clone}
};
int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
diff --git a/git-submodule.sh b/git-submodule.sh
index 9bc5c5f..9f554fb 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -664,17 +664,18 @@ cmd_update()
cmd_init "--" "$@" || return
fi
- cloned_modules=
- git submodule--helper list --prefix "$wt_prefix" "$@" | {
+ git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
+ ${wt_prefix:+--prefix "$wt_prefix"} \
+ ${prefix:+--recursive_prefix "$prefix"} \
+ ${update:+--update "$update"} \
+ ${reference:+--reference "$reference"} \
+ ${depth:+--depth "$depth"} \
+ "$@" | {
err=
- while read mode sha1 stage sm_path
+ while read mode sha1 stage just_cloned sm_path
do
die_if_unmatched "$mode"
- if test "$stage" = U
- then
- echo >&2 "Skipping unmerged submodule $prefix$sm_path"
- continue
- fi
+
name=$(git submodule--helper name "$sm_path") || exit
url=$(git config submodule."$name".url)
branch=$(get_submodule_config "$name" branch master)
@@ -691,27 +692,10 @@ cmd_update()
displaypath=$(relative_path "$prefix$sm_path")
- if test "$update_module" = "none"
- then
- echo "Skipping submodule '$displaypath'"
- continue
- fi
-
- if test -z "$url"
- then
- # Only mention uninitialized submodules when its
- # path have been specified
- test "$#" != "0" &&
- say "$(eval_gettext "Submodule path '\$displaypath' not initialized
-Maybe you want to use 'update --init'?")"
- continue
- fi
-
- if ! test -d "$sm_path"/.git && ! test -f "$sm_path"/.git
+ if test $just_cloned -eq 1
then
- git submodule--helper clone ${GIT_QUIET:+--quiet} --prefix "$prefix" --path "$sm_path" --name "$name" --url "$url" "$reference" "$depth" || exit
- cloned_modules="$cloned_modules;$name"
subsha1=
+ update_module=checkout
else
subsha1=$(clear_local_git_env; cd "$sm_path" &&
git rev-parse --verify HEAD) ||
@@ -751,13 +735,6 @@ Maybe you want to use 'update --init'?")"
die "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
fi
- # Is this something we just cloned?
- case ";$cloned_modules;" in
- *";$name;"*)
- # then there is no local change to integrate
- update_module=checkout ;;
- esac
-
must_die_on_failure=
case "$update_module" in
checkout)
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 540771c..5991e3c 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -462,7 +462,7 @@ test_expect_success 'update --init' '
git config --remove-section submodule.example &&
test_must_fail git config submodule.example.url &&
- git submodule update init > update.out &&
+ git submodule update init 2> update.out &&
cat update.out &&
test_i18ngrep "not initialized" update.out &&
test_must_fail git rev-parse --resolve-git-dir init/.git &&
@@ -480,7 +480,7 @@ test_expect_success 'update --init from subdirectory' '
mkdir -p sub &&
(
cd sub &&
- git submodule update ../init >update.out &&
+ git submodule update ../init 2>update.out &&
cat update.out &&
test_i18ngrep "not initialized" update.out &&
test_must_fail git rev-parse --resolve-git-dir ../init/.git &&
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 6/8] git submodule update: have a dedicated helper for cloning
2016-02-02 17:51 ` [PATCH 6/8] git submodule update: have a dedicated helper for cloning Stefan Beller
@ 2016-02-03 23:24 ` Junio C Hamano
2016-02-03 23:41 ` Stefan Beller
0 siblings, 1 reply; 28+ messages in thread
From: Junio C Hamano @ 2016-02-03 23:24 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, jrnieder, Jens.Lehmann
Stefan Beller <sbeller@google.com> writes:
> + if (ce_stage(ce)) {
> + if (pp->recursive_prefix)
> + strbuf_addf(err, "Skipping unmerged submodule %s/%s\n",
> + pp->recursive_prefix, ce->name);
> + else
> + strbuf_addf(err, "Skipping unmerged submodule %s\n",
> + ce->name);
> + continue;
This raised my eyebrow somewhat, until I realized that it is OK
because module_list prepared by the caller has only one of the
unmerged entries for the same path to avoid duplicates.
> + }
> +
> + sub = submodule_from_path(null_sha1, ce->name);
> + if (!sub) {
> + strbuf_addf(err, "BUG: internal error managing submodules. "
> + "The cache could not locate '%s'", ce->name);
> + pp->print_unmatched = 1;
> + continue;
Interesting.
I am wondering if this check should go to module_list_compute().
> + }
> +
> + if (pp->recursive_prefix)
> + displaypath = relative_path(pp->recursive_prefix, ce->name, &sb);
> + else
> + displaypath = ce->name;
> +
> + if (pp->update)
> + update_module = pp->update;
> + if (!update_module)
> + update_module = sub->update;
> + if (!update_module)
> + update_module = "checkout";
> + if (!strcmp(update_module, "none")) {
> + strbuf_addf(err, "Skipping submodule '%s'\n", displaypath);
> + continue;
> + }
> +
> + /*
> + * Looking up the url in .git/config.
> + * We must not fall back to .gitmodules as we only want to process
> + * configured submodules.
> + */
> + strbuf_reset(&sb);
Doesn't this invalidate displaypath, when pp->recursive_prefix is in
effect? It still seems to be used to create an error message just a
few lines below...
> + strbuf_addf(&sb, "submodule.%s.url", sub->name);
> + git_config_get_string(sb.buf, &url);
> + if (!url) {
> + /*
> + * Only mention uninitialized submodules when its
> + * path have been specified
> + */
> + if (pp->pathspec.nr)
> + strbuf_addf(err, _("Submodule path '%s' not initialized\n"
> + "Maybe you want to use 'update --init'?"), displaypath);
> + continue;
> + }
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 6/8] git submodule update: have a dedicated helper for cloning
2016-02-03 23:24 ` Junio C Hamano
@ 2016-02-03 23:41 ` Stefan Beller
2016-02-04 0:54 ` Junio C Hamano
0 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-03 23:41 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git@vger.kernel.org, Jonathan Nieder, Jens Lehmann
On Wed, Feb 3, 2016 at 3:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> + if (ce_stage(ce)) {
>> + if (pp->recursive_prefix)
>> + strbuf_addf(err, "Skipping unmerged submodule %s/%s\n",
>> + pp->recursive_prefix, ce->name);
As a side question: Do we care about proper visual directory
separators in Windows?
>> + else
>> + strbuf_addf(err, "Skipping unmerged submodule %s\n",
>> + ce->name);
>> + continue;
>
> This raised my eyebrow somewhat, until I realized that it is OK
> because module_list prepared by the caller has only one of the
> unmerged entries for the same path to avoid duplicates.
>
>> + }
>> +
>> + sub = submodule_from_path(null_sha1, ce->name);
>> + if (!sub) {
>> + strbuf_addf(err, "BUG: internal error managing submodules. "
>> + "The cache could not locate '%s'", ce->name);
>> + pp->print_unmatched = 1;
>> + continue;
>
> Interesting.
>
> I am wondering if this check should go to module_list_compute().
Actually this has evolved from a debugging leftover camouflaged as a
useful thing.
I added that and then realized I did not add
gitmodules_config();
git_config(git_submodule_config, NULL);
before, but that code remained there as it seemed useful to me at the time.
I never run into this BUG after having proper initialization, so maybe it's not
worth carrying this code around. (We have many other places where
submodule_from_{path, name} is used unchecked, so why would this place
be special?)
>
>> + }
>> +
>> + if (pp->recursive_prefix)
>> + displaypath = relative_path(pp->recursive_prefix, ce->name, &sb);
>> + else
>> + displaypath = ce->name;
>> +
>> + if (pp->update)
>> + update_module = pp->update;
>> + if (!update_module)
>> + update_module = sub->update;
>> + if (!update_module)
>> + update_module = "checkout";
>> + if (!strcmp(update_module, "none")) {
>> + strbuf_addf(err, "Skipping submodule '%s'\n", displaypath);
>> + continue;
>> + }
>> +
>> + /*
>> + * Looking up the url in .git/config.
>> + * We must not fall back to .gitmodules as we only want to process
>> + * configured submodules.
>> + */
>> + strbuf_reset(&sb);
>
> Doesn't this invalidate displaypath, when pp->recursive_prefix is in
> effect? It still seems to be used to create an error message just a
> few lines below...
Yes that is programmer error. Also the final cleanup of the strbuf is missing.
>
>> + strbuf_addf(&sb, "submodule.%s.url", sub->name);
>> + git_config_get_string(sb.buf, &url);
>> + if (!url) {
>> + /*
>> + * Only mention uninitialized submodules when its
>> + * path have been specified
>> + */
>> + if (pp->pathspec.nr)
>> + strbuf_addf(err, _("Submodule path '%s' not initialized\n"
>> + "Maybe you want to use 'update --init'?"), displaypath);
>> + continue;
>> + }
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 6/8] git submodule update: have a dedicated helper for cloning
2016-02-03 23:41 ` Stefan Beller
@ 2016-02-04 0:54 ` Junio C Hamano
2016-02-04 20:22 ` Stefan Beller
0 siblings, 1 reply; 28+ messages in thread
From: Junio C Hamano @ 2016-02-04 0:54 UTC (permalink / raw)
To: Stefan Beller; +Cc: git@vger.kernel.org, Jonathan Nieder, Jens Lehmann
Stefan Beller <sbeller@google.com> writes:
> On Wed, Feb 3, 2016 at 3:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>
>>> + if (ce_stage(ce)) {
>>> + if (pp->recursive_prefix)
>>> + strbuf_addf(err, "Skipping unmerged submodule %s/%s\n",
>>> + pp->recursive_prefix, ce->name);
>
> As a side question: Do we care about proper visual directory
> separators in Windows?
You know I do not do Windows ;-) I'll leave the question for others
to answer (I am trying not to be one of the the only small number of
people who review code around here).
> I never run into this BUG after having proper initialization, so maybe it's not
> worth carrying this code around. (We have many other places where
> submodule_from_{path, name} is used unchecked, so why would this place
> be special?)
That is why I wondered if moudule_list() is a better place to do so.
That is where the list of everybody works on come from.
>>> + /*
>>> + * Looking up the url in .git/config.
>>> + * We must not fall back to .gitmodules as we only want to process
>>> + * configured submodules.
>>> + */
>>> + strbuf_reset(&sb);
>>
>> Doesn't this invalidate displaypath, when pp->recursive_prefix is in
>> effect? It still seems to be used to create an error message just a
>> few lines below...
>
> Yes that is programmer error. Also the final cleanup of the strbuf is missing.
Yeah, the calling convention to relative_path() might be performant,
but I agree that it is not the easiest API function to use correctly.
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 6/8] git submodule update: have a dedicated helper for cloning
2016-02-04 0:54 ` Junio C Hamano
@ 2016-02-04 20:22 ` Stefan Beller
2016-02-04 21:57 ` Junio C Hamano
0 siblings, 1 reply; 28+ messages in thread
From: Stefan Beller @ 2016-02-04 20:22 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git@vger.kernel.org, Jonathan Nieder, Jens Lehmann
On Wed, Feb 3, 2016 at 4:54 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> On Wed, Feb 3, 2016 at 3:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Stefan Beller <sbeller@google.com> writes:
>>>
>>>> + if (ce_stage(ce)) {
>>>> + if (pp->recursive_prefix)
>>>> + strbuf_addf(err, "Skipping unmerged submodule %s/%s\n",
>>>> + pp->recursive_prefix, ce->name);
>>
>> As a side question: Do we care about proper visual directory
>> separators in Windows?
>
> You know I do not do Windows ;-) I'll leave the question for others
> to answer (I am trying not to be one of the the only small number of
> people who review code around here).
>
>> I never run into this BUG after having proper initialization, so maybe it's not
>> worth carrying this code around. (We have many other places where
>> submodule_from_{path, name} is used unchecked, so why would this place
>> be special?)
>
> That is why I wondered if moudule_list() is a better place to do so.
> That is where the list of everybody works on come from.
I do not think that is a better place as not every consumer of module_list
(and module_list_compute as the nested function) will need to use the
submodule caching API. So these consumers are not interested in possible
bugs in the submodule cache API nor do they want the performance hit
which comes from checking unrelated stuff in there.
As said, I only saw this bug when the cache was not initialized properly,
and then such a bug is to be expected. I'd rather remove it in a reroll.
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 6/8] git submodule update: have a dedicated helper for cloning
2016-02-04 20:22 ` Stefan Beller
@ 2016-02-04 21:57 ` Junio C Hamano
0 siblings, 0 replies; 28+ messages in thread
From: Junio C Hamano @ 2016-02-04 21:57 UTC (permalink / raw)
To: Stefan Beller; +Cc: git@vger.kernel.org, Jonathan Nieder, Jens Lehmann
Stefan Beller <sbeller@google.com> writes:
> I do not think that is a better place as not every consumer of module_list
> (and module_list_compute as the nested function) will need to use the
> submodule caching API.
Ah, if that is the case, then OK.
I was imagining that module-list may someday would return a list of
"struct" instances each of which describes a submodule and has .name
and .path fields, or something, instead of just being a "here is the
path to the submodule, go look it up yourself" interface. But we
are not there (at least not yet--and I am not saying we agreed that
we would eventually want to go there)
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 7/8] submodule update: expose parallelism to the user
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
` (5 preceding siblings ...)
2016-02-02 17:51 ` [PATCH 6/8] git submodule update: have a dedicated helper for cloning Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
2016-02-02 17:51 ` [PATCH 8/8] clone: allow an explicit argument for parallel submodule clones Stefan Beller
7 siblings, 0 replies; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
Expose possible parallelism either via the "--jobs" CLI parameter or
the "submodule.fetchJobs" setting.
By having the variable initialized to -1, we make sure 0 can be passed
into the parallel processing machine, which will then pick as many parallel
workers as there are CPUs.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Documentation/git-submodule.txt | 7 ++++++-
builtin/submodule--helper.c | 18 ++++++++++++++----
git-submodule.sh | 9 +++++++++
t/t7406-submodule-update.sh | 12 ++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 1572f05..13adebf 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -16,7 +16,7 @@ SYNOPSIS
'git submodule' [--quiet] deinit [-f|--force] [--] <path>...
'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
[-f|--force] [--rebase|--merge] [--reference <repository>]
- [--depth <depth>] [--recursive] [--] [<path>...]
+ [--depth <depth>] [--recursive] [--jobs <n>] [--] [<path>...]
'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
[commit] [--] [<path>...]
'git submodule' [--quiet] foreach [--recursive] <command>
@@ -377,6 +377,11 @@ for linkgit:git-clone[1]'s `--reference` and `--shared` options carefully.
clone with a history truncated to the specified number of revisions.
See linkgit:git-clone[1]
+-j <n>::
+--jobs <n>::
+ This option is only valid for the update command.
+ Clone new submodules in parallel with as many jobs.
+ Defaults to the `submodule.fetchJobs` option.
<path>...::
Paths to submodule(s). When specified this will restrict the command
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 9441f20..8002187 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -426,6 +426,7 @@ static int update_clone_task_finished(int result,
static int update_clone(int argc, const char **argv, const char *prefix)
{
+ int max_jobs = -1;
struct string_list_item *item;
struct submodule_update_clone pp = SUBMODULE_UPDATE_CLONE_INIT;
@@ -446,6 +447,8 @@ static int update_clone(int argc, const char **argv, const char *prefix)
OPT_STRING(0, "depth", &pp.depth, "<depth>",
N_("Create a shallow clone truncated to the "
"specified number of revisions")),
+ OPT_INTEGER('j', "jobs", &max_jobs,
+ N_("parallel jobs")),
OPT__QUIET(&pp.quiet, N_("do't print cloning progress")),
OPT_END()
};
@@ -467,10 +470,17 @@ static int update_clone(int argc, const char **argv, const char *prefix)
gitmodules_config();
/* Overlay the parsed .gitmodules file with .git/config */
git_config(git_submodule_config, NULL);
- run_processes_parallel(1, update_clone_get_next_task,
- update_clone_start_failure,
- update_clone_task_finished,
- &pp);
+
+ if (max_jobs < 0)
+ max_jobs = config_parallel_submodules();
+ if (max_jobs < 0)
+ max_jobs = 1;
+
+ run_processes_parallel(max_jobs,
+ update_clone_get_next_task,
+ update_clone_start_failure,
+ update_clone_task_finished,
+ &pp);
if (pp.print_unmatched) {
printf("#unmatched\n");
diff --git a/git-submodule.sh b/git-submodule.sh
index 9f554fb..10c5af9 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -645,6 +645,14 @@ cmd_update()
--depth=*)
depth=$1
;;
+ -j|--jobs)
+ case "$2" in '') usage ;; esac
+ jobs="--jobs=$2"
+ shift
+ ;;
+ --jobs=*)
+ jobs=$1
+ ;;
--)
shift
break
@@ -670,6 +678,7 @@ cmd_update()
${update:+--update "$update"} \
${reference:+--reference "$reference"} \
${depth:+--depth "$depth"} \
+ ${jobs:+$jobs} \
"$@" | {
err=
while read mode sha1 stage just_cloned sm_path
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index dda3929..7fd5142 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -774,4 +774,16 @@ test_expect_success 'submodule update --recursive drops module name before recur
test_i18ngrep "Submodule path .deeper/submodule/subsubmodule.: checked out" actual
)
'
+
+test_expect_success 'submodule update can be run in parallel' '
+ (cd super2 &&
+ GIT_TRACE=$(pwd)/trace.out git submodule update --jobs 7 &&
+ grep "7 tasks" trace.out &&
+ git config submodule.fetchJobs 8 &&
+ GIT_TRACE=$(pwd)/trace.out git submodule update &&
+ grep "8 tasks" trace.out &&
+ GIT_TRACE=$(pwd)/trace.out git submodule update --jobs 9 &&
+ grep "9 tasks" trace.out
+ )
+'
test_done
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH 8/8] clone: allow an explicit argument for parallel submodule clones
2016-02-02 17:51 [PATCHv7 0/8] Expose submodule parallelism to the user Stefan Beller
` (6 preceding siblings ...)
2016-02-02 17:51 ` [PATCH 7/8] submodule update: expose parallelism to the user Stefan Beller
@ 2016-02-02 17:51 ` Stefan Beller
7 siblings, 0 replies; 28+ messages in thread
From: Stefan Beller @ 2016-02-02 17:51 UTC (permalink / raw)
To: git, jrnieder; +Cc: Jens.Lehmann, Stefan Beller, Junio C Hamano
Just pass it along to "git submodule update", which may pick reasonable
defaults if you don't specify an explicit number.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Documentation/git-clone.txt | 6 +++++-
builtin/clone.c | 19 +++++++++++++------
t/t7406-submodule-update.sh | 15 +++++++++++++++
3 files changed, 33 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 6bf000d..6db7b6d 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,7 +14,7 @@ SYNOPSIS
[-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]
[--dissociate] [--separate-git-dir <git dir>]
[--depth <depth>] [--[no-]single-branch]
- [--recursive | --recurse-submodules] [--] <repository>
+ [--recursive | --recurse-submodules] [--jobs <n>] [--] <repository>
[<directory>]
DESCRIPTION
@@ -221,6 +221,10 @@ objects from the source repository into a pack in the cloned repository.
The result is Git repository can be separated from working
tree.
+-j <n>::
+--jobs <n>::
+ The number of submodules fetched at the same time.
+ Defaults to the `submodule.fetchJobs` option.
<repository>::
The (possibly remote) repository to clone from. See the
diff --git a/builtin/clone.c b/builtin/clone.c
index a0b3cd9..b004fb4 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -50,6 +50,7 @@ static int option_progress = -1;
static struct string_list option_config;
static struct string_list option_reference;
static int option_dissociate;
+static int max_jobs = -1;
static struct option builtin_clone_options[] = {
OPT__VERBOSITY(&option_verbosity),
@@ -72,6 +73,8 @@ static struct option builtin_clone_options[] = {
N_("initialize submodules in the clone")),
OPT_BOOL(0, "recurse-submodules", &option_recursive,
N_("initialize submodules in the clone")),
+ OPT_INTEGER('j', "jobs", &max_jobs,
+ N_("number of submodules cloned in parallel")),
OPT_STRING(0, "template", &option_template, N_("template-directory"),
N_("directory from which templates will be used")),
OPT_STRING_LIST(0, "reference", &option_reference, N_("repo"),
@@ -95,10 +98,6 @@ static struct option builtin_clone_options[] = {
OPT_END()
};
-static const char *argv_submodule[] = {
- "submodule", "update", "--init", "--recursive", NULL
-};
-
static const char *get_repo_path_1(struct strbuf *path, int *is_bundle)
{
static char *suffix[] = { "/.git", "", ".git/.git", ".git" };
@@ -724,8 +723,16 @@ static int checkout(void)
err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
sha1_to_hex(sha1), "1", NULL);
- if (!err && option_recursive)
- err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+ if (!err && option_recursive) {
+ struct argv_array args = ARGV_ARRAY_INIT;
+ argv_array_pushl(&args, "submodule", "update", "--init", "--recursive", NULL);
+
+ if (max_jobs != -1)
+ argv_array_pushf(&args, "--jobs=%d", max_jobs);
+
+ err = run_command_v_opt(args.argv, RUN_GIT_CMD);
+ argv_array_clear(&args);
+ }
return err;
}
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 7fd5142..090891e 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -786,4 +786,19 @@ test_expect_success 'submodule update can be run in parallel' '
grep "9 tasks" trace.out
)
'
+
+test_expect_success 'git clone passes the parallel jobs config on to submodules' '
+ test_when_finished "rm -rf super4" &&
+ GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 7 . super4 &&
+ grep "7 tasks" trace.out &&
+ rm -rf super4 &&
+ git config --global submodule.fetchJobs 8 &&
+ GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules . super4 &&
+ grep "8 tasks" trace.out &&
+ rm -rf super4 &&
+ GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 9 . super4 &&
+ grep "9 tasks" trace.out &&
+ rm -rf super4
+'
+
test_done
--
2.7.0.rc0.42.ge5f5e2d
^ permalink raw reply related [flat|nested] 28+ messages in thread