git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Stefan Beller <sbeller@google.com>
To: git@vger.kernel.org
Cc: jacob.keller@gmail.com, peff@peff.net, gitster@pobox.com,
	jrnieder@gmail.com, johannes.schindelin@gmail.com,
	Jens.Lehmann@web.de, ericsunshine@gmail.com,
	Stefan Beller <sbeller@google.com>
Subject: [PATCHv2 8/8] clone: allow an explicit argument for parallel submodule clones
Date: Wed, 28 Oct 2015 16:21:44 -0700	[thread overview]
Message-ID: <1446074504-6014-9-git-send-email-sbeller@google.com> (raw)
In-Reply-To: <1446074504-6014-1-git-send-email-sbeller@google.com>

Just pass it along to "git submodule update", which may pick reasonable
defaults if you don't specify an explicit number.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/git-clone.txt |  6 +++++-
 builtin/clone.c             | 23 +++++++++++++++++------
 t/t7406-submodule-update.sh | 15 +++++++++++++++
 3 files changed, 37 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index f1f2a3f..01bd6b7 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,7 +14,7 @@ SYNOPSIS
 	  [-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]
 	  [--dissociate] [--separate-git-dir <git dir>]
 	  [--depth <depth>] [--[no-]single-branch]
-	  [--recursive | --recurse-submodules] [--] <repository>
+	  [--recursive | --recurse-submodules] [--jobs <n>] [--] <repository>
 	  [<directory>]
 
 DESCRIPTION
@@ -216,6 +216,10 @@ objects from the source repository into a pack in the cloned repository.
 	The result is Git repository can be separated from working
 	tree.
 
+-j <n>::
+--jobs <n>::
+	The number of submodules fetched at the same time.
+	Defaults to the `submodule.jobs` option.
 
 <repository>::
 	The (possibly remote) repository to clone from.  See the
diff --git a/builtin/clone.c b/builtin/clone.c
index 9eaecd9..22b9924 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -50,6 +50,7 @@ static int option_progress = -1;
 static struct string_list option_config;
 static struct string_list option_reference;
 static int option_dissociate;
+static int max_jobs = -1;
 
 static struct option builtin_clone_options[] = {
 	OPT__VERBOSITY(&option_verbosity),
@@ -72,6 +73,8 @@ static struct option builtin_clone_options[] = {
 		    N_("initialize submodules in the clone")),
 	OPT_BOOL(0, "recurse-submodules", &option_recursive,
 		    N_("initialize submodules in the clone")),
+	OPT_INTEGER('j', "jobs", &max_jobs,
+		    N_("number of submodules cloned in parallel")),
 	OPT_STRING(0, "template", &option_template, N_("template-directory"),
 		   N_("directory from which templates will be used")),
 	OPT_STRING_LIST(0, "reference", &option_reference, N_("repo"),
@@ -95,10 +98,6 @@ static struct option builtin_clone_options[] = {
 	OPT_END()
 };
 
-static const char *argv_submodule[] = {
-	"submodule", "update", "--init", "--recursive", NULL
-};
-
 static const char *get_repo_path_1(struct strbuf *path, int *is_bundle)
 {
 	static char *suffix[] = { "/.git", "", ".git/.git", ".git" };
@@ -724,8 +723,20 @@ static int checkout(void)
 	err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
 			   sha1_to_hex(sha1), "1", NULL);
 
-	if (!err && option_recursive)
-		err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+	if (!err && option_recursive) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		argv_array_pushl(&args, "submodule", "update", "--init", "--recursive", NULL);
+
+		if (max_jobs != -1) {
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addf(&sb, "--jobs=%d", max_jobs);
+			argv_array_push(&args, sb.buf);
+			strbuf_release(&sb);
+		}
+
+		err = run_command_v_opt(args.argv, RUN_GIT_CMD);
+		argv_array_clear(&args);
+	}
 
 	return err;
 }
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 05ea66f..ade0524 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -786,4 +786,19 @@ test_expect_success 'submodule update can be run in parallel' '
 	 grep "9 children" trace.out
 	)
 '
+
+test_expect_success 'git clone passes the parallel jobs config on to submodules' '
+	test_when_finished "rm -rf super4" &&
+	GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 7 . super4 &&
+	grep "7 children" trace.out &&
+	rm -rf super4 &&
+	git config --global submodule.jobs 8 &&
+	GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules . super4 &&
+	grep "8 children" trace.out &&
+	rm -rf super4 &&
+	GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 9 . super4 &&
+	grep "9 children" trace.out &&
+	rm -rf super4
+'
+
 test_done
-- 
2.5.0.281.g4ed9cdb

  parent reply	other threads:[~2015-10-28 23:22 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-10-27 18:15 [PATCH 0/9] Expose the submodule parallelism to the user Stefan Beller
2015-10-27 18:15 ` [PATCH 1/9] submodule-config: "goto" removal in parse_config() Stefan Beller
2015-10-27 21:26   ` Jonathan Nieder
2015-10-27 21:39     ` Junio C Hamano
2015-10-27 18:15 ` [PATCH 2/9] submodule config: keep update strategy around Stefan Beller
2015-10-27 18:15 ` [PATCH 3/9] run_processes_parallel: Add output to tracing messages Stefan Beller
2015-10-27 18:15 ` [PATCH 4/9] git submodule update: have a dedicated helper for cloning Stefan Beller
2015-10-27 18:15 ` [PATCH 5/9] submodule update: expose parallelism to the user Stefan Beller
2015-10-27 20:59   ` Junio C Hamano
2015-10-28 21:40     ` Stefan Beller
2015-10-28 22:20       ` Junio C Hamano
2015-10-27 18:15 ` [PATCH 6/9] clone: allow an explicit argument for parallel submodule clones Stefan Beller
2015-10-27 20:57   ` Junio C Hamano
2015-10-28 20:50     ` Stefan Beller
2015-10-27 18:15 ` [PATCH 7/9] submodule config: remove name_and_item_from_var Stefan Beller
2015-10-27 18:15 ` [PATCH 8/9] submodule-config: parse_config Stefan Beller
2015-10-27 18:15 ` [PATCH 9/9] fetching submodules: Respect `submodule.jobs` config option Stefan Beller
2015-10-27 21:00   ` Junio C Hamano
2015-10-27 19:12 ` [PATCH 0/9] Expose the submodule parallelism to the user Junio C Hamano
2015-10-28 23:21   ` [PATCHv2 0/8] " Stefan Beller
2015-10-28 23:21     ` [PATCHv2 1/8] run_processes_parallel: Add output to tracing messages Stefan Beller
2015-10-30  1:10       ` Eric Sunshine
2015-10-30 17:32         ` Stefan Beller
2015-10-28 23:21     ` [PATCHv2 2/8] submodule config: keep update strategy around Stefan Beller
2015-10-30  1:14       ` Eric Sunshine
2015-10-30 17:38         ` Stefan Beller
2015-10-30 18:16           ` Eric Sunshine
2015-10-30 18:25             ` Stefan Beller
2015-10-28 23:21     ` [PATCHv2 3/8] submodule config: remove name_and_item_from_var Stefan Beller
2015-10-30  1:23       ` Eric Sunshine
2015-10-30 18:37         ` Stefan Beller
2015-10-28 23:21     ` [PATCHv2 4/8] submodule-config: parse_config Stefan Beller
2015-10-30  1:53       ` Eric Sunshine
2015-10-30 19:29         ` Stefan Beller
2015-10-28 23:21     ` [PATCHv2 5/8] fetching submodules: Respect `submodule.jobs` config option Stefan Beller
2015-10-30  2:17       ` Eric Sunshine
2015-10-28 23:21     ` [PATCHv2 6/8] git submodule update: have a dedicated helper for cloning Stefan Beller
2015-10-29 22:34       ` Junio C Hamano
2015-10-28 23:21     ` [PATCHv2 7/8] submodule update: expose parallelism to the user Stefan Beller
2015-10-28 23:21     ` Stefan Beller [this message]
2015-11-01  8:58       ` [PATCHv2 8/8] clone: allow an explicit argument for parallel submodule clones Eric Sunshine
2015-10-29 13:19     ` [PATCHv2 0/8] Expose the submodule parallelism to the user Ramsay Jones
2015-10-29 15:51       ` Stefan Beller
2015-10-29 17:23         ` Junio C Hamano
2015-10-29 17:30           ` Stefan Beller
2015-10-29 23:50         ` Ramsay Jones
2015-11-03 19:41           ` Stefan Beller
2015-10-29 20:12     ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1446074504-6014-9-git-send-email-sbeller@google.com \
    --to=sbeller@google.com \
    --cc=Jens.Lehmann@web.de \
    --cc=ericsunshine@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jacob.keller@gmail.com \
    --cc=johannes.schindelin@gmail.com \
    --cc=jrnieder@gmail.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).