From: Stefan Beller <sbeller@google.com>
To: git@vger.kernel.org
Cc: jacob.keller@gmail.com, peff@peff.net, gitster@pobox.com,
jrnieder@gmail.com, johannes.schindelin@gmail.com,
Jens.Lehmann@web.de, ericsunshine@gmail.com,
Stefan Beller <sbeller@google.com>
Subject: [PATCH 0/9] Expose the submodule parallelism to the user
Date: Tue, 27 Oct 2015 11:15:44 -0700 [thread overview]
Message-ID: <1445969753-418-1-git-send-email-sbeller@google.com> (raw)
Where does it apply?
---
This applies on 376d400f4c (run-command: fix missing output from late callbacks,
which is the latest commit in origin/sb/submodule-parallel-fetch which was
merged to origin/next)
The first patch is a duplicate of origin/sb/submodule-config-parse, so
it may make sense to drop the first patch and apply this series on top of a
merge of 376d400f4c and origin/sb/submodule-config-parse.
I realize sending refactorings in the area you'd be likely to touch as
a separate patch (series) is not necessarily a good idea as it leads to
situations like this.
What does it do?
---
This series should finish the on going efforts of parallelizing
submodule network traffic. The patches contain tests for clone,
fetch and submodule update to use the actual parallelism both via
command line as well as a configured option. I decided to go with
"submodule.jobs" for all three for now.
Detailed breakdown of the patches
---
Patch 1 is a duplicate of origin/sb/submodule-config-parse and may make
merging with that easier.
Patch 2 adds the update strategy to the struct submodule, which is required in
patch 4.
Patch 3 adds rudimentary tracing output to the parallel processing commands.
Patch 4 rewrites parts of "git submodule update" in C, such that the cloning
is done from within the parallel processing engine.
Patch 5 however exposes the possible parallelism of patch 4 to the user.
(doc + tests)
Patch 6 adds the parallel feature to clone, which just invokes "submodule update"
internally.
Patch 7 is a small refactoring preparing patch 8 to smoothly parse submodules.jobs.
Patch 9 teaches fetch to respect the desired parallelism both from command line
as well as the config option.
Thanks,
Stefan
Stefan Beller (9):
submodule-config: "goto" removal in parse_config()
submodule config: keep update strategy around
run_processes_parallel: Add output to tracing messages
git submodule update: have a dedicated helper for cloning
submodule update: expose parallelism to the user
clone: allow an explicit argument for parallel submodule clones
submodule config: remove name_and_item_from_var
submodule-config: parse_config
fetching submodules: Respect `submodule.jobs` config option
Documentation/config.txt | 7 ++
Documentation/git-clone.txt | 5 +-
Documentation/git-submodule.txt | 6 +-
builtin/clone.c | 26 ++++-
builtin/fetch.c | 2 +-
builtin/submodule--helper.c | 243 ++++++++++++++++++++++++++++++++++++++++
git-submodule.sh | 54 ++++-----
run-command.c | 4 +
submodule-config.c | 166 ++++++++++++++-------------
submodule-config.h | 3 +
submodule.c | 5 +
t/t5526-fetch-submodules.sh | 14 +++
t/t7400-submodule-basic.sh | 4 +-
t/t7406-submodule-update.sh | 27 +++++
14 files changed, 444 insertions(+), 122 deletions(-)
--
2.5.0.283.g1a79c94.dirty
next reply other threads:[~2015-10-27 18:17 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-27 18:15 Stefan Beller [this message]
2015-10-27 18:15 ` [PATCH 1/9] submodule-config: "goto" removal in parse_config() Stefan Beller
2015-10-27 21:26 ` Jonathan Nieder
2015-10-27 21:39 ` Junio C Hamano
2015-10-27 18:15 ` [PATCH 2/9] submodule config: keep update strategy around Stefan Beller
2015-10-27 18:15 ` [PATCH 3/9] run_processes_parallel: Add output to tracing messages Stefan Beller
2015-10-27 18:15 ` [PATCH 4/9] git submodule update: have a dedicated helper for cloning Stefan Beller
2015-10-27 18:15 ` [PATCH 5/9] submodule update: expose parallelism to the user Stefan Beller
2015-10-27 20:59 ` Junio C Hamano
2015-10-28 21:40 ` Stefan Beller
2015-10-28 22:20 ` Junio C Hamano
2015-10-27 18:15 ` [PATCH 6/9] clone: allow an explicit argument for parallel submodule clones Stefan Beller
2015-10-27 20:57 ` Junio C Hamano
2015-10-28 20:50 ` Stefan Beller
2015-10-27 18:15 ` [PATCH 7/9] submodule config: remove name_and_item_from_var Stefan Beller
2015-10-27 18:15 ` [PATCH 8/9] submodule-config: parse_config Stefan Beller
2015-10-27 18:15 ` [PATCH 9/9] fetching submodules: Respect `submodule.jobs` config option Stefan Beller
2015-10-27 21:00 ` Junio C Hamano
2015-10-27 19:12 ` [PATCH 0/9] Expose the submodule parallelism to the user Junio C Hamano
2015-10-28 23:21 ` [PATCHv2 0/8] " Stefan Beller
2015-10-28 23:21 ` [PATCHv2 1/8] run_processes_parallel: Add output to tracing messages Stefan Beller
2015-10-30 1:10 ` Eric Sunshine
2015-10-30 17:32 ` Stefan Beller
2015-10-28 23:21 ` [PATCHv2 2/8] submodule config: keep update strategy around Stefan Beller
2015-10-30 1:14 ` Eric Sunshine
2015-10-30 17:38 ` Stefan Beller
2015-10-30 18:16 ` Eric Sunshine
2015-10-30 18:25 ` Stefan Beller
2015-10-28 23:21 ` [PATCHv2 3/8] submodule config: remove name_and_item_from_var Stefan Beller
2015-10-30 1:23 ` Eric Sunshine
2015-10-30 18:37 ` Stefan Beller
2015-10-28 23:21 ` [PATCHv2 4/8] submodule-config: parse_config Stefan Beller
2015-10-30 1:53 ` Eric Sunshine
2015-10-30 19:29 ` Stefan Beller
2015-10-28 23:21 ` [PATCHv2 5/8] fetching submodules: Respect `submodule.jobs` config option Stefan Beller
2015-10-30 2:17 ` Eric Sunshine
2015-10-28 23:21 ` [PATCHv2 6/8] git submodule update: have a dedicated helper for cloning Stefan Beller
2015-10-29 22:34 ` Junio C Hamano
2015-10-28 23:21 ` [PATCHv2 7/8] submodule update: expose parallelism to the user Stefan Beller
2015-10-28 23:21 ` [PATCHv2 8/8] clone: allow an explicit argument for parallel submodule clones Stefan Beller
2015-11-01 8:58 ` Eric Sunshine
2015-10-29 13:19 ` [PATCHv2 0/8] Expose the submodule parallelism to the user Ramsay Jones
2015-10-29 15:51 ` Stefan Beller
2015-10-29 17:23 ` Junio C Hamano
2015-10-29 17:30 ` Stefan Beller
2015-10-29 23:50 ` Ramsay Jones
2015-11-03 19:41 ` Stefan Beller
2015-10-29 20:12 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1445969753-418-1-git-send-email-sbeller@google.com \
--to=sbeller@google.com \
--cc=Jens.Lehmann@web.de \
--cc=ericsunshine@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jacob.keller@gmail.com \
--cc=johannes.schindelin@gmail.com \
--cc=jrnieder@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).