* git submodule, multi-threading the update process for large number of registered submodules
@ 2014-09-02 7:53 Jose Paredes
2014-09-02 19:30 ` Jens Lehmann
0 siblings, 1 reply; 2+ messages in thread
From: Jose Paredes @ 2014-09-02 7:53 UTC (permalink / raw
To: git
Dear Git community
Is there any way to run "git submodule update" in multiple threads?
The use case:
-------------------
"git submodule update" seems to be inefficient when running
sequentially on a large .gitmodules file.
Assuming a git forest with over 7K gits it takes hours to complete the
update (running on Windows+Cygwin)
If not supported, this feature could be a good candidate for "git
submodule" enhancement.
What is your opinion or advice?
Thanks and regards
José
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: git submodule, multi-threading the update process for large number of registered submodules
2014-09-02 7:53 git submodule, multi-threading the update process for large number of registered submodules Jose Paredes
@ 2014-09-02 19:30 ` Jens Lehmann
0 siblings, 0 replies; 2+ messages in thread
From: Jens Lehmann @ 2014-09-02 19:30 UTC (permalink / raw
To: Jose Paredes, git; +Cc: Heiko Voigt
Hi José,
Am 02.09.2014 um 09:53 schrieb Jose Paredes:
> Is there any way to run "git submodule update" in multiple threads?
>
> The use case:
> -------------------
> "git submodule update" seems to be inefficient when running
> sequentially on a large .gitmodules file.
> Assuming a git forest with over 7K gits it takes hours to complete the
> update (running on Windows+Cygwin)
Wow, over 7000 submodules under Windows? I can imagine that a
submodule update takes some time under these circumstances.
> If not supported, this feature could be a good candidate for "git
> submodule" enhancement.
> What is your opinion or advice?
Hmm, are we really sure parallel execution would be faster? I can
imagine that "git fetch" could profit a lot from fetching more than
one submodule at a time (assuming you are not fetching from the same
upstream server), but I'm not so sure about submodule update. And
things might also be rather different e.g. on Windows and Linux.
Could you do some benchmarks comparing "git submodule update" with a
script that does something like:
(cd <submodule>; git checkout <commit recorded in superproject>)&
for each changed submodule to see if we could gain anything from
executing the update in parallel? (You should be able to extract
which submodules must be checked out to what commit by doing e.g.
"git diff --raw <start> <target> | grep ^:160000" and processing
the output a bit further).
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2014-09-02 19:30 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-02 7:53 git submodule, multi-threading the update process for large number of registered submodules Jose Paredes
2014-09-02 19:30 ` Jens Lehmann
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).