From: Junio C Hamano <gitster@pobox.com>
To: Stefan Beller <sbeller@google.com>
Cc: git@vger.kernel.org, Jens Lehmann <Jens.Lehmann@web.de>,
Heiko Voigt <hvoigt@hvoigt.net>
Subject: Re: [PATCH] add: remove dead code
Date: Fri, 31 Jul 2015 09:41:37 -0700 [thread overview]
Message-ID: <xmqqr3no5lfi.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <1438301956-1658-1-git-send-email-sbeller@google.com> (Stefan Beller's message of "Thu, 30 Jul 2015 17:19:16 -0700")
Stefan Beller <sbeller@google.com> writes:
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>
> So I was trying to understand how to implement "git add .gitmodules" as
> I intend to rewrite git submodules in C.
A request and a suggestion.
- Please keep Jens and Heiko in the loop and pick their brains ;-)
- It may not be the best use of your time to attempt rewriting the
whole of "git submodule" in C in a single pass (which is the
impression I am getting from seeing you say "git add
.gitmodules").
The largest pain point a rewrite in C would solve is that "git
submodule init" and "update" have to go sequencially even though
(1) there is no inherent reason for cloning and fetching of
different submodules to happen in some order;
(2) there is no inherent reason for a failure to clone or fetch
of one submodule to abort the entire session without cloning
or fetching other submodules; and
(3) the operation in each submodule takes human-scale time and
the users would benefit greatly by parallel operations.
One approach that may be beneficial would be to introduce "git
submodule--helper" written in C to have selected primitives used
in "git submodule" script to speed them up. Perhaps the first
subcommand would be "
$ git submodule--helper foreach-parallel --cmd=$cmd $args...
where it takes the arguments currently fed to module_list as
$args... and runs $cmd in parallel. The initial implementation
of "git submodule update" then would replace the expensive and
sequencial
module_list "$@" | {
...
while read mode sha1 stage sm_path
do
...
done
}
loop with a call to the foreach-parallel subcommand.
The end-user scripts that currently use "git submodule foreach"
may or may not depend on the sequencial nature of the current
implementation, so adding "git submodule foreach-parallel" may be
a good way to expose this as a new "do it in parallel" feature.
Once you have a solid infrastructure to implement the helper
subcommand "foreach-parallel" (I'd expect the interface inside C
into that function would be to give it a worker function with the
data for the function to consume, and the above --cmd=$cmd form
would use a worker function that essentially does run_command();
the spawn(2)ing and wait(2)ing would be done on the more generic
API side), you can rewrite the $cmd part in C little by little,
and eventually you would get a full C implemention while keeping
everything working and retaining debuggability during the course
of development.
Thanks.
next prev parent reply other threads:[~2015-07-31 16:41 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-31 0:19 [PATCH] add: remove dead code Stefan Beller
2015-07-31 16:41 ` Junio C Hamano [this message]
2015-07-31 23:09 ` [RFC/PATCH 0/2] Submodules: refactoring the `module_list` function Stefan Beller
2015-07-31 23:09 ` [RFC/PATCH 1/2] submodule: implement `module_list` as a builtin helper Stefan Beller
2015-08-01 1:01 ` Junio C Hamano
2015-08-03 21:30 ` Stefan Beller
2015-08-03 21:38 ` Junio C Hamano
2015-08-03 21:58 ` [PATCH] " Stefan Beller
2015-08-03 22:04 ` [RFC/PATCH 1/2] " Junio C Hamano
2015-08-03 22:13 ` Stefan Beller
2015-08-03 22:58 ` [PATCH] " Stefan Beller
2015-07-31 23:09 ` [RFC/PATCH 2/2] Testing the new code Stefan Beller
2015-08-01 1:02 ` Junio C Hamano
2015-08-03 16:23 ` Stefan Beller
2015-08-03 19:47 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqr3no5lfi.fsf@gitster.dls.corp.google.com \
--to=gitster@pobox.com \
--cc=Jens.Lehmann@web.de \
--cc=git@vger.kernel.org \
--cc=hvoigt@hvoigt.net \
--cc=sbeller@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).