From: Stefan Beller <sbeller@google.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: Jens Lehmann <Jens.Lehmann@web.de>,
Heiko Voigt <hvoigt@hvoigt.net>,
"git@vger.kernel.org" <git@vger.kernel.org>
Subject: Re: [PATCH] submodule: implement `module_name` as a builtin helper
Date: Fri, 7 Aug 2015 14:21:52 -0700 [thread overview]
Message-ID: <CAGZ79kYjaXtGurWgPk47FauLhC=k-gBjLYhepuz4gJE6Rm_8DA@mail.gmail.com> (raw)
In-Reply-To: <xmqqa8u2yf6e.fsf@gitster.dls.corp.google.com>
On Fri, Aug 7, 2015 at 2:14 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> On Fri, Aug 7, 2015 at 1:17 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>>
>>> This change...
>>>
>>>>> @@ -723,10 +733,8 @@ int fetch_populated_submodules(const struct argv_array *options,
>>>>> if (!S_ISGITLINK(ce->ce_mode))
>>>>> continue;
>>>>>
>>>>> - name = ce->name;
>>>>> - name_for_path = unsorted_string_list_lookup(&config_name_for_path, ce->name);
>>>>> - if (name_for_path)
>>>>> - name = name_for_path->util;
>>>>> + name_for_path = submodule_name_for_path(ce->name);
>>>>> + name = name_for_path ? name_for_path : ce->name;
>>>
>>> ... interacts with Heiko's cached submodule config work that seems
>>> to have stalled.
>>
>> We can drop that hunk as it only uses the new method
>> `submodule_name_for_path` but doesn't change functionality.
>> So if you want to keep Heikos work, I'll just resend the patch
>> without that hunk.
>
> Does such a result even make sense? Note that I wasn't talking
> about textual conflict.
>
> If we followed what you just said, that patch will try to directly
> read the data in config_name_for_path string list, which is removed
> by Heiko's series, if I am reading it right.
>
> In the new world order with Heiko's series, the way you grab
> submodule configuration data is to call submodule_from_path() or
> submodule_from_name() and they allow you to read from .gitmodules
> either in a commit (for historical state), the index, or from the
> working tree. Which should be much cleaner and goes in the right
> direction in the longer term.
>
> And the best part of the story is that your module_name would be
> just calling submodule_from_path() and peeking into a field.
>
>> 2) Come up with a good thread pool abstraction
>> (Started as "[RFC/PATCH 0/4] parallel fetch for submodules" )
>> This abstraction (if done right) will allow us to use it in different places
>> easily. I started it as part of "git fetch --recurse-submodules" because
>> it is submodule related and reasonably sized
>
> I personally think this gives the most bang-for-buck. Write that
> and expose it as "git submodule for-each-parallel", which takes the
> shell scriptlet that currently is the loop body of "while read mode
> sha1 stage sm_path" in update and clone. You will have immediate
> and large payback.
You said that before. I feel like this is a bit to narrow. A "git submodule
for-each-parallel" would be a very specific tool which we would use to
make the different submodule operations parallel with ease. But it would
be very submodule specifc I guess?
That's why I want to be a bit more generic and have this thread pool API
done in C, such that "any for loop" in git can be easily replaced by using
the thread pool. I think of "git fetch --all" specially.
>
> And you do not even need module_list and module_name written in C in
> order to do so. Not that these two are not trivial to implement, but
> the payoff from them is not as large as from for-each-parallel ;-)
>
I think I can do this for-each-parallel once I have the more generic thread
pooling done. All that is left now is a good handling of stdout/stderr, which
I am not yet convinced how to do it right. Maybe each task accumulates
messages in two string buffers and then the thread pool will output the
string buffer one a task is done.
next prev parent reply other threads:[~2015-08-07 21:22 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-08-05 0:04 [PATCH 1/4] submodule: implement `module_list` as a builtin helper Stefan Beller
2015-08-05 0:04 ` [PATCH 2/4] submodule: implement `module_name` " Stefan Beller
2015-08-05 0:05 ` Stefan Beller
2015-08-05 0:58 ` Eric Sunshine
2015-08-05 16:29 ` Stefan Beller
2015-08-05 19:06 ` Jens Lehmann
2015-08-05 19:55 ` Stefan Beller
2015-08-05 21:08 ` [PATCH] " Stefan Beller
2015-08-06 19:49 ` Jens Lehmann
2015-08-06 21:38 ` Stefan Beller
2015-08-07 20:03 ` Junio C Hamano
2015-08-07 20:54 ` Stefan Beller
2015-08-07 20:17 ` Junio C Hamano
2015-08-07 20:49 ` Stefan Beller
2015-08-07 21:14 ` Junio C Hamano
2015-08-07 21:21 ` Stefan Beller [this message]
2015-08-07 21:32 ` Junio C Hamano
2015-08-07 22:04 ` Stefan Beller
2015-08-07 22:18 ` Junio C Hamano
2015-08-07 23:12 ` Stefan Beller
2015-08-07 22:42 ` Junio C Hamano
2015-08-07 23:19 ` Stefan Beller
2015-08-08 0:21 ` Junio C Hamano
2015-08-08 6:20 ` Junio C Hamano
2015-08-10 17:03 ` Stefan Beller
2015-08-05 18:31 ` [PATCH 1/4] submodule: implement `module_list` " Jens Lehmann
2015-08-07 19:53 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGZ79kYjaXtGurWgPk47FauLhC=k-gBjLYhepuz4gJE6Rm_8DA@mail.gmail.com' \
--to=sbeller@google.com \
--cc=Jens.Lehmann@web.de \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=hvoigt@hvoigt.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).