From: Stefan Beller <sbeller@google.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: Johannes Sixt <j6t@kdbg.org>,
"git@vger.kernel.org" <git@vger.kernel.org>,
Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: Re: git.git as of tonight
Date: Tue, 3 Nov 2015 10:18:40 -0800 [thread overview]
Message-ID: <CAGZ79kbAj-tZS3X1EwFw_gGh1=YU8M0OmEX91W8f++fqs6TiBA@mail.gmail.com> (raw)
In-Reply-To: <xmqq37wndndp.fsf@gitster.mtv.corp.google.com>
On Tue, Nov 3, 2015 at 9:05 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Johannes Sixt <j6t@kdbg.org> writes:
>
>> My findings so far are negative. The only short-term and mid-term
>> solution I see so far is to opt-out from the framework during
>> build-time.
So I started reading up on that[1].
As far as I understand, we don't need to mark a file descriptor
to be non blocking, but rather we could use ReadFileEx[2] with
a flag set for "overlapped" operation.
So that said, we can make set_nonblocking a noop and
provide another implementation for strbuf_read_once
depending on NO_PTHREADS being set.
Maybe not even strbuf_read_once, but rather the underlying
xread_nonblock ?
[1] http://tinyclouds.org/iocp-links.html
[2] https://msdn.microsoft.com/en-us/library/aa365468(v=VS.85).aspx
>
> Now, from where I sit, it seems that the way forward would be
>
> 1. Make this an optional feature so that platforms can compile it
> out, if it is not already done. My preference, even if we go
> that route, would be to see if we can find a way to preserve the
> overall code structure (e.g. instead of spawning multiple
> workers, which is why the code needs NONBLOCK to avoid getting
> stuck on reading from one while others are working, perhaps we
> can spawn only one and not do a nonblock read?).
Yeah that would be my understanding as well. If we don't come up with
a good solution for parallelism in Windows now, we'd need to make it at
least working in the jobs=1 case as well as it worked before.
>
> 2. After that is done, the feature could graduate to 'master'. As
> this is a bigger framework change than others, however, we do
> not necessarily want to rush it. On the other hand, because
> this only affects submodules, which means it has fewer users and
> testers that would give us feedback while it is on 'next', we
> may want to push it to 'master' sooner to give it a larger
> exposure. I dunno, and I do not want to decide this myself the
> week before I'll go offline for a few weeks (i.e. today).
Yeah I guess cooking this well done has its benefits.
>
> 3. Then we would enlist help from folks who are more familiar with
> Windows platform (like you) to see how the "run parallel workers
> and collect from them" can be (re)done with a nice level of
> abstraction. I am hoping that we can continue the tradition of
> the evolution of run-command.c API (I am specifically impressed
> by what you did for "async" that allows the callers not to worry
> about threads and processes) aroundt this area. That is
> obviously a mid- to longer term goal.
I just wonder if we can skip step 1) and 2) by having the discussion
now how to change the framework to work well without posix file
descriptors here.
>
> Thanks for working together well, you two.
next prev parent reply other threads:[~2015-11-03 18:18 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-02 2:58 git.git as of tonight Junio C Hamano
2015-11-02 21:15 ` Johannes Sixt
2015-11-02 22:16 ` Junio C Hamano
2015-11-02 23:06 ` Stefan Beller
2015-11-03 6:34 ` Johannes Sixt
2015-11-03 17:05 ` Junio C Hamano
2015-11-03 18:18 ` Stefan Beller [this message]
2015-11-03 21:03 ` Johannes Sixt
2015-11-03 23:00 ` Stefan Beller
2015-11-04 19:59 ` O_NONBLOCK under Windows (was: git.git as of tonight) Torsten Bögershausen
2015-11-04 20:07 ` Stefan Beller
2015-11-04 22:43 ` [PATCH 0/2] Missing " Stefan Beller
2015-11-04 22:43 ` [PATCH 1/2] run-parallel: rename set_nonblocking to set_nonblocking_or_die Stefan Beller
2015-11-05 6:07 ` Torsten Bögershausen
2015-11-05 6:14 ` Junio C Hamano
2015-11-05 6:19 ` Junio C Hamano
2015-11-05 6:58 ` Jeff King
2015-11-04 22:43 ` [PATCH 2/2] run-parallel: Run sequential if nonblocking I/O is unavailable Stefan Beller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGZ79kbAj-tZS3X1EwFw_gGh1=YU8M0OmEX91W8f++fqs6TiBA@mail.gmail.com' \
--to=sbeller@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=j6t@kdbg.org \
--cc=johannes.schindelin@gmx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).