From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-2.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_PASS,T_RP_MATCHES_RCVD shortcircuit=no autolearn=no autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 9A3A51F576 for ; Sun, 28 Jan 2018 10:42:02 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 1CFA71209E1; Sun, 28 Jan 2018 19:42:01 +0900 (JST) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by neon.ruby-lang.org (Postfix) with ESMTPS id 665071209DD for ; Sun, 28 Jan 2018 19:41:51 +0900 (JST) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 5F5171FAE2; Sun, 28 Jan 2018 10:41:49 +0000 (UTC) Date: Sun, 28 Jan 2018 10:41:49 +0000 From: Eric Wong To: ruby-core@ruby-lang.org Message-ID: <20180128104149.GA9246@dcvr> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-ML-Name: ruby-core X-Mail-Count: 85170 Subject: [ruby-core:85170] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" sam.saffron@gmail.com wrote: > Hmmm, what about just bringing in the IO Manager APIs > including Ruby helpers prior to re-introducing the green > threads? One big problem I notice with existing IO manager APIs (libev/libevent/EventMachine) is multi-threading was as an afterthought to them. As in, throw a lock around a single-threaded event loop and call it a day. Ruby was this way, too; but want to work towards changing that and embracing the multi-thread friendliness baked into APIs provided by kqueue and epoll. Btw, some of the discussion/planning around this started in: https://public-inbox.org/ruby-core/20170402023514.GB30476@dcvr/t/ > As it stands kqueue/epoll abstractions always require another > fat dependency and there is no official API to consume them. I don't know if exposing a new API around them is desirable. For human-friendliness, it seems desirable to keep the Ruby API synchronous even if internal bits become async. I think it's also desirable to be able to change some/most existing Thread uses to auto-Fiber/Threadlet/Thriber without having to re-design things, just changing "Thread.new" to something else. > Even just solving this problem is enough of a hornets nest > prior to introduction of other complications. > > epoll is notoriously monstrous, > http://cvs.schmorp.de/libev/ev_epoll.c?view=markup so having > an officially supported abstraction would be a great start. I disagree. IMHO, Lehman's notes and complaints against epoll are either out-of-date or his mental model went wrong somewhere. Fwiw, fs/eventpoll.c is straightforward and easy-to-understand in git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > Wouldn't having these abstractions allow building this by hand > using existing Fiber? One question is, how painful will it be in Ruby? I've kinda soured on _nonblock APIs in Ruby over the years. For example, in https://bugs.ruby-lang.org/issues/14404 I don't think there's a non-painful Ruby way to resume a partial writev. Doable, of course, but it requires extra allocations and copies. Resuming a partial write_nonblock today without writev isn't great, either... With a synchronous interface (IO#write), dealing with partial writev in C is only a few adds/subracts; and we wont expose pointer arithmetic in Ruby :) And then there's also stuff like IO.copy_stream not having a _nonblock analogue...