From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.1 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_PASS,T_RP_MATCHES_RCVD shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 9C6541F407 for ; Mon, 11 Dec 2017 01:38:01 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 2D6AF120932; Mon, 11 Dec 2017 10:37:59 +0900 (JST) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by neon.ruby-lang.org (Postfix) with ESMTPS id 7353C120931 for ; Mon, 11 Dec 2017 10:37:50 +0900 (JST) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 5BB851F407; Mon, 11 Dec 2017 01:37:47 +0000 (UTC) Date: Mon, 11 Dec 2017 01:37:47 +0000 From: Eric Wong To: ruby-core@ruby-lang.org Message-ID: <20171211013747.GA27106@starla> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-ML-Name: ruby-core X-Mail-Count: 84153 Subject: [ruby-core:84153] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" samuel@oriontransfer.org wrote: > I think that the work being done here is great. However I feel that this PR requires far more scrutiny than it's receiving. Of course, which is why you don't see me pushing for it's inclusion in 2.5. I only present and update it so people can test it if they're bored. And I only started working on it because ko1 seemed interested in it at the time. I'd be surprised if this gets into 2.6 or any release in the future. Nobody besides you and I seems interested in discussing this anymore; so likely it'll sit here quietly for a few more years. Again, I don't make API decisions, I only present options. > > I am using "double" for timeout since it is more convenient for arithmetic like parts of thread.c. Most platforms have good FP, I think. > > e.g. https://github.com/socketry/nio4r/issues/140 Right. We already have plenty of threading internals using FP for timing, as well as the public Ruby APIs for IO.select and IO#wait_*able. Internally, at least it's a minor thing to change all the internal APIs to use "struct timespec" all around for maximum precision. > I think it's a great idea to have non-blocking evented IO. > However, it's not as simple as making read/write non-blocking. > How about DNS lookups? Filesystem access? The benefit of a > library based approach as I proposed is that these limitations > can be clearly part of the contract of a specific library, and > people can make different libraries to suit their needs, but > making it part of core Ruby is a slippery slope. If anything, > it would be better to depend on an established solution for > this, so that cases like using the system DNS resolver are > handled correctly (e.g. libuv). Otherwise, this is a HUGE > addition to the surface area of the ruby interpreter. We have resolv.rb in stdlib; which was at least popular in 1.8 days. It's implemented entirely in Ruby, so it automatically takes advantage of these Thriber changes, and has seen a fair amount of use back in the 1.8 days (not that DNS has changed drastically). So really, the network I/O part is not a big, or even complex change, it's 1.8 Threads being made an option again for Ruby 2.x. I miss 1.8 Threads, but I also like native threads in 1.9/2.x; they each have their place. And the lightweight threading for network I/O is what people seem to care about in other languages (Go, Erlang). nio4r/libuv and async can still be an option and I have no intention of breaking compatibility. Filesystem access: out-of-scope for this... I definitely do NOT want to try and make this use callbacks and threadpools behind users' backs, even internally. It pessimizes the common hot cache case (which doesn't require waiting); and more importantly, and I do not want Ruby or any library to interfere with mountpoint-aware code. Mountpoint awareness is 100% necessary for me so there's no queue blocking when one native thread is doing IO on a fast FS while another native thread is doing IO on a slow FS. I end up with dozens or hundreds of threads, because I have dozens or hundreds of mountpoints of different speeds. This is an uncommon use case, I know, but some people need it and the VM must not get in the way. So I think anything to deal with FS access specifically is out-of-scope for this issue. We already have native Thread support, and I use it to implement mountpoint-awareness. Some of the GVL-release changes to File and Dir for 2.5 will help with this (which reminds me, I still need to document some of that in NEWS :x).