From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.5 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id F357A200B9 for ; Thu, 3 May 2018 01:15:59 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 0D0AF120A32; Thu, 3 May 2018 10:15:58 +0900 (JST) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by neon.ruby-lang.org (Postfix) with ESMTPS id 6113E1209F2 for ; Thu, 3 May 2018 10:15:50 +0900 (JST) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 75A56200B9; Thu, 3 May 2018 01:15:48 +0000 (UTC) Date: Thu, 3 May 2018 01:15:47 +0000 From: Eric Wong To: ruby-core@ruby-lang.org Message-ID: <20180503011547.GA1226@ailurophile> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-ML-Name: ruby-core X-Mail-Count: 86853 Subject: [ruby-core:86853] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" samuel@oriontransfer.org wrote: > Eric Wong wrote: > > yahns is designed to deal with apps with both slow and fast > > endpoints simultaneously. Given N threads running, (N-1) may be > > stuck servicing slow endpoints, while the Nth one remains free > > to service ANY other client. > I'm surprised by this, how would slow clients be a problem? I did not mention "slow clients", I said "slow endpoints". Some endpoints make expensive DB queries and so take longer. Endpoint speed and client speed are totally different. > You can work around TTY problems using something like > http://code.saghul.net/index.php/2016/05/24/libuv-internals-the-osx-select2-trick/ Yes, that's what I had in mind (along with various char devices, fifos and all the other oddball corner cases). Maybe an extra thread won't be necessary, even, if one can select/poll directly on the kqueue FD. > > Really, I think it's a waste of time and resources to support > > these things. > > If you think this, why add this proposed feature to Ruby at all. It's to fix the lack of lightweight concurrency we lost with 1.9 I wish I didn't have to, but I'd rather keep using Ruby and not depend on a truckload of out-of-tree dependencies to fix a regression. Leftpad, anyone? > Thread.new do > Thread.current.selector = NIO::Selector.new > # selector responds to :wait_for_single_fd > > # Thread at exit calls selector.run I don't expect a user to have to care or know what a "selector" is. I try to avoid tuning knobs because they cause cognitive overhead for users (and they become out-of-date and wrong). I'd rather try to make things work well out-of-the-box. I think having a choice of selectors will make the problem worse because we'd have more variations to deal with when users report bugs. > As has been demonstrated, there are lots of trade-offs. > Personally, I'd rather use libev or libuv which is actively > maintained than what you've proposed here. They cover a much > larger chunk of functionality, and they are maintained and > updated independently of Ruby. Why repeat all that work? Are > you going to maintain this feature for the next 20 years? Many reasons: - core features of Ruby shouldn't depend too much on 3rd party code. I recall matz has a similar sentiment (so no LLVM, either) - I consider lightweight concurrency a core feature of a language. Languages like Go and Erlang have won over Ruby users with good lightweight concurrency out-of-the-box. - libev and libevent consider multithreading an afterthought (not sure about libuv, here) - GVL or not, Ruby needs to continue to work well with many native threads, so having a 1:1 nativethread:reactor relationship is unreasonably expensive. - libev ev(3) manpage has a misinformed rant on epoll. That tells me more about how bad libev is at using epoll than epoll itself. - contributing to libuv requires GitHub, a proprietary communications platform that's off limits to me - the rest of the Ruby core team can help maintain this. I don't expect to live 20 years, but I do my best to document the internals while I'm around. - some of us can fix bugs and improve the Linux kernel for this. Maybe we'll get some *BSD kernel hackers on board, too.