From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 9CCA421841 for ; Wed, 2 May 2018 08:38:25 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id AD414120A0E; Wed, 2 May 2018 17:38:22 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 887E31209FA for ; Wed, 2 May 2018 17:38:19 +0900 (JST) Received: by filter0012p3iad2.sendgrid.net with SMTP id filter0012p3iad2-19815-5AE978F7-A6 2018-05-02 08:38:15.945082227 +0000 UTC Received: from herokuapp.com (ec2-54-91-163-8.compute-1.amazonaws.com [54.91.163.8]) by ismtpd0023p1mdw1.sendgrid.net (SG) with ESMTP id zJdpiSSoT3iPbqLqbm_5_w Wed, 02 May 2018 08:38:15.800 +0000 (UTC) Date: Wed, 02 May 2018 08:38:16 +0000 (UTC) From: samuel@oriontransfer.org To: ruby-core@ruby-lang.org Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 62184 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 13618 X-Redmine-Issue-Author: normalperson X-Redmine-Issue-Assignee: normalperson X-Redmine-Sender: ioquatix X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: ync6xU2WACa70kv/Ymy4QrNMhiuLXJG8OTL2vJD1yS7n3zVomL6/XbZZ6A8R+39WppvpSFHjzlnl0/ ip2AoqEsOIbPkllUw4eWJLosR8CKz8fYO6fm971L73oXiOfnmM6JxloXTE96UaOtt/bMBit0C3fIwo GQ3xN8wxpfQhLDtPjUz05A9Lnh4++0cG6vjwyKvSt0vTlCvpAuWV2bauPg== X-ML-Name: ruby-core X-Mail-Count: 86829 Subject: [ruby-core:86829] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #13618 has been updated by ioquatix (Samuel Williams). I found an interesting summary of EPOLLET, which I think explains it better than I did: https://stackoverflow.com/a/46634185/29381 Basically, it minimise OS IPC. > According to Go user reports, being able to move goroutines > between native threads is a big feature to them. But I don't > think it's possible with current Ruby C API, anyways :< By definition Fibers shouldn't move between threads. If you can move the coroutine between threads, it's a green thread (user-scheduled thread). Writing code that use multiple fibers in a single thread provides strong guarantees about asynchronous execution. You can avoid things like mutex, condition variable, etc, and minimise (avoid) deadlocks and other problems of multiple threads. And as you say, GVL is a big problem so there is little reason to use it anyway. > Fwiw, yahns makes large performance sacrifices(*) to avoid HOL > blocking. And yet it has 2x the latency of `async-http`. Can you tell me how to test it in more favourable configuration? > The main thing which bothers me about both ET and LT is you have > to remember to disable/reenable events (to avoid unfairness or DoS). Fortunately C++ RAII takes care of this. > Under ideal conditions (clients not trying to DoS or be unfair > to other clients), ET can probably be fastest. Just totally > unrealistic to expect ideal conditions. We've used it in production systems and it's been great, serving millions of requests with no issues. I can see that we can discuss these things for a long time, and while I find it really interesting, we do need to move forward for Ruby's sake. I think the work you've done here is really great. I just think it needs to be slightly more modular; but not in a way that detracts from becoming a ubiquitous solution for non-blocking IO. It needs to be possible for concurrency library authors to process blocking operations with their own selector/reactor design. I really think there would be value in being able to write something like: ``` selector = NIO::Selector.new # or EventMachine, etc Fiber.new(selector: selector) do io.read # invokes selector.wait_readable(io) if EWOULDBLOCK # nested fibers can inherit parent selector. end.resume selector.run ``` Your selector implementation could fit into that, along with NIO4R, EventMachine, etc. I would REALLY like to see something like this. So, we can explore different models of concurrency. Sometimes we would like to choose different selector implementation for pragmatic reasons: On macOS, kqueue doesn't work with `tty` devices. But `select` does work fine, with lower performance. ``` # If program needs to block on TTY: selector = Thread::Selector.new(:select) # Otherwise selector = Thread::Selector.new(:kqueue) ``` In addition, such a design let's you easily tune parameters (like size of event queue, other details of the implementation that can significantly affect performance). In addition, I recently implemented a debug wrapper for `NIO::Selector` which detects unusual conditions. It detects undesirable conditions but it's typically only enabled when running tests. https://github.com/socketry/async/blob/master/lib/async/debug/selector.rb With such a design as proposed above, such a feature becomes trivial to implement. We can have a sane defaults. I don't mind how the API works, just that I can supply my own selector/reactor on a per-fiber basis. ---------------------------------------- Feature #13618: [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid https://bugs.ruby-lang.org/issues/13618#change-71792 * Author: normalperson (Eric Wong) * Status: Assigned * Priority: Normal * Assignee: normalperson (Eric Wong) * Target version: ---------------------------------------- ``` auto fiber schedule for rb_wait_for_single_fd and rb_waitpid Implement automatic Fiber yield and resume when running rb_wait_for_single_fd and rb_waitpid. The Ruby API changes for Fiber are named after existing Thread methods. main Ruby API: Fiber#start -> enable auto-scheduling and run Fiber until it automatically yields (due to EAGAIN/EWOULDBLOCK) The following behave like their Thread counterparts: Fiber.start - Fiber.new + Fiber#start (prelude.rb) Fiber#join - run internal scheduler until Fiber is terminated Fiber#value - ditto Fiber#run - like Fiber#start (prelude.rb) Right now, it takes over rb_wait_for_single_fd() and rb_waitpid() function if the running Fiber is auto-enabled (cont.c::rb_fiber_auto_sched_p) Changes to existing functions are minimal. New files (all new structs and relations should be documented): iom.h - internal API for the rest of RubyVM (incomplete?) iom_internal.h - internal header for iom_(select|epoll|kqueue).h iom_epoll.h - epoll-specific pieces iom_kqueue.h - kqueue-specific pieces iom_select.h - select-specific pieces iom_pingable_common.h - common code for iom_(epoll|kqueue).h iom_common.h - common footer for iom_(select|epoll|kqueue).h Changes to existing data structures: rb_thread_t.afrunq - list of fibers to auto-resume rb_vm_t.iom - Ruby I/O Manager (rb_iom_t) :) Besides rb_iom_t, all the new structs are stack-only and relies extensively on ccan/list for branch-less, O(1) insert/delete. As usual, understanding the data structures first should help you understand the code. Right now, I reuse some static functions in thread.c, so thread.c includes iom_(select|epoll|kqueue).h TODO: Hijack other blocking functions (IO.select, ...) I am using "double" for timeout since it is more convenient for arithmetic like parts of thread.c. Most platforms have good FP, I think. Also, all "blocking" functions (rb_iom_wait*) will have timeout support. ./configure gains a new --with-iom=(select|epoll|kqueue) switch libkqueue: libkqueue support is incomplete; corner cases are not handled well: 1) multiple fibers waiting on the same FD 2) waiting for both read and write events on the same FD Bugfixes to libkqueue may be necessary to support all corner cases. Supporting these corner cases for native kqueue was challenging, even. See comments on iom_kqueue.h and iom_epoll.h for nuances. Limitations Test script I used to download a file from my server: ----8<--- require 'net/http' require 'uri' require 'digest/sha1' require 'fiber' url = 'http://80x24.org/git-i-forgot-to-pack/objects/pack/pack-97b25a76c03b489d4cbbd85b12d0e1ad28717e55.idx' uri = URI(url) use_ssl = "https" == uri.scheme fibs = 10.times.map do Fiber.start do cur = Fiber.current.object_id # XXX getaddrinfo() and connect() are blocking # XXX resolv/replace + connect_nonblock Net::HTTP.start(uri.host, uri.port, use_ssl: use_ssl) do |http| req = Net::HTTP::Get.new(uri) http.request(req) do |res| dig = Digest::SHA1.new res.read_body do |buf| dig.update(buf) #warn "#{cur} #{buf.bytesize}\n" end warn "#{cur} #{dig.hexdigest}\n" end end warn "done\n" :done end end warn "joining #{Time.now}\n" fibs[-1].join(4) warn "joined #{Time.now}\n" all = fibs.dup warn "1 joined, wait for the rest\n" until fibs.empty? fibs.each(&:join) fibs.keep_if(&:alive?) warn fibs.inspect end p all.map(&:value) Fiber.new do puts 'HI' end.run.join ``` ---Files-------------------------------- 0001-auto-fiber-schedule-for-rb_wait_for_single_fd-and-rb.patch (82.8 KB) -- https://bugs.ruby-lang.org/