From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 2F2811F55B for ; Thu, 14 May 2020 09:25:48 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 037C6120B41; Thu, 14 May 2020 18:25:22 +0900 (JST) Received: from o1678948x4.outbound-mail.sendgrid.net (o1678948x4.outbound-mail.sendgrid.net [167.89.48.4]) by neon.ruby-lang.org (Postfix) with ESMTPS id D60E8120B41 for ; Thu, 14 May 2020 18:25:18 +0900 (JST) Received: by filter0084p3las1.sendgrid.net with SMTP id filter0084p3las1-1974-5EBD0E8F-E2 2020-05-14 09:25:35.807911872 +0000 UTC m=+2460227.945418787 Received: from herokuapp.com (unknown) by ismtpd0053p1iad1.sendgrid.net (SG) with ESMTP id lV5kbS26TAy2Mlbq8cAslQ for ; Thu, 14 May 2020 09:25:35.669 +0000 (UTC) Date: Thu, 14 May 2020 09:25:35 +0000 (UTC) From: duerst@it.aoyama.ac.jp Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 74141 X-Redmine-Project: ruby-master X-Redmine-Issue-Tracker: Feature X-Redmine-Issue-Id: 16786 X-Redmine-Issue-Author: ioquatix X-Redmine-Sender: duerst X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?uQY=2F2xNrNfHHTWbKn6MBvvzfU5Pqk9I4lnOVb0CFDutiLvDyPwjc65CRoigVRr?= =?us-ascii?Q?WaAXvbkO8cJ7H5uVQD3lyNTTqASUGjn=2Fk9qZYP0?= =?us-ascii?Q?nE65zt0zYAizMXBU5oB00Nat8aBgq7jPeMjArax?= =?us-ascii?Q?CRyLTYiStzq26Cy6=2F3GUG0LYNiVGg5tsaHvWiW4?= =?us-ascii?Q?bVKJxqT0Hk5juDBICjNOaV5AXowiRyipSnA=3D=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 98349 Subject: [ruby-core:98349] [Ruby master Feature#16786] Light-weight scheduler for improved concurrency. X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #16786 has been updated by duerst (Martin D=FCrst). ioquatix (Samuel Williams) wrote in #note-34: > Thanks Matz. > = > > since the fiber created by the method is not the original fiber at all. > = > Can you clarify "not the original fiber at all"? It's the same way `Integ= er(...)` creates instance of `class Integer`. I can't speak for Matz, but my guess is that he meant "not the original typ= e of fiber", i.e. not the same as you'd get e.g. with `Fiber.new`. ---------------------------------------- Feature #16786: Light-weight scheduler for improved concurrency. https://bugs.ruby-lang.org/issues/16786#change-85616 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal ---------------------------------------- # Abstract We propose to introduce a light weight fiber scheduler, to improve the conc= urrency of Ruby code with minimal changes. # Background We have been discussing and considering options to improve Ruby scalability= for several years. More context can be provided by the following discussio= ns: - https://bugs.ruby-lang.org/issues/14736 - https://bugs.ruby-lang.org/issues/13618 The final Ruby Concurrency report provides some background on the various i= ssues considered in the latest iteration: https://www.codeotaku.com/journal= /2020-04/ruby-concurrency-final-report/index # Proposal We propose to introduce the following concepts: - A `Scheduler` interface which provides hooks for user-supplied event loop= s. - Non-blocking `Fiber` which can invoke the scheduler when it would otherwi= se block. ## Scheduler The per-thread fiber scheduler interface is used to intercept blocking oper= ations. A typical implementation would be a wrapper for a gem like EventMac= hine or Async. This design provides separation of concerns between the even= t loop implementation and application code. It also allows for layered sche= dulers which can perform instrumentation, enforce constraints (e.g. during = testing) and provide additional logging. You can see a [sample implementati= on here](https://github.com/socketry/async/pull/56). ```ruby class Scheduler # Wait for the given file descriptor to become readable. def wait_readable(io) end # Wait for the given file descriptor to become writable. def wait_writable(io) end # Wait for the given file descriptor to match the specified events within # the specified timeout. # @param event [Integer] a bit mask of +IO::WAIT_READABLE+, # `IO::WAIT_WRITABLE` and `IO::WAIT_PRIORITY`. # @param timeout [#to_f] the amount of time to wait for the event. def wait_any(io, events, timeout) end # Sleep the current task for the specified duration, or forever if not # specified. # @param duration [#to_f] the amount of time to sleep. def wait_sleep(duration =3D nil) end # The Ruby virtual machine is going to enter a system level blocking # operation. def enter_blocking_region end # The Ruby virtual machine has completed the system level blocking # operation. def exit_blocking_region end # Intercept the creation of a non-blocking fiber. def fiber(&block) Fiber.new(blocking: false, &block) end # Invoked when the thread exits. def run # Implement event loop here. end end ``` A thread has a non-blocking fiber scheduler. All blocking operations on non= -blocking fibers are hooked by the scheduler and the scheduler can switch t= o another fiber. If any mutex is acquired by a fiber, then a scheduler is n= ot called; the same behaviour as blocking Fiber. Schedulers can be written in Ruby. This is a desirable property as it allow= s them to be used in different implementations of Ruby easily. To enable non-blocking fiber switching on blocking operations: - Specify a scheduler: `Thread.current.scheduler =3D Scheduler.new`. - Create several non-blocking fibers: `Fiber.new(blocking:false) {...}`. - As the main fiber exits, `Thread.current.scheduler.run` is invoked which begins executing the event loop until all fibers are finished. ### Time/Duration Arguments Tony Arcieri suggested against using floating point values for time/duratio= ns, because they can accumulate rounding errors and other issues. He has a = wealth of experience in this area so his advice should be considered carefu= lly. However, I have yet to see these issues happen in an event loop. That = being said, round tripping between `struct timeval` and `double`/`VALUE` se= ems a bit inefficient. One option is to have an opaque argument that respon= ds to `to_f` as well as potentially `seconds` and `microseconds` or some ot= her such interface (could be opaque argument supported by `IO.select` for e= xample). ### File Descriptor Arguments Because of the public C interface we may need to support a specific set of = wrappers for CRuby. ```c int rb_io_wait_readable(int); int rb_io_wait_writable(int); int rb_wait_for_single_fd(int fd, int events, struct timeval *tv); ``` One option is to introduce hooks specific to CRuby: ```ruby class Scheduler # Wrapper for rb_io_wait_readable(int) C function. def wait_readable_fd(fd) wait_readable(::IO.from_fd(fd, autoclose: false)) end # Wrapper for rb_io_wait_readable(int) C function. def wait_writable_fd(fd) wait_writable(::IO.from_fd(fd, autoclose: false)) end # Wrapper for rb_wait_for_single_fd(int) C function. def wait_for_single_fd(fd, events, duration) wait_any(::IO.from_fd(fd, autoclose: false), events, duration) end end ``` Alternatively, in CRuby, it may be possible to map from `fd` -> `IO` instan= ce. Most C schedulers only care about file descriptor, so such a mapping wi= ll introduce a small performance penalty. In addition, most C level schedul= ers will not care about `IO` instance. ## Non-blocking Fiber We propose to introduce per-fiber flag `blocking: true/false`. A fiber created by `Fiber.new(blocking: true)` (the default `Fiber.new`) be= comes a "blocking Fiber" and has no changes from current Fiber implementati= on. This includes the root fiber. A fiber created by `Fiber.new(blocking: false)` becomes a "non-blocking Fib= er" and it will be scheduled by the per-thread scheduler when the blocking = operations (blocking I/O, sleep, and so on) occurs. ```ruby Fiber.new(blocking: false) do puts Fiber.current.blocking? # false # May invoke `Thread.scheduler&.wait_readable`. io.read(...) # May invoke `Thread.scheduler&.wait_writable`. io.write(...) # Will invoke `Thread.scheduler&.wait_sleep`. sleep(n) end.resume ``` Non-blocking fibers also supports `Fiber#resume`, `Fiber#transfer` and `Fib= er.yield` which are necessary to create a scheduler. ### Fiber Method We also introduce a new method which simplifes the creation of these non-bl= ocking fibers: ```ruby Fiber do puts Fiber.current.blocking? # false end ``` This method invokes `Scheduler#fiber(...)`. The purpose of this method is t= o allow the scheduler to internally decide the policy for when to start the= fiber, and whether to use symmetric or asymmetric fibers. If no scheduler is specified, it is a error: `RuntimeError.new("No schedule= r is available")`. In the future we may expand this to support some kind of default scheduler. ## Non-blocking I/O `IO#nonblock` is an existing interface to control whether I/O uses blocking= or non-blocking system calls. We can take advantage of this: - `IO#nonblock =3D false` prevents that particular IO from utilising the sc= heduler. This should be the default for `stderr`. - `IO#nonblock =3D true` enables that particular IO to utilise the schedule= r. We should enable this where possible. As proposed by Eric Wong, we believe that making I/O non-blocking by defaul= t is the right approach. We have expanded his work in the current implement= ation. By doing this, when the user writes `Fiber do ... end` they are guar= anteed the best possible concurrency possible, without any further changes = to code. As an example, one of the tests shows `Net::HTTP.get` being used i= n this way with no further modifications required. To support this further, consider the counterpoint, that `Net::HTTP.get(...= , blocking: false)` is required for concurrent requests. Library code may n= ot expose the relevant options, sevearly limiting the user's ability to imp= rove concurrency, even if that is what they desire. # Implementation We have an evolving implementation here: https://github.com/ruby/ruby/pull/= 3032 which we will continue to update as the proposal changes. # Evaluation This proposal provides the hooks for scheduling fibers. With regards to per= formance, there are several things to consider: - The impact of the scheduler design on non-concurrent workloads. We believ= e it's acceptable. - The impact of the scheduler design on concurrent workloads. Our results a= re promising. - The impact of different event loops on throughput and latency. We have in= dependent tests which confirm the scalability of the approach. We can control for the first two in this proposal, and depending on the des= ign we may help or hinder the wrapper implementation. In the tests, we provide a basic implementation using `IO.select`. As this = proposal is finalised, we will introduce some basic benchmarks using this a= pproach. # Discussion The following points are good ones for discussion: - Handling of file descriptors vs `IO` instances. - Handling of time/duration arguments. - General design and naming conventions. - Potential platform issues (e.g. CRuby vs JRuby vs TruffleRuby, etc). The following is planned to be described by @eregon in another design docum= ent: - Semantics of non-blocking mutex (e.g. `Mutex.new(blocking: false)` or som= e other approach). In the future we hope to extend the scheduler to handle other blocking oper= ations, including name resolution, file I/O (by `io_uring`) and others. We = may need to introduce additional hooks. If these hooks are not defined on t= he scheduler implementation, we will revert back to the blocking implementa= tion where possible. -- = https://bugs.ruby-lang.org/ Unsubscribe: