From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-4.0 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 7F7641F453 for ; Wed, 1 May 2019 22:41:36 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 9D7FC120A68; Thu, 2 May 2019 07:41:31 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id 99B89120A13 for ; Thu, 2 May 2019 07:41:29 +0900 (JST) Received: by filter0163p3mdw1.sendgrid.net with SMTP id filter0163p3mdw1-890-5CCA209A-18 2019-05-01 22:41:30.761172967 +0000 UTC m=+526442.919081996 Received: from herokuapp.com (unknown [54.159.201.37]) by ismtpd0005p1iad1.sendgrid.net (SG) with ESMTP id JHgtZQrcQpewy_ZD28IJ7w for ; Wed, 01 May 2019 22:41:30.796 +0000 (UTC) Date: Wed, 01 May 2019 22:41:30 +0000 (UTC) From: samuel@oriontransfer.net Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 68005 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 14736 X-Redmine-Issue-Author: ioquatix X-Redmine-Sender: ioquatix X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?cjxb6GWHefMLoR50bkJBcGo6DRiDl=2FNYcMZdY+Wj30TaKc43nIUgukcHh3757=2F?= =?us-ascii?Q?rpBhSydNo2+EspDhSVBm39kyUFk+O69+C9jHURW?= =?us-ascii?Q?5sXeFU0rN1j6LXCduGVUEs7NhMJPhNvlebR2GkR?= =?us-ascii?Q?CZswXCemTmgvJ8DR0HgmFyPbzBI8ikrMFdxGOIZ?= =?us-ascii?Q?SbPV53z4KdS8IMJiY5EqwUIAyv5VORXxivQ=3D=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 92522 Subject: [ruby-core:92522] [Ruby trunk Feature#14736] Thread selector for flexible cooperative fiber based concurrency X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #14736 has been updated by ioquatix (Samuel Williams). > how to handle disk IO such as File.read While many platforms don't support non-blocking disk IO, `liburing` for linux and `IOCP` on Windows might present some opportunities for increased concurrency. Alternatively, some implementations (e.g. `libuv`) use a thread pool for blocking disk IO. So, I believe it should be up to the selector to decide how to handle it. > The selectors from the PR are per-Thread, but maybe we want something finer-grained. So, there is no reason why someone couldn't implement a thread-safe selector and assign it on multiple threads, or assign to Thread#selector multiple times depending on context. That's how `async` works using thread locals. ---------------------------------------- Feature #14736: Thread selector for flexible cooperative fiber based concurrency https://bugs.ruby-lang.org/issues/14736#change-77885 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- Ruby concurrency can be greatly enhanced by concurrent, deterministic IO. Fibers have been shown many times to be a great abstraction for this purpose. The retain normal code flow and don't require any kind of Thread synchronisation. They are enjoyable to write code with because you don't have to concern yourself with thread synchronisation or other complicated issues. The basic idea is that if some operation would block, it yields the Fiber, and other Fibers within the thread can continue to execute. There are a number of ways to implement this. Here is a proof of concept to amend the existing `rb_io_wait_readable`/`rb_io_wait_writable`. https://github.com/ruby/ruby/pull/1870 This design minimally affects the Ruby implementation and allows flexibility for selector implementation. With a small amount of work, we can support EventMachine (65 million downloads), NIO4r (21 million downloads). It would be trivial to back port. This PR isn't complete but I am seeking feedback. If it's a good idea, I will do my best to see it through to completion, including support for EventMachine and NIO4r. ---Files-------------------------------- port_scanner_threadlet.rb (925 Bytes) -- https://bugs.ruby-lang.org/