From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 6260F1F404 for ; Sat, 1 Sep 2018 13:13:12 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 572C9120997; Sat, 1 Sep 2018 22:13:10 +0900 (JST) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by neon.ruby-lang.org (Postfix) with ESMTPS id 0E73D120994 for ; Sat, 1 Sep 2018 22:13:05 +0900 (JST) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 530761F404; Sat, 1 Sep 2018 13:13:01 +0000 (UTC) Date: Sat, 1 Sep 2018 13:13:01 +0000 From: Eric Wong To: ruby-core@ruby-lang.org Message-ID: <20180901131301.5peghdyrtmks5mka@dcvr> References: <20180827202714.tqbwqrdbykywlqvq@dcvr> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180827202714.tqbwqrdbykywlqvq@dcvr> X-ML-Name: ruby-core X-Mail-Count: 88800 Cc: Greg Subject: [ruby-core:88800] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" > https://bugs.ruby-lang.org/issues/13618 Greg, do you think you can try this change for portability? No rush, though, I will be mostly AFK and Queue/SizedQueue changes will be trivial with the prep work so far... Future changes to this feature shouldn't impact, portability. The main one is [PATCH 2/2] * [PATCH 1/2] unify sync_waiter, waitpid_state, waiting_fd w/ rb_sched_waiter https://80x24.org/spew/20180901131012.22138-1-e@80x24.org/raw * [PATCH 2/2] Thread::Coro: green threads implemented using fibers https://80x24.org/spew/20180901131012.22138-2-e@80x24.org/raw The following changes since commit 929e9713bbfd76140bced29c6f398904ae9d4a85: complex.c: simplify division result (2018-09-01 07:34:31 +0000) are available in the Git repository at: https://80x24.org/ruby.git coro-r64610 for you to fetch changes up to 13c51c9c9ae39dae08497cfe0eb119244c4d2224: Thread::Coro: green threads implemented using fibers (2018-09-01 12:56:08 +0000) Newest updates in this version: - new name: Thread::Coro (reject Thriber/Threadlet) - Fiber#transfer used for switching Thanks to funny.falcon for suggestion [ruby-core:87776], it made the code better, even :) - improve IO.select mapping in kqueue/epoll (st_table => ccan/list), since there is no need for lookup, only scan - sync to use MJIT-friendly rb_waitpid (added th->nogvl_runq since waitpid(2) can run without GVL) - "ensure" support. Only for Thread::Coro, Fiber has no ensure (see [Bug #595]). "ensure" is REQUIRED for auto-scheduling safety. I am not sure if regular Fiber can support "ensure" in a way which is both performant and backwards-compatible. With Thread::Coro being a new feature, there is no backwards compatibility to worry about, so the "ensure" support adds practically no overhead - more code sharing between iom_{select,kqueue,epoll.h} - switch to rb_hrtime_t for timeouts - extract timeout API, so non-timeout-arg users can benefit from reduced. This will make Timeout-in-VM support easier and more orthogonal to this one. Changes to existing data structures: * rb_thread_t .runq - list of Thread::Coros to auto-resume .nogvl_runq - same as above for out-of-GVL use .waiting_coro - list of blocked Coros, for "ensure" support. For auto-scheduling, we must have ensure support because Fiber does not yet support ensure Autoload has similar thread waiting, but I'm not sure if should > be affected by auto-fiber switching. Along those lines, I'm not sure what the semantic should be with regards to Mutex. Mutex is to protect data from parallel modifications, but Thread::Coro do not run in parallel. if a native Thread and Thread::Coro contend on the same Mutex; what should happen?