From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (smtp.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id 8D6311B60168 for ; Sun, 2 Apr 2017 10:58:26 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id 50D47B5D8E7 for ; Sun, 2 Apr 2017 11:35:21 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id 8ED6D18CC81C for ; Sun, 2 Apr 2017 11:35:21 +0900 (JST) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 1E0CE120726; Sun, 2 Apr 2017 11:35:20 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by neon.ruby-lang.org (Postfix) with ESMTPS id 413481206EA; Sun, 2 Apr 2017 11:35:16 +0900 (JST) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 71B1C20964; Sun, 2 Apr 2017 02:35:14 +0000 (UTC) Date: Sun, 2 Apr 2017 02:35:14 +0000 From: Eric Wong To: SASADA Koichi Message-ID: <20170402023514.GB30476@dcvr> References: <20170402011414.AEA9B64CEE@svn.ruby-lang.org> <8a2b82e3-dc07-1945-55f9-5a474e89130b@ruby-lang.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <8a2b82e3-dc07-1945-55f9-5a474e89130b@ruby-lang.org> X-ML-Name: ruby-core X-Mail-Count: 80531 Cc: Ruby developers Subject: [ruby-core:80531] Re: [ruby-cvs:65407] normal:r58236 (trunk): thread.c: comments on M:N threading [ci skip] X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" SASADA Koichi wrote: > Hi Eric, > > I agree it is possible to use M:N model. > > There are several problem mainly because of C extensions (and libraries > which C extensions use). > > 1.We can't move execution context across native threads by userland > because some libraries can use thread local variables used by > native thread system. > Also because some libraries can depends on C-stack layout. Hi ko1, thank you for response. Correct, I'm not sure if this can be changed while maintaining compatibility. Anyways I think I am fine with this limitation where a Fiber is always tied to a particular native thread. > 2.Some libraries can stop threads because of uncontrollable > I/O operations (wait for IO (network) with system calls), > uncontrollable system synchronization (mutex, semaphore, ...), > big computation (like mathematical computation). > > Current 1:1 thread model does not have such issues (if C extensions > release GVL correctly), so we employee it. > However, the overhead of thread creation and thread switching is high, > as you say. > > However, the issues 1 and 2 are *possible* issues. We don't know which > libraries have a problem. If we only use managed C extensions, there are > no problem to use M:N mode. 2 is tricky. I think in worst case (no modifying existing C exts or API), M:N will degrade to current 1:1 model, which retains 100% compatibility with current Ruby 1.9/2.x code. > For Ruby 3, Matz want to encourage such fine grain context switching. We > discussed before and we planed to introduce automatic Fiber switching at > specific I/O operation. It is small version of your proposal, it is one > possibility (and it is easy than complete M:N model). I'll change Fiber > context management to lightwieght switching for Ruby 2.5 and try it at > later versions. Cool! I was thinking along the same lines. I think Ruby Thread class can become a subclass of Fiber with automatic switching. However, to spawn native threads: If a Thread uses existing GVL release C-API, then the _next_ Thread.new call will create a native thread (and future Thread.new will be subclass of Fiber in new native thread). So, in pseudo code: class Thread < Fiber def self.new case Thread.current[:gvl_state] when :none # default super # M += 1 when :released # this is set by BLOCKING_REGION GVL release # only allow a user-level thread to spawn one new native thread Thread.current[:gvl_state] = :spawned NativeThread.new { Thread.new } # N += 1 when :spawned # We already spawned on native thread from this user-level # thread, only spawn new user-level thread for now. super # M += 1 end end end Current GVL release operations will change Thread.current[:gvl_state] from :none -> :released