From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id 21BDE1BA0019 for ; Wed, 10 May 2017 11:40:48 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id DA28EB5D891 for ; Wed, 10 May 2017 12:24:38 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id 5544B18CC80E for ; Wed, 10 May 2017 12:24:39 +0900 (JST) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 3E3DC120723; Wed, 10 May 2017 12:24:38 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from mail.atdot.net (ik1-326-23156.vs.sakura.ne.jp [153.126.180.160]) by neon.ruby-lang.org (Postfix) with ESMTP id B8B6F1205E0 for ; Wed, 10 May 2017 12:24:34 +0900 (JST) To: Ruby developers References: <20170508030120.GB24763@starla> <94ca8f9a-7001-12d3-323d-8c5751569c51@atdot.net> <20170508063633.GA6821@starla> <4a83bbeb-b22b-61ec-a03f-657746843431@atdot.net> <20170509033806.GA27973@starla> <20170509051223.GA31857@whir> <20170509062353.GB8654@starla> <20170509185140.GA17410@starla> From: SASADA Koichi X-Enigmail-Draft-Status: N1110 Message-ID: <5c6c1af4-109f-ebd7-07aa-0f8dd77df9b8@atdot.net> Date: Wed, 10 May 2017 12:24:32 +0900 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170509185140.GA17410@starla> X-ML-Name: ruby-core X-Mail-Count: 81083 Subject: [ruby-core:81083] Re: [ruby-cvs:65407] normal:r58236 (trunk): thread.c: comments on M:N threading [ci skip] X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Thank you. (quote it first) > Do you have any deadlines or priorities? No. I want to make clear your priority to avoid conflicts with me. On 2017/05/10 3:51, Eric Wong wrote: >> (1) lightweight fiber switching by pointer-exchange >> (w/o copying context). > > Out of all tasks here, I am least familiar with this (1). > This will be learning experience for me. > >> (2) auto-fiber swiching >> (2-1) implement with epoll/kqueue/select >> (2-2) design APIs to use it > > I think I will start on the select implementation first for > portability, but model our internal API around epoll(*). > I will probably implement epoll support last, since I am > most familiar with it. > > (*) with current GVL, I expect our kqueue+kevent implementation > will be faster than epoll in most cases (the API requires > fewer syscalls). select might be fastest with few FDs. (1) and (2) are independent so that we can do it parallel. Do you want to try (1) first or can I try (1)? Yes, Doing (1) is good to learn core internal, but maybe (1) affect many places in VMs. So I want to try. Anyway we should make new ticket and discuss on it. I guess (2) is not so easy to design APIs. * We need to survey other languages * We need to define blocking operations: * blocking operation can switch Fibers automatically (I/O read, ...) * blocking operation can switch Fibers automatically (other than epoll/kqueue/select manage-able operations, extra-C exts providing blocking operations, ...) And how to provide such difference to users? * Idea: Documentation * example: POSIX singal safe functions * example: Java's thread-safety Generally, it is hard to use because users should them carefully (and usually people don't). * Idea: Provide new APIs which support auto-fibers (and other blocking operations don't support) * example: EventMachine, ... (other language example? Python?) * it is clear for users. * it is hard to import existing code * Idea: Provide a new TracePoint probe to know blocking operation which does not support auto-fibers. * This idea is for advanced user to check their scheduling * I think it is enough because * advanced user should be production maker. Automatic tools are preferable. * not advanced user don't care which operation can stop forever w/o auto-fiber switching * We need to define auto-Fiber constructor * ... Ah, I remember that we have (2') providing epoll/kqueue like Ruby interface. (2) use them in scheduler internally and only auto-fibers use it. However, someone want to use them and want to write their own scheduler (like nodejs culture). I'm not sure we should expose such interface but it is valuable to consider. If we decide to provide such APIs, we need to share the implementation (or shouldn't?). Furthermore, it is more easy to provide such APIs compare with providing auto-fibers. >> (3) Implement GVL with futex (in your comment) > > Maybe last. Linux-only, single (but most important) > platform; and the single CPU regression needs to be fixed. Cool. >> (4) Re-implement Queue (some days ago you wrote) > > I already had some work-in-progress patches I can cleanup and > send out to redmine for review later (also ConditionVariable). > Last I remember, there was a small performance regression for > small Queue/Condvar waiter lists due to better locality on embed > structs. However, I think avoiding O(n) rb_ary_delete behavior > is more important for busy queues. OK. > >> (please add your plan if you have others) > > I might break out thread.c and io.c into smaller files > (select/epoll/kqueue/timer_thread/copy_stream/...) > to make code organization easier. Not sure we can do it for io.c. Please ask someone else. > Also, I will try not to break platforms I don't use. As you > know I only use Free Software, so I would appreciate if you or > platform maintainers can help fix portability bugs on non-Free > systems. Absolutely. -- // SASADA Koichi at atdot dot net