From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: poffice@blade.nagaokaut.ac.jp Delivered-To: poffice@blade.nagaokaut.ac.jp Received: from kankan.nagaokaut.ac.jp (kankan.nagaokaut.ac.jp [133.44.2.24]) by blade.nagaokaut.ac.jp (Postfix) with ESMTP id 0E3591BA007E for ; Tue, 9 May 2017 13:28:50 +0900 (JST) Received: from voscc.nagaokaut.ac.jp (voscc.nagaokaut.ac.jp [133.44.1.100]) by kankan.nagaokaut.ac.jp (Postfix) with ESMTP id 603EEB5D83B for ; Tue, 9 May 2017 14:12:30 +0900 (JST) Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by voscc.nagaokaut.ac.jp (Postfix) with ESMTP id C976818D1D8A for ; Tue, 9 May 2017 14:12:30 +0900 (JST) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 940E612074F; Tue, 9 May 2017 14:12:29 +0900 (JST) X-Original-To: ruby-core@ruby-lang.org Delivered-To: ruby-core@ruby-lang.org Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by neon.ruby-lang.org (Postfix) with ESMTPS id 97B47120718 for ; Tue, 9 May 2017 14:12:25 +0900 (JST) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 149221FF34; Tue, 9 May 2017 05:12:24 +0000 (UTC) Date: Tue, 9 May 2017 05:12:23 +0000 From: Eric Wong To: ruby-core@ruby-lang.org Message-ID: <20170509051223.GA31857@whir> References: <20170403044254.GA16328@starla> <20170508003315.GA3789@starla> <38090d10-c6a1-5097-66af-130275d773ea@atdot.net> <2b47c736-08d8-095b-0454-2dd0b1020b03@atdot.net> <20170508030120.GB24763@starla> <94ca8f9a-7001-12d3-323d-8c5751569c51@atdot.net> <20170508063633.GA6821@starla> <4a83bbeb-b22b-61ec-a03f-657746843431@atdot.net> <20170509033806.GA27973@starla> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-ML-Name: ruby-core X-Mail-Count: 81047 Subject: [ruby-core:81047] Re: [ruby-cvs:65407] normal:r58236 (trunk): thread.c: comments on M:N threading [ci skip] X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" SASADA Koichi wrote: > On 2017/05/09 12:38, Eric Wong wrote: > > 100 epoll FDs is a waste of FDs; especially since it is common > > to have a 1024 FD limit. I already feel bad about timer thread > > taking up two FDs; but maybe epoll/kevent can cut reduce that. > > 1024 soft limit and 4096 hard limit is an issue. However, if we employ > > > I can easily imagine Ruby doing 100 native threads in one process > > (8 cores, 10-20 rotational disks, 2 SSD), but 20000-30000 fibers. > > 20000-30000 fibers, it is also problem if they have corresponding fds. > So that I think people increase this limit upto 65K, don't? Yes, for people that run 20000-30000 fibers maybe it is not a problem to have 100 epoll FD... However, for existing apps like puma, webrick and net/http-based scripts: they can spawn dozens/hundreds of threads and only use one socket per thread. It is a waste to use epoll/kqueue to watch a few number of FD per thread (ppoll is more appropriate for watching a single FD). On the contrary; software like nginx and cmogstored watch thousands of FDs with a single epoll|kqueue FD. > > In the kernel, every "struct eventpoll" + "struct file" in > > Linux is at least 400 bytes of unswappable kernel memory. > > 400B * 100 = 40KB. Is it problem? I have no knowledge to evaluate this > size (10 pages seems not so small, I guess). I'd rather not use that much memory and save whereever possible. > > OK, I can rename my work-in-progress patch with > > s/rb_thread_context_t/rb_execution_context_t/ and commit > > later tonight. > > Ah, that was my plan and I'm not sure what is suitable name (always I > consumes long time for naming problem). But if you don't feel weird, > please use execution_context (ec). OK, I committed as r58614 > Do you want to commit your patch into trunk immediately and change them > for "(2-1: extend Fiber)" later? Another way is to make "(2-1: extend > Fiber)" first (in another branch or git repository) and commit it. The > latter can reduce total patch size. OK, I will work on implementing epoll/kqueue support late this week or weekend. I will also keep a select() fallback for portability to systems w/o epoll|kqueue.