From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-3.2 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_PASS,T_RP_MATCHES_RCVD shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 1D77F20D0A for ; Thu, 1 Jun 2017 21:52:10 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id 6F1E6120764; Fri, 2 Jun 2017 06:52:02 +0900 (JST) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by neon.ruby-lang.org (Postfix) with ESMTPS id EE0A512072A for ; Fri, 2 Jun 2017 06:51:55 +0900 (JST) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 2F41E20D0A; Thu, 1 Jun 2017 21:51:54 +0000 (UTC) Date: Thu, 1 Jun 2017 21:51:54 +0000 From: Eric Wong To: ruby-core@ruby-lang.org Message-ID: <20170601215154.GA8771@dcvr> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-ML-Name: ruby-core X-Mail-Count: 81514 Subject: [ruby-core:81514] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" ko1@atdot.net wrote: > My opinion is opposite. I think "For human being using threading is too hard to use correctly" or "Rubyist shouldn't care about threading difficulties". I agree my opinion is extreme and many "advanced" programmers like Eric can write correct thread programs. But most (many? some? a few?) of ruby programmer (including me) can not write correct code I believe. I do not believe I can write correct code of any type, actually. Everything I write; even trivial single-threaded scripts has bugs. On the other hand, my likelyhood of introducing bugs seems nearly identical across any environment and programming models. However, having less/simpler code (and less dependencies) seems to result in fewer bugs, in my experience. > (In addition: I heard some advanced programmers say "people can write". I doubt because it is something survivor bias) Yes. > (recent days I fixed rubygems' threading problem it is difficult to reproduce) > > I often use this metaphor: It is like GC jtrategy. If people can manage object lifetime, it is faster than using GC (at some case. Some case GC is more faster than manual memory management). However we choose GC because we want to concentrate on writing application code. Right. However, it seems choosing "easier" strategies means less focus on overall design, leading to more problems down the line. Since around 2010; I believe unicorn caused major, irreparable damage to Rack ecosystem by promoting single-threaded design and having a SIGKILL timeout feature. unicorn made Rubyists stop caring to fix concurrency bugs and do proper timeouts. Nowadays Rack apps are both too buggy AND use too much memory :< I know some people disagree with my assessment of unicorn; but I prefer to hate everything I've done: it's easier to find improvements that way :) > I agree auto-fibers is safer than threads. In my mind: > > ``` > danger <-> safe (this is my opinion) > > parallel threads (JRuby, ...) > concurrent threads (MRI) >> > auto-fibers (full-auto) > auto-fiber (restricted) >> > Guild > single thread Agree. So maybe we can design API for "auto-fiber (restricted)"? > But auto-fiber can introduce accident and it should be not so frequent, and it is difficult to reproduce. This means it is difficult to debug. > > Ruby has many pit falls to shoot our own legs (meta-programming features, open class and so on) but they are deterministic (at most of case). Yes. I think these (along too much code + dependencies) cause more problems than concurrency bugs. > normalperson (Eric Wong) wrote: > > Yes; we will document all switch points in RDoc and NEWS, > > of course (maybe write a separate doc/auto-fiber.rdoc) > > My point is, if method "foo" is switching point, then any method can call "foo" (bar, and baz, the caller of bar, ...) should be noted. Maybe it is impossible to complete because of Ruby's dynamic nature. Right. Maybe that is a lot of documentation... What if the API were the opposite of Thread.exclusive/Mutex#synchronize? Perhaps: Fiber.new do Fiber.auto do # enable auto-fiber inside this block end # disable auto-fiber again end Maybe Fiber.exclusive can disable Fiber.auto temporarily: Fiber.new do Fiber.auto do # enable auto-fiber Fiber.exclusive do # temporarily disable auto-fiber end # enable auto-fiber again ... end end Fiber.auto/Fiber.exclusive would be no-ops unless inside a Fiber.new block... But maybe that is too much code and nesting levels; so I still like Fiber.start more. > > I would like to use existing stdlib (net/*, webrick, drb, ...) > > as much as possible without modifications. That means many > > existing Ruby libraries can work transparently. > > I understand your point. Thanks; that is my biggest wish for this feature. Anyways, I will leave matz, you and others deal with final API decisions.