From mboxrd@z Thu Jan 1 00:00:00 1970 Delivered-To: chneukirchen@gmail.com Received: by 10.142.191.1 with SMTP id o1cs210558wff; Wed, 16 Dec 2009 04:39:01 -0800 (PST) Received: from mr.google.com ([10.224.140.14]) by 10.224.140.14 with SMTP id g14mr320398qau.48.1260967141024 (num_hops = 1); Wed, 16 Dec 2009 04:39:01 -0800 (PST) Received: by 10.224.140.14 with SMTP id g14mr37997qau.48.1260967139641; Wed, 16 Dec 2009 04:38:59 -0800 (PST) X-BeenThere: rack-devel@googlegroups.com Received: by 10.224.103.10 with SMTP id i10ls342247qao.1.p; Wed, 16 Dec 2009 04:38:58 -0800 (PST) Received: by 10.224.43.17 with SMTP id u17mr138908qae.22.1260967138305; Wed, 16 Dec 2009 04:38:58 -0800 (PST) Received: by 10.224.43.17 with SMTP id u17mr138907qae.22.1260967138271; Wed, 16 Dec 2009 04:38:58 -0800 (PST) Return-Path: Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.24]) by gmr-mx.google.com with ESMTP id 24si231397qyk.14.2009.12.16.04.38.57; Wed, 16 Dec 2009 04:38:57 -0800 (PST) Received-SPF: pass (google.com: domain of jftucker@gmail.com designates 74.125.92.24 as permitted sender) client-ip=74.125.92.24; Received: by qw-out-2122.google.com with SMTP id 5so159351qwd.49 for ; Wed, 16 Dec 2009 04:38:57 -0800 (PST) Received: by 10.224.116.137 with SMTP id m9mr584034qaq.319.1260967137036; Wed, 16 Dec 2009 04:38:57 -0800 (PST) Return-Path: Received: from ?192.168.101.102? ([199.172.234.251]) by mx.google.com with ESMTPS id 6sm2449336qwk.51.2009.12.16.04.38.55 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 16 Dec 2009 04:38:56 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1077) Subject: Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows! From: James Tucker In-Reply-To: <20091215213225.GA15190@dcvr.yhbt.net> Date: Wed, 16 Dec 2009 12:38:53 +0000 Message-Id: References: <20091211201958.GD2121@dcvr.yhbt.net> <20091213205355.GB30229@dcvr.yhbt.net> <20091214184227.GB12789@dcvr.yhbt.net> <102BE9BA-FC74-4C5C-A37B-A526C0E7A010@gmail.com> <20091215043758.GB29127@dcvr.yhbt.net> <20091215213225.GA15190@dcvr.yhbt.net> To: rack-devel@googlegroups.com X-Mailer: Apple Mail (2.1077) X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jftucker@gmail.com designates 74.125.92.24 as permitted sender) smtp.mail=jftucker@gmail.com; dkim=pass (test mode) header.i=@gmail.com X-Original-Sender: jftucker@gmail.com Reply-To: rack-devel@googlegroups.com Precedence: list Mailing-list: list rack-devel@googlegroups.com; contact rack-devel+owners@googlegroups.com List-ID: List-Post: , List-Help: , List-Archive: X-Thread-Url: http://groups.google.com/group/rack-devel/t/1214d460ed982748 X-Message-Url: http://groups.google.com/group/rack-devel/msg/919ccd1709ea4576 Sender: rack-devel@googlegroups.com List-Unsubscribe: , List-Subscribe: , Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable On 15 Dec 2009, at 21:32, Eric Wong wrote: > James Tucker wrote: >> On 15 Dec 2009, at 04:37, Eric Wong wrote: >>> James Tucker wrote: >>>> On 14 Dec 2009, at 18:42, Eric Wong wrote: >>>>> James Tucker wrote: >>>>>> I really want to work out an abstraction away from IO instances, = #read >>>>>> and #write for this stuff. It's highly coupled, getting in the = way of >>>>>> tests, and heavy lifting environments. I have big plans for Rack = 2.0 >>>>>> to remove all IO that has not been properly abstracted and = decoupled >>>>>> from implementation details, but that's a long way off, mostly = due to >>>>>> lack of time and incentive. In the meantime, I can implore you = all to >>>>>> take steps in the right direction :-) >>>>>=20 >>>>> Huh? I don't see what the problem with IO instances/semantics is, >>>>> especially with the availability of StringIO for testing. = "rack.input" >>>>> is clearly specified and works fine as-is IMHO, though the = rewindability >>>>> requirement does add some unnecessary overhead. >>>>=20 >>>> I disagree with "works fine". It does not work fine with Thin, in >>>> fact, on the contrary, it forces some real ugliness into the = system. I >>>> also think that StringIO is a really unfortunate thing to have to >>>> resort to, as it has so many short comings. >>>=20 >>> Mind elaborating? I do not see what's ugly about it[1]. >=20 >> Sure. It forces authors to several (bad) APIs options for handling >> complex IO scenarios. >>=20 >> 1. Buffer the entire request and then republish to the application >> 2. Pass forward the raw IO >=20 > 3. stream input with a rewindable backing store like TeeInput: > http://unicorn.bogomips.org/Unicorn/TeeInput.html > This is already supported by several (but not all) Rainbows! > concurrency models: http://rainbows.rubyforge.org/Summary.html >=20 > I'll need to use Fibers to get this working with Rev and EventMachine, > (the FiberSpawn/FiberPool models are select-based), but exposing an > outwardly synchronous model is far easier for application developers = to > grok. >=20 >>> [1] - I admit to having a Unix bias here where >>> nearly everything is a file, or ought to be :) >=20 >> We're not stream processing here. It's a request/response cycle. And >> they aren't files, they're sockets. >=20 > We can be stream processing, see examples/echo.ru in the Unicorn = source > which streams input to the output body: >=20 > http://git.bogomips.org/cgit/unicorn.git/tree/examples/echo.ru >=20 >> Unix philosophy is good for the shell, but it doesn't extend well = into >> efficient systems. The simplest proof for this is the existence of >> kqueue, epoll, completion ports, and eventlets. >=20 > "Unix philosophy" means that Unix is not bound by "Unix philosophy" > (as in we shouldn't be bound by one true way of solving a problem :) >=20 > Yes doing concurrency efficiently can get ugly. But getting the most > performance/efficiency out of _anything_ will get ugly[1]. >=20 >> To optimise for efficiency or for performance (they needn't be the >> same), it is absolutely required that abstractions move away from >> strictly synchronous and external-lock scheduled IO (like group = select >> or per-socket select, etc). >=20 > Fibers in Ruby 1.9 are fairly lightweight and likely the most = efficient > way for Ruby networking concurrency right now (balancing "efficient" = in > both programmer cycles and hardware cycles). >=20 > Performance is "good enough" for most applications, but that's the = trade > off for using Ruby (vs C vs asm). Most of the lack of performance = I've > seen lies in the underlying applications served. I need to dig out the old discussions with ryah when he was working on = flow (a rebuild of Ebb), this was fairly well covered by looking at = performance and efficiency stats from apps based on the async_app.ru = tests i wrote in the Thin repo. Specifically, you can get a measure of = the scheduling efficiency by looking at the request rate for this = example, and 5% overhead is pretty darn low. >=20 >=20 > [1] - Though I've been tempted to write a Linux-only synchronous > clone(2)-based server with sane[2] stack sizes just to see how well a > synchronous model performs against (mentally-)uglier event-based > solutions. >=20 > [2] - I do not consider the 64K stacks enforced by pthreads to be a > sane lower bound. >=20 > --=20 > Eric Wong