[ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

Hi all,

I've started getting Web Sockets going for Rainbows! (git version-only).

If people like it, maybe this can be put into Rack itself (and it'll
have a better name and more exposure).

=== Demo first

Just grab the echo_client.py from pywebsocket here:
  http://pywebsocket.googlecode.com/svn/trunk/src/example/echo_client.py
  (tested with r139)

And run:
$ python echo_client.py -p 9999 -s yhbt.net
$ python echo_client.py -p 9999 -s yhbt.net -m "hello,world"
...

== Code

  http://git.bogomips.org/cgit/lmgtwty.git
  git://git.bogomips.org/lmgtwty.git
  # s/lmgtwty/rainbows/ to get rainbows.git

The only change Rainbows! I made is to expose the client IO object via:

  # if it becomes part of the Rack spec, then s/h/r/ :>
  env["hack.io"] = client

Right now, env["hack.io"] needs to respond to #readpartial and #write
(so it's compatible with core Ruby IO objects and also with
Rainbows::Fiber::IO).

== LMGTWTY code walk-through

The entirety of LMGTWTY was done on a whim way past my normal bedtime.
Feedback on design/code/semantics would be greatly appreciated.

=== Lmgtwty::IO is a wrapper for the "raw" env["hack.io"] object

It responds to #gets and #write while encoding/decoding the
WebSocket framing transparently to the application.

For reading, Lmgtwty::IO#gets() returns a single web socket frame.
So it removes the first and last bytes of (\x00 frame \xff)

For writing, Lmgtwty::IO#write(buf) - writes the contents of +buf+ to
the client socket, transparently framing it with "\x00" and "\xff"
bytes.

=== Lmgtwty::Request is a subclass of Rack::Request

It adds a few helper methods to handle handshaking and using
Lmgtwty::IO.

Handshaking/responses can't be reliably done with the normal Rack
resmost Rack servers/handlers yet because client (and last I checked,
the RFC) currently rely on a hard-coded response where case-sensitivity
and header order matters(!).

Web Sockets does NOT use HTTP chunked encoding, so the normal TeeInput
class in Unicorn/Rainbows! won't work.  Also the rewindable requirement
of "rack.input" gives it unnecessary overhead for long-running requests.

The handshake uses the "raw" env["hack.io"] object for writes,
afterwards using the Lmgtwty::IO-wrapped version of env["hack.io"] is
required for framing/deframing Web Socket messages.

== Caveats

* Web Sockets is still evolving quickly and may change incompatibly...

* iobuffer is a C extension gem dependency, I expect replacing it with
  a pure Ruby Array of Strings with minimal/no performance loss since
  the Strings are expected to be short anyways.

* I've only tested this with the python command-line client.  I
  avoid GUIs as much as possible and using Chrome at this stage
  (without a Vimperator equivalent) would cause me too much pain.

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> Handshaking/responses can't be reliably done with the normal Rack
> resmost Rack servers/handlers yet because client (and last I checked,

Oops, that should read:

  Handshaking/responses can't be reliably done with the normal Rack
  responses with most Rack servers/handlers because clients (and
  last I checked, the RFC) currently rely on a hard-coded response where
  case-sensitivity and header order matters(!).

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Daniel N]

[-- Attachment #1: Type: text/plain, Size: 639 bytes --]

This looks pretty awesome Eric :D

Thanx for writing it

~Daniel

On Sat, Dec 12, 2009 at 8:37 AM, Eric Wong <normalperson@yhbt.net> wrote:

> Eric Wong <normalperson@yhbt.net> wrote:
> > Handshaking/responses can't be reliably done with the normal Rack
> > resmost Rack servers/handlers yet because client (and last I checked,
>
> Oops, that should read:
>
>  Handshaking/responses can't be reliably done with the normal Rack
>   responses with most Rack servers/handlers because clients (and
>   last I checked, the RFC) currently rely on a hard-coded response where
>  case-sensitivity and header order matters(!).
>
> --
> Eric Wong
>

[-- Attachment #2: Type: text/html, Size: 1122 bytes --]

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> For reading, Lmgtwty::IO#gets() returns a single web socket frame.
> So it removes the first and last bytes of (\x00 frame \xff)

Lmgtwty::IO#gets now also understands the binary framing format,
too.  This does not appear to be generated by Chrome or pywebsocket,
but both understand it.

> For writing, Lmgtwty::IO#write(buf) - writes the contents of +buf+ to
> the client socket, transparently framing it with "\x00" and "\xff"
> bytes.

Lmgtwty::IO#write(buf) will now call either Lmgtwty::IO#write_utf8(buf)
or Lmgtwty::IO#write_binary(buf) depending on the encoding of +buf+

write(buf) will attempt to use write_utf8(buf) since that seems to be
the prevalent framing, under 1.9 just make sure +buf+ is of
Encoding::UTF_8.

> === Lmgtwty::Request is a subclass of Rack::Request
> 
> It adds a few helper methods to handle handshaking and using
> Lmgtwty::IO.

I've moved all the logic into a Lmgtwty::WebSocket module, so
users of other Rack::Request subclasses (like Sinatra::Request)
can also get that stuff included as they wish.

> * iobuffer is a C extension gem dependency, I expect replacing it with
>   a pure Ruby Array of Strings with minimal/no performance loss since
>   the Strings are expected to be short anyways.

The iobuffer dependency is gone with no noticeable performance hit.  If
you're writing valid UTF-8 (and encoding strings as such under 1.9),
then performance has actually improved now (especially if you use
write_utf8 directly).

> * I've only tested this with the python command-line client.  I
>   avoid GUIs as much as possible and using Chrome at this stage
>   (without a Vimperator equivalent) would cause me too much pain.

At least now I've read the Chrome WebSockets code.

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> I've started getting Web Sockets going for Rainbows! (git version-only).
> 
> If people like it, maybe this can be put into Rack itself (and it'll
> have a better name and more exposure).

I'm going to call it Sunshowers for now, the old name was too hard
for other folks to remember.

I'm going to make a release of both Rainbows!+Sunshowers later
today, but the git versions should be ready to play with.

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Lakshan Perera]

[-- Attachment #1.1: Type: text/plain, Size: 1232 bytes --]

Hi Eric,

This is awesome! Thanks for coming up something like this, in a short period
of time.
I hope this would be part of Rack, which would enable all Ruby Frameworks to
work effortlessly with WebSockets.

BTW, I found a small fix is needed to get this work correctly with Google
Chrome Beta. As described in Section 3.2
<http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-55#section-3.2>in
IETF's "The Web Socket protocol" resource name(uri) needs to be appended
when constructing the WebSocket URLs. This needs to be adhered in the
WebSocket-Location value of the handshake header.

I have attached a patch which fulfills this requirement.

--
Lakshan

On Mon, Dec 14, 2009 at 2:23 AM, Eric Wong <normalperson@yhbt.net> wrote:

> Eric Wong <normalperson@yhbt.net> wrote:
> > I've started getting Web Sockets going for Rainbows! (git version-only).
> >
> > If people like it, maybe this can be put into Rack itself (and it'll
> > have a better name and more exposure).
>
> I'm going to call it Sunshowers for now, the old name was too hard
> for other folks to remember.
>
> I'm going to make a release of both Rainbows!+Sunshowers later
> today, but the git versions should be ready to play with.
>
> --
> Eric Wong
>

[-- Attachment #1.2: Type: text/html, Size: 1749 bytes --]

[-- Attachment #2: resource_name_fix.diff --]
[-- Type: application/octet-stream, Size: 966 bytes --]

From 9465fc7b00e310e5e7fd85eebc8884a999234396 Mon Sep 17 00:00:00 2001
From: Lakshan <lakshan@web2media.net>
Date: Mon, 14 Dec 2009 05:01:07 +0530
Subject: [PATCH] append the resource name to ws_location 

---
 lib/sunshowers/web_socket.rb |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/lib/sunshowers/web_socket.rb b/lib/sunshowers/web_socket.rb
index c9ceb8d..ad4bce4 100644
--- a/lib/sunshowers/web_socket.rb
+++ b/lib/sunshowers/web_socket.rb
@@ -58,6 +58,11 @@ module Sunshowers
       protocol
     end
 
+    # returns the requested resource name
+    def ws_resource_name
+      @env["REQUEST_URI"]
+    end
+
     # returns the ws://... or wss://... URL
     def ws_location
       s = ws_scheme
@@ -65,6 +70,7 @@ module Sunshowers
       if (s == "wss" && port != 443) || (s == "ws" && port != 80)
         rv << ":#{port}"
       end
+      rv << ws_resource_name #concat the resource name
       rv
     end
 
-- 
1.6.5.2

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

Lakshan Perera <perera.lakshan@gmail.com> wrote:
> Hi Eric,
> 
> This is awesome! Thanks for coming up something like this, in a short period
> of time.
> I hope this would be part of Rack, which would enable all Ruby Frameworks to
> work effortlessly with WebSockets.

Yes, agreed, but the other thing is that Sunshowers can also be used
for clients (without needing extra code), so the gem doesn't depend
on Rack nor Rainbows! at the moment.

> BTW, I found a small fix is needed to get this work correctly with Google
> Chrome Beta. As described in Section 3.2
> <http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-55#section-3.2>in
> IETF's "The Web Socket protocol" resource name(uri) needs to be appended
> when constructing the WebSocket URLs. This needs to be adhered in the
> WebSocket-Location value of the handshake header.
> 
> I have attached a patch which fulfills this requirement.

--- a/lib/sunshowers/web_socket.rb
+++ b/lib/sunshowers/web_socket.rb
@@ -58,6 +58,11 @@ module Sunshowers
       protocol
     end
 
+    # returns the requested resource name
+    def ws_resource_name
+      @env["REQUEST_URI"]
+    end

Thanks Lackshan!

But shouldn't it be (our good friend) PATH_INFO instead of REQUEST_URI?
REQUEST_URI is both:

a) not required by Rack
b) includes QUERY_STRING

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> --- a/lib/sunshowers/web_socket.rb
> +++ b/lib/sunshowers/web_socket.rb
> @@ -58,6 +58,11 @@ module Sunshowers
>        protocol
>      end
>  
> +    # returns the requested resource name
> +    def ws_resource_name
> +      @env["REQUEST_URI"]
> +    end
> 
> Thanks Lackshan!
> 
> But shouldn't it be (our good friend) PATH_INFO instead of REQUEST_URI?
> REQUEST_URI is both:

Nevermind, section 8.1 of revision-66 seems to clarify it to include
QUERY_STRING.  I'll fall back on full_path if REQUEST_URI is not
available, though.  Thanks again!

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[James Tucker]

[-- Attachment #1: Type: text/plain, Size: 1758 bytes --]


On 14 Dec 2009, at 00:23, Lakshan Perera wrote:

> Hi Eric,
> 
> This is awesome! Thanks for coming up something like this, in a short period of time.
> I hope this would be part of Rack, which would enable all Ruby Frameworks to work effortlessly with WebSockets.

I really want to work out an abstraction away from IO instances, #read and #write for this stuff. It's highly coupled, getting in the way of tests, and heavy lifting environments. I have big plans for Rack 2.0 to remove all IO that has not been properly abstracted and decoupled from implementation details, but that's a long way off, mostly due to lack of time and incentive. In the meantime, I can implore you all to take steps in the right direction :-)

> 
> BTW, I found a small fix is needed to get this work correctly with Google Chrome Beta. As described in Section 3.2 in IETF's "The Web Socket protocol" resource name(uri) needs to be appended when constructing the WebSocket URLs. This needs to be adhered in the WebSocket-Location value of the handshake header.
> 
> I have attached a patch which fulfills this requirement.
> 
> --
> Lakshan
> 
> On Mon, Dec 14, 2009 at 2:23 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Eric Wong <normalperson@yhbt.net> wrote:
> > I've started getting Web Sockets going for Rainbows! (git version-only).
> >
> > If people like it, maybe this can be put into Rack itself (and it'll
> > have a better name and more exposure).
> 
> I'm going to call it Sunshowers for now, the old name was too hard
> for other folks to remember.
> 
> I'm going to make a release of both Rainbows!+Sunshowers later
> today, but the git versions should be ready to play with.
> 
> --
> Eric Wong
> 
> <resource_name_fix.diff>


[-- Attachment #2: Type: text/html, Size: 2573 bytes --]

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

James Tucker <jftucker@gmail.com> wrote:
> On 14 Dec 2009, at 00:23, Lakshan Perera wrote:
> > This is awesome! Thanks for coming up something like this, in a
> > short period of time.  I hope this would be part of Rack, which
> > would enable all Ruby Frameworks to work effortlessly with
> > WebSockets.
> 
> I really want to work out an abstraction away from IO instances, #read
> and #write for this stuff. It's highly coupled, getting in the way of
> tests, and heavy lifting environments. I have big plans for Rack 2.0
> to remove all IO that has not been properly abstracted and decoupled
> from implementation details, but that's a long way off, mostly due to
> lack of time and incentive. In the meantime, I can implore you all to
> take steps in the right direction :-)

Huh?  I don't see what the problem with IO instances/semantics is,
especially with the availability of StringIO for testing.  "rack.input"
is clearly specified and works fine as-is IMHO, though the rewindability
requirement does add some unnecessary overhead.

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[James Tucker]

On 14 Dec 2009, at 18:42, Eric Wong wrote:

> James Tucker <jftucker@gmail.com> wrote:
>> On 14 Dec 2009, at 00:23, Lakshan Perera wrote:
>>> This is awesome! Thanks for coming up something like this, in a
>>> short period of time.  I hope this would be part of Rack, which
>>> would enable all Ruby Frameworks to work effortlessly with
>>> WebSockets.
>> 
>> I really want to work out an abstraction away from IO instances, #read
>> and #write for this stuff. It's highly coupled, getting in the way of
>> tests, and heavy lifting environments. I have big plans for Rack 2.0
>> to remove all IO that has not been properly abstracted and decoupled
>> from implementation details, but that's a long way off, mostly due to
>> lack of time and incentive. In the meantime, I can implore you all to
>> take steps in the right direction :-)
> 
> Huh?  I don't see what the problem with IO instances/semantics is,
> especially with the availability of StringIO for testing.  "rack.input"
> is clearly specified and works fine as-is IMHO, though the rewindability
> requirement does add some unnecessary overhead.

I disagree with "works fine". It does not work fine with Thin, in fact, on the contrary, it forces some real ugliness into the system. I also think that StringIO is a really unfortunate thing to have to resort to, as it has so many short comings.

> 
> -- 
> Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

James Tucker <jftucker@gmail.com> wrote:
> On 14 Dec 2009, at 18:42, Eric Wong wrote:
> > James Tucker <jftucker@gmail.com> wrote:
> >> On 14 Dec 2009, at 00:23, Lakshan Perera wrote:
> >>> This is awesome! Thanks for coming up something like this, in a
> >>> short period of time.  I hope this would be part of Rack, which
> >>> would enable all Ruby Frameworks to work effortlessly with
> >>> WebSockets.
> >> 
> >> I really want to work out an abstraction away from IO instances, #read
> >> and #write for this stuff. It's highly coupled, getting in the way of
> >> tests, and heavy lifting environments. I have big plans for Rack 2.0
> >> to remove all IO that has not been properly abstracted and decoupled
> >> from implementation details, but that's a long way off, mostly due to
> >> lack of time and incentive. In the meantime, I can implore you all to
> >> take steps in the right direction :-)
> > 
> > Huh?  I don't see what the problem with IO instances/semantics is,
> > especially with the availability of StringIO for testing.  "rack.input"
> > is clearly specified and works fine as-is IMHO, though the rewindability
> > requirement does add some unnecessary overhead.
> 
> I disagree with "works fine". It does not work fine with Thin, in
> fact, on the contrary, it forces some real ugliness into the system. I
> also think that StringIO is a really unfortunate thing to have to
> resort to, as it has so many short comings.

Mind elaborating?  I do not see what's ugly about it[1].

I haven't used Thin with uploads much, but I do have apps that handle
lots of large uploads (via curl -T to make PUT requests) with Unicorn
and Mongrel before that.

StringIO isn't perfect (can't stat/select/chmod on it), but for
Rack tests it's plenty alright...


[1] - I admit to having a Unix bias here where
      nearly everything is a file, or ought to be :)

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[James Tucker]

On 15 Dec 2009, at 04:37, Eric Wong wrote:

> James Tucker <jftucker@gmail.com> wrote:
>> On 14 Dec 2009, at 18:42, Eric Wong wrote:
>>> James Tucker <jftucker@gmail.com> wrote:
>>>> On 14 Dec 2009, at 00:23, Lakshan Perera wrote:
>>>>> This is awesome! Thanks for coming up something like this, in a
>>>>> short period of time.  I hope this would be part of Rack, which
>>>>> would enable all Ruby Frameworks to work effortlessly with
>>>>> WebSockets.
>>>> 
>>>> I really want to work out an abstraction away from IO instances, #read
>>>> and #write for this stuff. It's highly coupled, getting in the way of
>>>> tests, and heavy lifting environments. I have big plans for Rack 2.0
>>>> to remove all IO that has not been properly abstracted and decoupled
>>>> from implementation details, but that's a long way off, mostly due to
>>>> lack of time and incentive. In the meantime, I can implore you all to
>>>> take steps in the right direction :-)
>>> 
>>> Huh?  I don't see what the problem with IO instances/semantics is,
>>> especially with the availability of StringIO for testing.  "rack.input"
>>> is clearly specified and works fine as-is IMHO, though the rewindability
>>> requirement does add some unnecessary overhead.
>> 
>> I disagree with "works fine". It does not work fine with Thin, in
>> fact, on the contrary, it forces some real ugliness into the system. I
>> also think that StringIO is a really unfortunate thing to have to
>> resort to, as it has so many short comings.
> 
> Mind elaborating?  I do not see what's ugly about it[1].

Sure. It forces authors to several (bad) APIs options for handling complex IO scenarios.

1. Buffer the entire request and then republish to the application
2. Pass forward the raw IO

Both of these options put significant restrictions on the ability to do complex (and efficient) scheduling.

1. Prevents the ability for the request to begin processing asynchronously to the receipt of the request body without a custom future of some kind. Adding a future is not a bad solution, but, it's so far out from everything else at the moment because the rack spec says use an IO. The StringIO could be used as a future, however, the reality is there's no way for the server to tell the app that the future is ready for consumption. This is a total lack of proper encapsulation that would be required for improved scheduling and IO handling.

2. This approach forces the application developer to be responsible for buffering and scheduling - something which application authors typically know nothing about. The result is common use of slurp approaches, and sometimes even multiple reads of the same buffer.

> I haven't used Thin with uploads much, but I do have apps that handle
> lots of large uploads (via curl -T to make PUT requests) with Unicorn
> and Mongrel before that.
> 
> StringIO isn't perfect (can't stat/select/chmod on it), but for
> Rack tests it's plenty alright...

Sure, for trivial, low efficiency, slurping, synchronously scheduled work, it's fine, and I do use it for that quite frequently.

> [1] - I admit to having a Unix bias here where
>      nearly everything is a file, or ought to be :)

We're not stream processing here. It's a request/response cycle. And they aren't files, they're sockets.

Unix philosophy is good for the shell, but it doesn't extend well into efficient systems. The simplest proof for this is the existence of kqueue, epoll, completion ports, and eventlets.

To optimise for efficiency or for performance (they needn't be the same), it is absolutely required that abstractions move away from strictly synchronous and external-lock scheduled IO (like group select or per-socket select, etc).

> 
> -- 
> Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

James Tucker <jftucker@gmail.com> wrote:
> On 15 Dec 2009, at 04:37, Eric Wong wrote:
> > James Tucker <jftucker@gmail.com> wrote:
> >> On 14 Dec 2009, at 18:42, Eric Wong wrote:
> >>> James Tucker <jftucker@gmail.com> wrote:
> >>>> I really want to work out an abstraction away from IO instances, #read
> >>>> and #write for this stuff. It's highly coupled, getting in the way of
> >>>> tests, and heavy lifting environments. I have big plans for Rack 2.0
> >>>> to remove all IO that has not been properly abstracted and decoupled
> >>>> from implementation details, but that's a long way off, mostly due to
> >>>> lack of time and incentive. In the meantime, I can implore you all to
> >>>> take steps in the right direction :-)
> >>> 
> >>> Huh?  I don't see what the problem with IO instances/semantics is,
> >>> especially with the availability of StringIO for testing.  "rack.input"
> >>> is clearly specified and works fine as-is IMHO, though the rewindability
> >>> requirement does add some unnecessary overhead.
> >> 
> >> I disagree with "works fine". It does not work fine with Thin, in
> >> fact, on the contrary, it forces some real ugliness into the system. I
> >> also think that StringIO is a really unfortunate thing to have to
> >> resort to, as it has so many short comings.
> > 
> > Mind elaborating?  I do not see what's ugly about it[1].

> Sure. It forces authors to several (bad) APIs options for handling
> complex IO scenarios.
> 
> 1. Buffer the entire request and then republish to the application
> 2. Pass forward the raw IO

3. stream input with a rewindable backing store like TeeInput:
   http://unicorn.bogomips.org/Unicorn/TeeInput.html
   This is already supported by several (but not all) Rainbows!
   concurrency models: http://rainbows.rubyforge.org/Summary.html

I'll need to use Fibers to get this working with Rev and EventMachine,
(the FiberSpawn/FiberPool models are select-based), but exposing an
outwardly synchronous model is far easier for application developers to
grok.

> > [1] - I admit to having a Unix bias here where
> >      nearly everything is a file, or ought to be :)

> We're not stream processing here. It's a request/response cycle. And
> they aren't files, they're sockets.

We can be stream processing, see examples/echo.ru in the Unicorn source
which streams input to the output body:

  http://git.bogomips.org/cgit/unicorn.git/tree/examples/echo.ru

> Unix philosophy is good for the shell, but it doesn't extend well into
> efficient systems. The simplest proof for this is the existence of
> kqueue, epoll, completion ports, and eventlets.

"Unix philosophy" means that Unix is not bound by "Unix philosophy"
(as in we shouldn't be bound by one true way of solving a problem :)

Yes doing concurrency efficiently can get ugly.  But getting the most
performance/efficiency out of _anything_ will get ugly[1].

> To optimise for efficiency or for performance (they needn't be the
> same), it is absolutely required that abstractions move away from
> strictly synchronous and external-lock scheduled IO (like group select
> or per-socket select, etc).

Fibers in Ruby 1.9 are fairly lightweight and likely the most efficient
way for Ruby networking concurrency right now (balancing "efficient" in
both programmer cycles and hardware cycles).

Performance is "good enough" for most applications, but that's the trade
off for using Ruby (vs C vs asm).  Most of the lack of performance I've
seen lies in the underlying applications served.


[1] - Though I've been tempted to write a Linux-only synchronous
clone(2)-based server with sane[2] stack sizes just to see how well a
synchronous model performs against (mentally-)uglier event-based
solutions.

[2] - I do not consider the 64K stacks enforced by pthreads to be a
sane lower bound.

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[James Tucker]

On 15 Dec 2009, at 21:32, Eric Wong wrote:

> James Tucker <jftucker@gmail.com> wrote:
>> On 15 Dec 2009, at 04:37, Eric Wong wrote:
>>> James Tucker <jftucker@gmail.com> wrote:
>>>> On 14 Dec 2009, at 18:42, Eric Wong wrote:
>>>>> James Tucker <jftucker@gmail.com> wrote:
>>>>>> I really want to work out an abstraction away from IO instances, #read
>>>>>> and #write for this stuff. It's highly coupled, getting in the way of
>>>>>> tests, and heavy lifting environments. I have big plans for Rack 2.0
>>>>>> to remove all IO that has not been properly abstracted and decoupled
>>>>>> from implementation details, but that's a long way off, mostly due to
>>>>>> lack of time and incentive. In the meantime, I can implore you all to
>>>>>> take steps in the right direction :-)
>>>>> 
>>>>> Huh?  I don't see what the problem with IO instances/semantics is,
>>>>> especially with the availability of StringIO for testing.  "rack.input"
>>>>> is clearly specified and works fine as-is IMHO, though the rewindability
>>>>> requirement does add some unnecessary overhead.
>>>> 
>>>> I disagree with "works fine". It does not work fine with Thin, in
>>>> fact, on the contrary, it forces some real ugliness into the system. I
>>>> also think that StringIO is a really unfortunate thing to have to
>>>> resort to, as it has so many short comings.
>>> 
>>> Mind elaborating?  I do not see what's ugly about it[1].
> 
>> Sure. It forces authors to several (bad) APIs options for handling
>> complex IO scenarios.
>> 
>> 1. Buffer the entire request and then republish to the application
>> 2. Pass forward the raw IO
> 
> 3. stream input with a rewindable backing store like TeeInput:
>   http://unicorn.bogomips.org/Unicorn/TeeInput.html
>   This is already supported by several (but not all) Rainbows!
>   concurrency models: http://rainbows.rubyforge.org/Summary.html
> 
> I'll need to use Fibers to get this working with Rev and EventMachine,
> (the FiberSpawn/FiberPool models are select-based), but exposing an
> outwardly synchronous model is far easier for application developers to
> grok.

That's API dependent. And exactly my point, you're forcing them to use stack based concurrency models. Doing this means one can never escape the weight of stack storage, and moreover, stack wrangling will never be as fast as object level state containers. This is not a fact that can be escaped.

> 
>>> [1] - I admit to having a Unix bias here where
>>>     nearly everything is a file, or ought to be :)
> 
>> We're not stream processing here. It's a request/response cycle. And
>> they aren't files, they're sockets.
> 
> We can be stream processing, see examples/echo.ru in the Unicorn source
> which streams input to the output body:
> 
>  http://git.bogomips.org/cgit/unicorn.git/tree/examples/echo.ru
> 
>> Unix philosophy is good for the shell, but it doesn't extend well into
>> efficient systems. The simplest proof for this is the existence of
>> kqueue, epoll, completion ports, and eventlets.
> 
> "Unix philosophy" means that Unix is not bound by "Unix philosophy"
> (as in we shouldn't be bound by one true way of solving a problem :)
> 
> Yes doing concurrency efficiently can get ugly.  But getting the most
> performance/efficiency out of _anything_ will get ugly[1].
> 
>> To optimise for efficiency or for performance (they needn't be the
>> same), it is absolutely required that abstractions move away from
>> strictly synchronous and external-lock scheduled IO (like group select
>> or per-socket select, etc).
> 
> Fibers in Ruby 1.9 are fairly lightweight and likely the most efficient
> way for Ruby networking concurrency right now (balancing "efficient" in
> both programmer cycles and hardware cycles).
> 
> Performance is "good enough" for most applications, but that's the trade
> off for using Ruby (vs C vs asm).  Most of the lack of performance I've
> seen lies in the underlying applications served.
> 
> 
> [1] - Though I've been tempted to write a Linux-only synchronous
> clone(2)-based server with sane[2] stack sizes just to see how well a
> synchronous model performs against (mentally-)uglier event-based
> solutions.
> 
> [2] - I do not consider the 64K stacks enforced by pthreads to be a
> sane lower bound.
> 
> -- 
> Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[James Tucker]

On 15 Dec 2009, at 21:32, Eric Wong wrote:

> James Tucker <jftucker@gmail.com> wrote:
>> On 15 Dec 2009, at 04:37, Eric Wong wrote:
>>> James Tucker <jftucker@gmail.com> wrote:
>>>> On 14 Dec 2009, at 18:42, Eric Wong wrote:
>>>>> James Tucker <jftucker@gmail.com> wrote:
>>>>>> I really want to work out an abstraction away from IO instances, #read
>>>>>> and #write for this stuff. It's highly coupled, getting in the way of
>>>>>> tests, and heavy lifting environments. I have big plans for Rack 2.0
>>>>>> to remove all IO that has not been properly abstracted and decoupled
>>>>>> from implementation details, but that's a long way off, mostly due to
>>>>>> lack of time and incentive. In the meantime, I can implore you all to
>>>>>> take steps in the right direction :-)
>>>>> 
>>>>> Huh?  I don't see what the problem with IO instances/semantics is,
>>>>> especially with the availability of StringIO for testing.  "rack.input"
>>>>> is clearly specified and works fine as-is IMHO, though the rewindability
>>>>> requirement does add some unnecessary overhead.
>>>> 
>>>> I disagree with "works fine". It does not work fine with Thin, in
>>>> fact, on the contrary, it forces some real ugliness into the system. I
>>>> also think that StringIO is a really unfortunate thing to have to
>>>> resort to, as it has so many short comings.
>>> 
>>> Mind elaborating?  I do not see what's ugly about it[1].
> 
>> Sure. It forces authors to several (bad) APIs options for handling
>> complex IO scenarios.
>> 
>> 1. Buffer the entire request and then republish to the application
>> 2. Pass forward the raw IO
> 
> 3. stream input with a rewindable backing store like TeeInput:
>   http://unicorn.bogomips.org/Unicorn/TeeInput.html
>   This is already supported by several (but not all) Rainbows!
>   concurrency models: http://rainbows.rubyforge.org/Summary.html
> 
> I'll need to use Fibers to get this working with Rev and EventMachine,
> (the FiberSpawn/FiberPool models are select-based), but exposing an
> outwardly synchronous model is far easier for application developers to
> grok.
> 
>>> [1] - I admit to having a Unix bias here where
>>>     nearly everything is a file, or ought to be :)
> 
>> We're not stream processing here. It's a request/response cycle. And
>> they aren't files, they're sockets.
> 
> We can be stream processing, see examples/echo.ru in the Unicorn source
> which streams input to the output body:
> 
>  http://git.bogomips.org/cgit/unicorn.git/tree/examples/echo.ru
> 
>> Unix philosophy is good for the shell, but it doesn't extend well into
>> efficient systems. The simplest proof for this is the existence of
>> kqueue, epoll, completion ports, and eventlets.
> 
> "Unix philosophy" means that Unix is not bound by "Unix philosophy"
> (as in we shouldn't be bound by one true way of solving a problem :)
> 
> Yes doing concurrency efficiently can get ugly.  But getting the most
> performance/efficiency out of _anything_ will get ugly[1].
> 
>> To optimise for efficiency or for performance (they needn't be the
>> same), it is absolutely required that abstractions move away from
>> strictly synchronous and external-lock scheduled IO (like group select
>> or per-socket select, etc).
> 
> Fibers in Ruby 1.9 are fairly lightweight and likely the most efficient
> way for Ruby networking concurrency right now (balancing "efficient" in
> both programmer cycles and hardware cycles).
> 
> Performance is "good enough" for most applications, but that's the trade
> off for using Ruby (vs C vs asm).  Most of the lack of performance I've
> seen lies in the underlying applications served.

I need to dig out the old discussions with ryah when he was working on flow (a rebuild of Ebb), this was fairly well covered by looking at performance and efficiency stats from apps based on the async_app.ru tests i wrote in the Thin repo. Specifically, you can get a measure of the scheduling efficiency by looking at the request rate for this example, and 5% overhead is pretty darn low.

> 
> 
> [1] - Though I've been tempted to write a Linux-only synchronous
> clone(2)-based server with sane[2] stack sizes just to see how well a
> synchronous model performs against (mentally-)uglier event-based
> solutions.
> 
> [2] - I do not consider the 64K stacks enforced by pthreads to be a
> sane lower bound.
> 
> -- 
> Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

James Tucker <jftucker@gmail.com> wrote:
> On 15 Dec 2009, at 21:32, Eric Wong wrote:
> >> On 15 Dec 2009, at 04:37, Eric Wong wrote:
> >>> James Tucker <jftucker@gmail.com> wrote:
> >>>> On 14 Dec 2009, at 18:42, Eric Wong wrote:
> >>>>> James Tucker <jftucker@gmail.com> wrote:
> >>>>>> I really want to work out an abstraction away from IO instances, #read
> >>>>>> and #write for this stuff. It's highly coupled, getting in the way of
> >>>>>> tests, and heavy lifting environments. I have big plans for Rack 2.0
> >>>>>> to remove all IO that has not been properly abstracted and decoupled
> >>>>>> from implementation details, but that's a long way off, mostly due to
> >>>>>> lack of time and incentive. In the meantime, I can implore you all to
> >>>>>> take steps in the right direction :-)
> >>>>> 
> >>>>> Huh?  I don't see what the problem with IO instances/semantics is,
> >>>>> especially with the availability of StringIO for testing.  "rack.input"
> >>>>> is clearly specified and works fine as-is IMHO, though the rewindability
> >>>>> requirement does add some unnecessary overhead.
> >>>> 
> >>>> I disagree with "works fine". It does not work fine with Thin, in
> >>>> fact, on the contrary, it forces some real ugliness into the system. I
> >>>> also think that StringIO is a really unfortunate thing to have to
> >>>> resort to, as it has so many short comings.
> >>> 
> >>> Mind elaborating?  I do not see what's ugly about it[1].
> > 
> >> Sure. It forces authors to several (bad) APIs options for handling
> >> complex IO scenarios.
> >> 
> >> 1. Buffer the entire request and then republish to the application
> >> 2. Pass forward the raw IO
> > 
> > 3. stream input with a rewindable backing store like TeeInput:
> >   http://unicorn.bogomips.org/Unicorn/TeeInput.html
> >   This is already supported by several (but not all) Rainbows!
> >   concurrency models: http://rainbows.rubyforge.org/Summary.html
> > 
> > I'll need to use Fibers to get this working with Rev and EventMachine,
> > (the FiberSpawn/FiberPool models are select-based), but exposing an
> > outwardly synchronous model is far easier for application developers to
> > grok.
> 
> That's API dependent. And exactly my point, you're forcing them to use
> stack based concurrency models. Doing this means one can never escape
> the weight of stack storage, and moreover, stack wrangling will never
> be as fast as object level state containers. This is not a fact that
> can be escaped.

Everything is API dependent.  TeeInput just conforms to what
Rack::Lint::InputWrapper allows.

Of course stack-based concurrency models will always be slower on the
machine than state machine + event-based models.

Ruby will also always be slower than a lot of other
less-pleasant-to-write languages out there.  It's a tradeoff we make as
Rubyists.  I believe most developers find things like
fibers/actors/threads easier to work with/maintain than
events+callbacks, so Sunshowers got support for linear programming
models first.

With a env["rack.io"] that exposes an IO-ish object, then nothing
prevents developers from using EM.attach or Rev::IO.new on it, either.
If people like that, I'd be happy to include it EM and Rev-based
concurrency models for Rainbows! and add support for it in Sunshowers.

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[James Tucker]

On 16 Dec 2009, at 22:14, Eric Wong wrote:

> James Tucker <jftucker@gmail.com> wrote:
>> On 15 Dec 2009, at 21:32, Eric Wong wrote:
>>>> On 15 Dec 2009, at 04:37, Eric Wong wrote:
>>>>> James Tucker <jftucker@gmail.com> wrote:
>>>>>> On 14 Dec 2009, at 18:42, Eric Wong wrote:
>>>>>>> James Tucker <jftucker@gmail.com> wrote:
>>>>>>>> I really want to work out an abstraction away from IO instances, #read
>>>>>>>> and #write for this stuff. It's highly coupled, getting in the way of
>>>>>>>> tests, and heavy lifting environments. I have big plans for Rack 2.0
>>>>>>>> to remove all IO that has not been properly abstracted and decoupled
>>>>>>>> from implementation details, but that's a long way off, mostly due to
>>>>>>>> lack of time and incentive. In the meantime, I can implore you all to
>>>>>>>> take steps in the right direction :-)
>>>>>>> 
>>>>>>> Huh?  I don't see what the problem with IO instances/semantics is,
>>>>>>> especially with the availability of StringIO for testing.  "rack.input"
>>>>>>> is clearly specified and works fine as-is IMHO, though the rewindability
>>>>>>> requirement does add some unnecessary overhead.
>>>>>> 
>>>>>> I disagree with "works fine". It does not work fine with Thin, in
>>>>>> fact, on the contrary, it forces some real ugliness into the system. I
>>>>>> also think that StringIO is a really unfortunate thing to have to
>>>>>> resort to, as it has so many short comings.
>>>>> 
>>>>> Mind elaborating?  I do not see what's ugly about it[1].
>>> 
>>>> Sure. It forces authors to several (bad) APIs options for handling
>>>> complex IO scenarios.
>>>> 
>>>> 1. Buffer the entire request and then republish to the application
>>>> 2. Pass forward the raw IO
>>> 
>>> 3. stream input with a rewindable backing store like TeeInput:
>>>  http://unicorn.bogomips.org/Unicorn/TeeInput.html
>>>  This is already supported by several (but not all) Rainbows!
>>>  concurrency models: http://rainbows.rubyforge.org/Summary.html
>>> 
>>> I'll need to use Fibers to get this working with Rev and EventMachine,
>>> (the FiberSpawn/FiberPool models are select-based), but exposing an
>>> outwardly synchronous model is far easier for application developers to
>>> grok.
>> 
>> That's API dependent. And exactly my point, you're forcing them to use
>> stack based concurrency models. Doing this means one can never escape
>> the weight of stack storage, and moreover, stack wrangling will never
>> be as fast as object level state containers. This is not a fact that
>> can be escaped.
> 
> Everything is API dependent.  TeeInput just conforms to what
> Rack::Lint::InputWrapper allows.

Quite right and the point I'm simply trying to make, is that there are APIs that can work with all concurrency models. Moreover, it is trivial to turn an asynchronous api into a synchronous api, by contrast it is almost impossible to do the reverse. This is why my planned changes to rack are so far from becoming a reality at this time - as I have to rewrite the entire stack. By contrast, supporting rack 1.0 protocoled apps from the vapour 2.0 api, whatever it is, is absolutely trivial.

> 
> Of course stack-based concurrency models will always be slower on the
> machine than state machine + event-based models.
> 
> Ruby will also always be slower than a lot of other
> less-pleasant-to-write languages out there.  It's a tradeoff we make as
> Rubyists.  I believe most developers find things like
> fibers/actors/threads easier to work with/maintain than
> events+callbacks, so Sunshowers got support for linear programming
> models first.

Agreed. I'm not saying "must", but I would like to request that if you're going to try and support asynchronous models, please do it for real, rather than hacking it in the way we're having to in rack / thin / flow / etc. The more hacks we end up with, the more people get the impression that this is "the right way". As it is, the lack of protocol - app layer separation in for example, eventmachine applications is appalling, and something from which the community may never recover. The same kind of history is about to repeat itself in the async + long running cycle web areas - if it hasn't already gone past the point of no return.

Of course, if it seems like too much work, please do just ignore my request, maybe I'm an ideologist, or maybe I need to formalise my thoughts on paper before they'll really have an impact, but I have experience that tells me that async apis needn't be ugly, or hard, or slower to write - indeed javascript and C programmers use them all the time, unwittingly. One part of the failure in ruby is the implicit temptation to "solve" these "problems" by just "slapping in a few blocks" or the like. Proper OO design does a much better job, and of late the favoured patterns are most definitely channels, queues, and closure-less callbacks - but most of this stuff is in real use in commercial code, as there is no high scale high efficiency open source application server in ruby at this time (that i know of). I do not believe this is due to language limitations, but simply lack of demand and developer time.

> With a env["rack.io"] that exposes an IO-ish object, then nothing
> prevents developers from using EM.attach or Rev::IO.new on it, either.

Depending of course, one whether or not that's even a useful thing to do...

request -> app -> attach | release is required here, otherwise the attach was useless

Contrary to a sync approach, this now means we're back in hack-town to push our state somewhere other than the stack, then release the reactor (in a reactor pattern) to actually complete any more IO.

Of course, if we passed a channel, with a correctly restricted API, all these issues go away. It's down to the server to schedule IO (big tick here), and the app just consumes a callback based API in a standard manner.

Synchronous apps can simply drain the channel in a synchronous (busy wait) manner, or have their callbacks implicitly and immediately scheduled. Implicit and immediate scheduling of callbacks is the simplest common approach, leaving only a call that might be considered slightly excess in a "pure synchronous" environment, but really ends up being akin to #read and such.

It is only through proper abstraction that you can make it easy to introduce or maintain concepts such as zero copy, wait free blocking, demand based scheduling, and so on and so forth. I do realise that examples of more complex scheduling and IO patterns are for the most part limited to kernels, academia and telecomms at present, but, funnily enough, I think this has a lot to do with the restrictive APIs that more common patterns expose - I've mentioned already that it's nearly impossible to retro-fit these things.

> If people like that, I'd be happy to include it EM and Rev-based
> concurrency models for Rainbows! and add support for it in Sunshowers.

Like I say, I could just be too much of an efficiency ideologist. If you don't have the demand / desire for it, then you may not have a good reason to put the effort in - certainly there's nothing else in the middle-ground space that you're filling presently, so I don't think you're "losing a significant market" by not doing so. As it is, I know of a couple of handfuls of apps either in production or close to production doing relatively high load async web work in ruby, and those folks seem to have done alright with the nasty hack that is the current async.callback and deferrable body apis. Bigger deterrents to this area actually exist outside of the web server domain itself, as the 1.2+ Thin api works just fine, by contrast, there's no ORM or process framework for doing async work well, and the closest thing there really is to a "nice" async api in ruby open source at present is the sequel monkey patches that Aman published iirc with em-mysql.

P.S. it's late, so i apologise if this is too wordy/vague.

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[Eric Wong]

James Tucker <jftucker@gmail.com> wrote:
<snip>
> Agreed. I'm not saying "must", but I would like to request that if
> you're going to try and support asynchronous models, please do it for
> real, rather than hacking it in the way we're having to in rack / thin
> / flow / etc. The more hacks we end up with, the more people get the
> impression that this is "the right way". As it is, the lack of
> protocol - app layer separation in for example, eventmachine
> applications is appalling, and something from which the community may
> never recover. The same kind of history is about to repeat itself in
> the async + long running cycle web areas - if it hasn't already gone
> past the point of no return.

I'm glad I'm not alone in feeling the current async add-ons in Rack
servers are very hacky.  I'll leave async callback support out of
Sunshowers until we can have an async spec that's as elegant the
rest of Rack.

> Like I say, I could just be too much of an efficiency ideologist. If
> you don't have the demand / desire for it, then you may not have a
> good reason to put the effort in - certainly there's nothing else in
> the middle-ground space that you're filling presently, so I don't
> think you're "losing a significant market" by not doing so. As it is,
> I know of a couple of handfuls of apps either in production or close
> to production doing relatively high load async web work in ruby, and
> those folks seem to have done alright with the nasty hack that is the
> current async.callback and deferrable body apis. Bigger deterrents to
> this area actually exist outside of the web server domain itself, as
> the 1.2+ Thin api works just fine, by contrast, there's no ORM or
> process framework for doing async work well, and the closest thing
> there really is to a "nice" async api in ruby open source at present
> is the sequel monkey patches that Aman published iirc with em-mysql.

Heh, I don't know of a single public site using Rainbows! (in any form)
nor Sunshowers in production, yet.  It's winter right now and probably
not the season for Rainbows! and Sunshowers :)

> P.S. it's late, so i apologise if this is too wordy/vague.

Nope, I pretty agree with everything you said and will look forward to a
Rack 2.0 spec that covers async nicely.  Meanwhile, I'll keep the focus
on 1.9 Fibers since they're largely "good enough".

-- 
Eric Wong

Re: [ANN/RFC] LMGTWTY - Web Sockets for Rack+Rainbows!

[James Tucker]

On 17 Dec 2009, at 08:47, Eric Wong wrote:

> James Tucker <jftucker@gmail.com> wrote:
> <snip>
>> Agreed. I'm not saying "must", but I would like to request that if
>> you're going to try and support asynchronous models, please do it for
>> real, rather than hacking it in the way we're having to in rack / thin
>> / flow / etc. The more hacks we end up with, the more people get the
>> impression that this is "the right way". As it is, the lack of
>> protocol - app layer separation in for example, eventmachine
>> applications is appalling, and something from which the community may
>> never recover. The same kind of history is about to repeat itself in
>> the async + long running cycle web areas - if it hasn't already gone
>> past the point of no return.
> 
> I'm glad I'm not alone in feeling the current async add-ons in Rack
> servers are very hacky.  I'll leave async callback support out of
> Sunshowers until we can have an async spec that's as elegant the
> rest of Rack.

Yup, this is part of the reason I refused to release my Thin patches for such a long time. They're not graceful at all. Unfortunately, after some of the heavy lifters had been using it in production for a while, and it was becoming a core component of some of the nanite infrastructure, the amount of noise it was causing in my inbox overrode my desire to hold back the unfortunate API from the public as a whole. It's getting an increasing amount of attention, for better or worse at the moment.

>> Like I say, I could just be too much of an efficiency ideologist. If
>> you don't have the demand / desire for it, then you may not have a
>> good reason to put the effort in - certainly there's nothing else in
>> the middle-ground space that you're filling presently, so I don't
>> think you're "losing a significant market" by not doing so. As it is,
>> I know of a couple of handfuls of apps either in production or close
>> to production doing relatively high load async web work in ruby, and
>> those folks seem to have done alright with the nasty hack that is the
>> current async.callback and deferrable body apis. Bigger deterrents to
>> this area actually exist outside of the web server domain itself, as
>> the 1.2+ Thin api works just fine, by contrast, there's no ORM or
>> process framework for doing async work well, and the closest thing
>> there really is to a "nice" async api in ruby open source at present
>> is the sequel monkey patches that Aman published iirc with em-mysql.
> 
> Heh, I don't know of a single public site using Rainbows! (in any form)
> nor Sunshowers in production, yet.  It's winter right now and probably
> not the season for Rainbows! and Sunshowers :)

:D

I don't know if you follow the Thin mailinglist, but there's a discussion relating to some of this stuff going on over there that you might find useful to read through, just from the point of view of how general community members tend to approach the idea of async, and the common misconceptions etc. My really long term goal is to get the APIs down to something simple enough that these issues don't come up, but part of this is also the requirement for a new breed of framework that's not built around inherently synchronous ORM systems.

>> P.S. it's late, so i apologise if this is too wordy/vague.
> 
> Nope, I pretty agree with everything you said and will look forward to a
> Rack 2.0 spec that covers async nicely.  Meanwhile, I'll keep the focus
> on 1.9 Fibers since they're largely "good enough".

Absolutely, I know that oldmoe has had some great success using them in NeverBlock, and who knows, if ruby can be made to be more efficient at stack wrangling one day, then maybe they could defeat the async approach. Certainly for the foreseeable decades, that's not going to happen though :)

I'll be sure to include you in any future discussions on the APIs discussed.

> 
> -- 
> Eric Wong