Middleware and post-request processing

Middleware and post-request processing

[ghazel]

It seems to me that Rack is in need of a new post-response stage of
processing. This stage would occur after the response is fully written
and the client is unblocked, and before the next request is processed.

Similar to what OobGC ( http://bogomips.org/unicorn.git/tree/lib/unicorn/oob_gc.rb#n59
) accomplishes, it is sometimes useful to perform additional
operations after the response is written without blocking the client.
For example, the Oink middleware logs statistics about the request,
but blocks the response since it has no ability not to: (
https://github.com/noahd1/oink/blob/4158d71bc9150f011072b2c6eefe73c720a78d46/lib/oink/middleware.rb#L16
). This processing takes time, and needlessly delays the response.

This proposal would entail something like a single function which is
called on each middleware after the response is written to the client
and the socket is closed (depending on the server implementation). For
servers which have no ability to not block the client or delay further
requests the function should still be called, and the impact would be
similar to the behavior today.

Thoughts?

Re: Middleware and post-request processing

[Eric Wong]

ghazel <ghazel@gmail.com> wrote:
> It seems to me that Rack is in need of a new post-response stage of
> processing. This stage would occur after the response is fully written
> and the client is unblocked, and before the next request is processed.

I remember Aaron Patterson started thinking/blogging about it, at least.
search: "rack api awkward"

> Similar to what OobGC ( http://bogomips.org/unicorn.git/tree/lib/unicorn/oob_gc.rb#n59
> ) accomplishes, it is sometimes useful to perform additional
> operations after the response is written without blocking the client.

I should note OobGC used to (really) be middleware, but I had to rewrite
it (as a Unicorn module) because Unicorn also needs to support
middleware that captures the block passed to body#each and reuses the
block in body#close.

> This proposal would entail something like a single function which is
> called on each middleware after the response is written to the client
> and the socket is closed (depending on the server implementation). For
> servers which have no ability to not block the client or delay further
> requests the function should still be called, and the impact would be
> similar to the behavior today.

Any new post-response stage should give information about whether a
client is persistent or not (and maybe even if it started pipelining the
next request).

It would be great to have a standardized API that makes logging whether
a connection is keepalive/pipelined, but Rack doesn't get into that
right now.


As an author of Rack servers, I'll continue the my conservative personal
policy of not implementing new APIs unless:

1) it's added to the Rack spec, or
2) another widely-used server implements it (e.g. Thin
   with the async/deferred? stuff)

-- 
Eric Wong

Re: Middleware and post-request processing

[George]

On Jun 3, 7:23 pm, ghazel <gha...@gmail.com> wrote:
> It seems to me that Rack is in need of a new post-response stage of
> processing. This stage would occur after the response is fully written
> and the client is unblocked, and before the next request is processed.
>
> Similar to what OobGC (http://bogomips.org/unicorn.git/tree/lib/unicorn/oob_gc.rb#n59
> ) accomplishes, it is sometimes useful to perform additional
> operations after the response is written without blocking the client.
> For example, the Oink middleware logs statistics about the request,
> but blocks the response since it has no ability not to: (https://github.com/noahd1/oink/blob/4158d71bc9150f011072b2c6eefe73c72...
> ). This processing takes time, and needlessly delays the response.
>
> This proposal would entail something like a single function which is
> called on each middleware after the response is written to the client
> and the socket is closed (depending on the server implementation). For
> servers which have no ability to not block the client or delay further
> requests the function should still be called, and the impact would be
> similar to the behavior today.
>
> Thoughts?

I'm very much in favor of adding such a hook also.

I wrote a gem which does it for a bunch of common web servers:
http://github.com/oggy/rack_after_reply . It adds a list of callback
functions which just get added to the rack environment. I like this
interface in that keeps the callbacks request-local, rather than just
exposing a global hook. Beyond that, I'm totally open to suggestions.

Of course I'd love it if web server authors would provide such a hook
natively, or even maintain the adapter for their server in this gem.

George

Re: Middleware and post-request processing

[James Tucker]

On Jun 3, 2011, at 4:23 PM, ghazel wrote:

> It seems to me that Rack is in need of a new post-response stage of
> processing. This stage would occur after the response is fully written
> and the client is unblocked, and before the next request is processed.
> 
> Similar to what OobGC ( http://bogomips.org/unicorn.git/tree/lib/unicorn/oob_gc.rb#n59
> ) accomplishes, it is sometimes useful to perform additional
> operations after the response is written without blocking the client.
> For example, the Oink middleware logs statistics about the request,
> but blocks the response since it has no ability not to: (
> https://github.com/noahd1/oink/blob/4158d71bc9150f011072b2c6eefe73c720a78d46/lib/oink/middleware.rb#L16
> ). This processing takes time, and needlessly delays the response.
> 
> This proposal would entail something like a single function which is
> called on each middleware after the response is written to the client
> and the socket is closed (depending on the server implementation). For
> servers which have no ability to not block the client or delay further
> requests the function should still be called, and the impact would be
> similar to the behavior today.
> 
> Thoughts?


The problem is, this isn't simple. Different servers have different scheduling mechanisms, and deferred operations specifications reach into scheduling in a horrible way.

Should these run linearly? Should they be able to be pooled if env['rack.multithreaded']. In that case should they receive the same number of workers as the main request/response pool? Should they work out of the same pool?

And that's just some basics with threads...

You can quite easily handle this on your own in middleware or servers a number of ways today, without introducing either far reaching / extensive specs or incomplete restrictions that parallel some we already have (like stack based control).

Re: Middleware and post-request processing

[ghazel]

[-- Attachment #1: Type: text/plain, Size: 2431 bytes --]

On Tue, Jun 7, 2011 at 10:20 AM, James Tucker <jftucker@gmail.com> wrote:

>
> On Jun 3, 2011, at 4:23 PM, ghazel wrote:
>
> > It seems to me that Rack is in need of a new post-response stage of
> > processing. This stage would occur after the response is fully written
> > and the client is unblocked, and before the next request is processed.
> >
> > Similar to what OobGC (
> http://bogomips.org/unicorn.git/tree/lib/unicorn/oob_gc.rb#n59
> > ) accomplishes, it is sometimes useful to perform additional
> > operations after the response is written without blocking the client.
> > For example, the Oink middleware logs statistics about the request,
> > but blocks the response since it has no ability not to: (
> >
> https://github.com/noahd1/oink/blob/4158d71bc9150f011072b2c6eefe73c720a78d46/lib/oink/middleware.rb#L16
> > ). This processing takes time, and needlessly delays the response.
> >
> > This proposal would entail something like a single function which is
> > called on each middleware after the response is written to the client
> > and the socket is closed (depending on the server implementation). For
> > servers which have no ability to not block the client or delay further
> > requests the function should still be called, and the impact would be
> > similar to the behavior today.
> >
> > Thoughts?
>
>
> The problem is, this isn't simple. Different servers have different
> scheduling mechanisms, and deferred operations specifications reach into
> scheduling in a horrible way.
>
> Should these run linearly? Should they be able to be pooled if
> env['rack.multithreaded']. In that case should they receive the same number
> of workers as the main request/response pool? Should they work out of the
> same pool?
>
> And that's just some basics with threads...
>
>
Ideally, the callback would be issued from the same process / thread /
fiber / context as the request itself. So all the questions of
parallelization are answered exactly like you would for requests. I'd be
interested to know of a server where this is not possible.


> You can quite easily handle this on your own in middleware or servers a
> number of ways today, without introducing either far reaching / extensive
> specs or incomplete restrictions that parallel some we already have (like
> stack based control).


How can this be quite easily handled today? Many plugins and frameworks do
not manage to do it when they should.

-Greg

[-- Attachment #2: Type: text/html, Size: 3318 bytes --]