On Thu, Jul 17, 2014 at 3:34 PM, Eric Wong <e@80x24.org> wrote:

(resending my original message for the archives, original seems lost,
Torsten already got this privately)

Torsten Robitzki <Torsten@Robitzki.de> wrote:
> Hello,
> I'm implementing a C++ comet web server, that (tries) to implement rack to
> adapt ruby applications. Currently I'm reading a body very naively put the
> body into a ruby String and wrap it with a StringIO to provide the
> rack.input for the implementation. As I'm going to use the server for
> uploading images, I would like to implement a real, stream-like object to
> circumvent the need to buffer the POST body before handing it to the
> application.

unicorn implements input like tee(1) doing lazy, rewindable buffering:
http://unicorn.bogomips.org/Unicorn/TeeInput.html

This is the approach I would take. I'm not sure if TeeInput supports it, but for a generic server supporting websocket type use cases, I'd add a discardable buffer API too, so you can "hijack" the input stream and release any stale resources.

> Now, I've read the rack specs and read about rewind(). To implement
> rewind(), I would have to store the whole body, even when the upstream
> application just calculates some kind of checksum on the body, or uploads
> it to s3. What's the rational behind this part of the specification? Is it
> possible to not implement rewind() and to tell applications that need
> rewind(), to keep there own copy of the body, in case it is needed?

I don't know the rationale, but I suspect it's to make life easier for
application/API authors to know the data will exist for the lifetime
of the request/response cycle.

But I think having rewind as part of the spec sucks.

Agreed. I generally think a lot of concerns that used to be in middleware should not be so (e.g. chunking and content-length), but this concern actually could be handled by middleware so apps that want it can add it in the appropriate place in the chain. This would remove the responsibility from the server authors, which opens up a lot of potential optimizations and flexibility. We also lack a lot of potentially useful security considerations in this area (i.e. both static and dynamic configurable limits).

I seem to recall
there was hope of getting rid of it for Rack 2 (if it ever happens).

Yup.

However, since I'm not sure if Rack 2 will happen, rewind can already
be disabled and break spec in servers I implement:

unicorn has a "rewindable_input <true|false>" option (default=true):
http://unicorn.bogomips.org/Unicorn/Configurator.html

yahns is a tri-state: "input_buffering <:lazy|true|false>" option:
http://yahns.yhbt.net/yahns_config.txt (default=true)

Fwiw, rewindable_input(true) in unicorn is equivalent to
input_buffering(:lazy) in yahns. This is because unicorn expects nginx
to buffer for slow clients, whereas yahns does not require nginx and
buffers asynchronously even with trickling clients.

This is the approach I recommend today.

--

---
You received this message because you are subscribed to the Google Groups "Rack Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rack-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.