From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS15169 209.85.128.0/17 X-Spam-Status: No, score=-3.8 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from mail-qt0-f185.google.com (mail-qt0-f185.google.com [209.85.216.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 487AB2021E for ; Tue, 15 Nov 2016 23:10:42 +0000 (UTC) Received: by mail-qt0-f185.google.com with SMTP id n6sf27537990qtd.0 for ; Tue, 15 Nov 2016 15:10:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20120806; h=sender:date:from:to:subject:message-id:mime-version :content-disposition:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=EWsP0WehsZmaabW024wQGKtiQ26VkWHtXIR7VsGkTG8=; b=D2wVj3wYrVTQROS43uUvorHHRizdlYAGwQDqJNWsBb5Uzq785WjgK1WCwLjok0BEly At/OSwoEHmEIajMOLHuf/OX5A9UKCBpZDQC9IxRLx6csuYvs7lojuhbrgXtK0FZxNqkU nf9B3DlfzGOJomuDCm6ie92MwPuG3IHfW9VFw6tVCpO8vXdmxhuhNcpBvcV1nXGYNa7S VKpaXBN5OT51ZxJMqdTGABNex9DXp8FKptoxQWkBy+4lv7AV55YqySm0dXRsLDWkat9w fiQ3W5PqpEMYKlAischRQI/tEp+FdVljVTgZIlBYX4KVMlr2SrHBqhqLXmUqjlMIlae/ lvig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=sender:x-gm-message-state:date:from:to:subject:message-id :mime-version:content-disposition:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=EWsP0WehsZmaabW024wQGKtiQ26VkWHtXIR7VsGkTG8=; b=WBjCKCEzQt8DL/kyD5j4YjE/8ekULMO/2XPlfjFOfBJjik/47Rq4UVg/011bV3gnwP dQbFYA8/s7cCTY4Grd1a+awf+UlcDGLm3AcKX4wiwWqHWGbKiSmzUUoCrdtR/zTVYTL6 4S/JCB5ZFe9RVDJW4CnXNx8fycypyLjuJzVI3lrWe57626r2vKjBuYFAwMzleSMe/cuY jwO43knYBOYPRYvjZ6e8XNJ+QzsICFGF8tihGtOIuj9enMAMFOfkcGsOQXu96C868q2h AgzlKC1JkK35nsoXuGTGNjVn9E3Ki3FxQrWdfptT0Hfx5JYXCB7BQcZh1qt/4NxvQw9v 6ZgA== Sender: rack-devel@googlegroups.com X-Gm-Message-State: ABUngvduNMSjm9EPfHwGyRD+em2FRxN3Ps48+ShKvA97+55IVrwnCfTAC5pcZBm3KDHG4Q== X-Received: by 10.157.40.215 with SMTP id s81mr1918687ota.7.1479251441433; Tue, 15 Nov 2016 15:10:41 -0800 (PST) X-BeenThere: rack-devel@googlegroups.com Received: by 10.157.31.122 with SMTP id x55ls7207820otx.36.gmail; Tue, 15 Nov 2016 15:10:41 -0800 (PST) X-Received: by 10.200.47.147 with SMTP id l19mr4199445qta.4.1479251441069; Tue, 15 Nov 2016 15:10:41 -0800 (PST) Received: from dcvr.yhbt.net (dcvr.yhbt.net. [64.71.152.64]) by gmr-mx.google.com with ESMTPS id n18si6761135pfb.2.2016.11.15.15.10.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Nov 2016 15:10:40 -0800 (PST) Received-SPF: pass (google.com: domain of e@80x24.org designates 64.71.152.64 as permitted sender) client-ip=64.71.152.64; Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 446992021E; Tue, 15 Nov 2016 23:10:40 +0000 (UTC) Date: Tue, 15 Nov 2016 23:10:40 +0000 From: Eric Wong To: rack-devel@googlegroups.com Subject: big responses to slow clients: Rack vs PSGI Message-ID: <20161115-slow-clients-rack-vs-psgi@80x24.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline X-Original-Sender: e@80x24.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of e@80x24.org designates 64.71.152.64 as permitted sender) smtp.mailfrom=e@80x24.org Reply-To: rack-devel@googlegroups.com Precedence: list Mailing-list: list rack-devel@googlegroups.com; contact rack-devel+owners@googlegroups.com List-ID: X-Google-Group-Id: 486215384060 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , I've been poking around in Plack/PSGI for Perl5 some months, and am liking it in some ways more than Rack. This only covers server-agnostic web applications; IMHO exposing applications to server-specific stuff defeats the purpose of these common specs. In Rack, one major problem I have is streaming large responses requires calling body.each synchronously. For handling writing large responses to slow clients, this means a Rack web server has 2 choices: 1) Block the calling Thread, Fiber, or process until the slow client can consume the input. This hurts if you have many slow clients blocking all your threads. body.each { |buf| client.write(buf) } body.close Simple, but your app is at the mercy of how fast the client chooses to read the response. 2) Detect :wait_writable/:wait_readable (EAGAIN) when writing to the slow client and start buffering the response to memory or filesystem. This may lead to out-of-memory or out-of-storage conditions. nginx does this by default when proxying, so Rubyists are often unaware of this as it's common to use nginx in front of Rack servers for this purpose. Something like the following should handle slow clients without relying on nginx for buffering: tmp = nil body.each do |buf| if tmp tmp.write(buf) else # the optimistic case: case ret = client.write_nonblock(buf, exception: false) when :wait_writable, :wait_writable # EAGAIN :< tmp = Tempfile.new(ret.to_s) tmp.write(buf) when Integer exp = buf.bytesize if exp > ret # partial write :< tmp = Tempfile.new('partial') tmp.write(buf.byteslice(ret, exp - ret)) end end end end if tmp server_specific_finish(client, tmp, body) else body.close if body.respond_to?(:close) end Gross; but smaller responses never get buffered this way. Any server-specific logic is still contained within the server itself, the Rack app may remain completely unaware of how a server handles slow clients. PSGI allows at least two methods for streaming large responses. I will only cover the "pull" method of getline+close below. Naively, getline+close is usable like the Rack method 1) for body.each: # Note: "getline" in Plack/PSGI is not required to return # a "line", so it can behave like "readpartial" in Ruby. while (defined(my $buf = $body->getline)) { $client->write($buf); } $body->close; ...With all the problems of blocking on the $client->write call. On the surface, the difference between Rack and PSGI here is minor. However, "getline" yielding control to the server entirely has a significant advantage over the Rack app calling a Proc provided by the server: The server can stop calling $body->getline once it detects a client is slow. # For the non-Perl-literate, it's pretty similar to Ruby. # Scalar variables are prefixed with $, and method. # calls are "$foo->METHOD" instead of "foo.METHOD" in Ruby # if/else/elsif/while all work the same as in Ruby # I will over-comment here assuming readers here are not # familiar with Perl. # Make client socking non-blocking, equivalent to # "IO#nonblock = true" in Ruby; normal servers would only # call this once after accept()-ing a connection. $client->blocking(0); my $blocked; # "my" declares a locally-scoped variable # "undef" in Perl are the equivalent of "nil" in Ruby, # so "defined" checks here are equivalent to Ruby nil checks while (defined(my $buf = $body->getline)) { # length($buf) is roughly buf.bytesize in Ruby; # I'll assume all data is binary since Perl's Unicode # handling confuses me no matter how many times I RTFM. my $exp = length($buf); # Behaves like Ruby IO#write_nonblock after the # $client->blocking(0) call above: my $ret = $client->syswrite($buf); # $ret is the number of bytes written on success: if (defined $ret) { if ($exp > $ret) { # partial write :< # similar to String#byteslice in Ruby: $blocked = substr($buf, $ret, $exp - $ret); last; # break out of the while loop } # else { continue looping on while } # $! is the system errno from syswrite (see perlvar manpage # for details), $!{E****} just checks for $! matching the # particular error number. } elsif ($!{EAGAIN} || $!{EWOULDBLOCK}) { # A minor detail in this example: # this assignment is a copy, so equivalent to # "blocked = buf.dup" in Ruby, NOT merely # "blocked = buf". $blocked = $buf; last; # break out of the while loop } else { # Perl does not raise exceptions by default on # syscall errors, "die" is the standard exception # throwing mechanism: die "syswrite failed: $!\n"; } } if (defined $blocked) { server_specific_finish($client, $blocked, $body); } else { $body->close; } In both my Rack and PSGI examples, I have a reference to a server_specific_finish call. In the Rack example, this method will stream the entire contents of tmp (a Tempfile) to the client. The problem is tmp in the Rack example may be as large as the entire response. This sucks for big responses. In the PSGI example, the server_specific_finish call will only have the contents of one buffer from $body->getline in memory at a time. The server will make further calls to $body->getline when (and only when) the previous buffer is fully-written to the client socket. There is only one (app-provided) buffer in server memory at once, not entire response. Both server_specific_finish calls will call the "close" method on the body when the entire response is written to the client socket. Delaying the "close" call may make sense for logging purposes in Rack, even if body.each is long done running, and is obviously required in the PSGI case since further "getline" calls need to be made before "close". The key difference is that in Rack, the data is "pushed" to the server by the Rack app. In PSGI, the app may instead ask the server to "pull" that data. Anyways, thanks for reading this far. I just felt like writing something down for future Rack/Ruby-related projects. I'm not sure if Rack can change without breaking all existing apps and middlewares. -- --- You received this message because you are subscribed to the Google Groups "Rack Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to rack-devel+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.