From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 6A4001F4C1 for ; Sat, 29 Jun 2019 19:59:54 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 11/11] http: use bigger, but shorter-lived buffers for pipes Date: Sat, 29 Jun 2019 19:59:51 +0000 Message-Id: <20190629195951.32160-12-e@80x24.org> In-Reply-To: <20190629195951.32160-1-e@80x24.org> References: <20190629195951.32160-1-e@80x24.org> List-Id: Linux pipes default to 65536 bytes in size, and we want to read external processes as fast as possible now that we don't use Danga::Socket or buffer to heap. However, drop the buffer ASAP if we need to wait on anything; since idle buffers can be idle for eons. This lets other execution contexts can reuse that memory right away. --- lib/PublicInbox/HTTPD/Async.pm | 11 ++++++----- lib/PublicInbox/Qspawn.pm | 2 +- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/lib/PublicInbox/HTTPD/Async.pm b/lib/PublicInbox/HTTPD/Async.pm index a468ed91..e195fab0 100644 --- a/lib/PublicInbox/HTTPD/Async.pm +++ b/lib/PublicInbox/HTTPD/Async.pm @@ -33,13 +33,13 @@ sub new { $self; } -sub main_cb ($$$) { - my ($http, $fh, $bref) = @_; +sub main_cb ($$) { + my ($http, $fh) = @_; sub { my ($self) = @_; - my $r = sysread($self->{sock}, $$bref, 8192); + my $r = sysread($self->{sock}, my $buf, 65536); if ($r) { - $fh->write($$bref); # may call $http->close + $fh->write($buf); # may call $http->close if ($http->{sock}) { # !closed $self->requeue; # let other clients get some work done, too @@ -64,7 +64,8 @@ sub async_pass { # will automatically close this ($self) object. $http->{forward} = $self; $fh->write($$bref); # PublicInbox:HTTP::{chunked,identity}_wcb - my $cb = $self->{cb} = main_cb($http, $fh, $bref); + $$bref = undef; # we're done with this + my $cb = $self->{cb} = main_cb($http, $fh); $cb->($self); # either hit EAGAIN or ->requeue to keep EPOLLET happy } diff --git a/lib/PublicInbox/Qspawn.pm b/lib/PublicInbox/Qspawn.pm index f2630a0f..8f0b9fe2 100644 --- a/lib/PublicInbox/Qspawn.pm +++ b/lib/PublicInbox/Qspawn.pm @@ -128,7 +128,7 @@ sub psgi_qx { my $rpipe; # comes from popen_rd my $async = $env->{'pi-httpd.async'}; my $cb = sub { - my $r = sysread($rpipe, my $buf, 8192); + my $r = sysread($rpipe, my $buf, 65536); if ($async) { $async->async_pass($env->{'psgix.io'}, $qx, \$buf); } elsif (defined $r) { -- EW