* [PATCH] http: yield body->getline running time
@ 2016-05-30 4:54 5% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2016-05-30 4:54 UTC (permalink / raw)
To: meta
We cannot let a client monopolize the single-threaded server
even if it can drain the socket buffer faster than we can
emit data.
While we're at it, acknowledge the this behavior (which happens
naturally) in httpd/async.
The same idea is present in NNTP for the long_response code.
This is the HTTP followup to:
commit 0d0fde0bff97 ("nntp: introduce long response API for streaming")
commit 79d8bfedcdd2 ("nntp: avoid signals for long responses")
---
lib/PublicInbox/HTTP.pm | 5 ++++-
lib/PublicInbox/HTTPD/Async.pm | 4 +++-
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/lib/PublicInbox/HTTP.pm b/lib/PublicInbox/HTTP.pm
index fcbd758..6df1c3f 100644
--- a/lib/PublicInbox/HTTP.pm
+++ b/lib/PublicInbox/HTTP.pm
@@ -265,10 +265,13 @@ sub getline_response {
my $pull = $self->{pull} = sub {
local $/ = \8192;
my $forward = $self->{forward};
+ # limit our own running time for fairness with other
+ # clients and to avoid buffering too much:
+ my $n = 100;
while ($forward && defined(my $buf = $forward->getline)) {
$write->($buf);
last if $self->{closed};
- if ($self->{write_buf_size}) {
+ if ((--$n) <= 0 || $self->{write_buf_size}) {
$self->write($self->{pull});
return;
}
diff --git a/lib/PublicInbox/HTTPD/Async.pm b/lib/PublicInbox/HTTPD/Async.pm
index add07ce..fadf2d3 100644
--- a/lib/PublicInbox/HTTPD/Async.pm
+++ b/lib/PublicInbox/HTTPD/Async.pm
@@ -45,7 +45,9 @@ sub async_pass {
$self->watch_read(0);
$io->write($restart_read); # D::S::write
}
- return; # stay in watch_read
+ # stay in watch_read, but let other clients
+ # get some work done, too.
+ return;
} elsif (!defined $r) {
return if $!{EAGAIN} || $!{EINTR};
}
^ permalink raw reply related [relevance 5%]
* [PATCH 8/8] nntp: avoid signals for long responses
2015-09-25 2:27 5% [PATCH 0/8] nntp: more fixes and tiny speedups Eric Wong
@ 2015-09-25 2:27 7% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2015-09-25 2:27 UTC (permalink / raw)
To: meta
Using a signal-based timer can hurt throughput on a machine that's
overloaded. Ensure there's always forward progress and reduce the
number of syscalls we make, too.
---
lib/PublicInbox/NNTP.pm | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 3490a09..95aa4af 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -11,7 +11,7 @@ use PublicInbox::MID qw(mid2path);
use Email::MIME;
use Data::Dumper qw(Dumper);
use POSIX qw(strftime);
-use Time::HiRes qw(clock_gettime ualarm CLOCK_MONOTONIC);
+use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
use constant {
r501 => '501 command syntax error',
r221 => '221 Header follows',
@@ -502,16 +502,13 @@ sub long_response ($$$$) {
$self->{long_res} = sub {
# limit our own running time for fairness with other
# clients and to avoid buffering too much:
- my $yield;
- local $SIG{ALRM} = sub { $yield = 1 };
- ualarm(100000);
+ my $lim = 100;
my $err;
do {
eval { $cb->(\$beg) };
} until (($err = $@) || $self->{closed} ||
- ++$beg > $end || $yield || $self->{write_buf_size});
- ualarm(0);
+ ++$beg > $end || !--$lim || $self->{write_buf_size});
if ($err || $self->{closed}) {
$self->{long_res} = undef;
@@ -527,7 +524,7 @@ sub long_response ($$$$) {
} else {
$self->watch_read(1);
}
- } elsif ($yield || $self->{write_buf_size}) {
+ } elsif (!$lim || $self->{write_buf_size}) {
# no recursion, schedule another call ASAP
# but only after all pending writes are done
Danga::Socket->AddTimer(0, sub {
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 0/8] nntp: more fixes and tiny speedups
@ 2015-09-25 2:27 5% Eric Wong
2015-09-25 2:27 7% ` [PATCH 8/8] nntp: avoid signals for long responses Eric Wong
0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2015-09-25 2:27 UTC (permalink / raw)
To: meta
Still trying my best to avoid introducing more metadata or Xapian
index overhead to support OVER/XOVER, but it seems futile at this
point.
Eric Wong (8):
nntp: HDR allows metadata prefixed with ':'
nntp: consistently use 501 for unsupported LIST
searchidx: remove unused sub: next_doc_id
nntp: do not repeat result on blocked write
nntp: prefix FD on every log line
git: signal-safety for pipe writes
git: use fields for GitCatFile
nntp: avoid signals for long responses
lib/PublicInbox/GitCatFile.pm | 17 +++++++----------
lib/PublicInbox/NNTP.pm | 30 ++++++++++++++----------------
lib/PublicInbox/SearchIdx.pm | 2 --
3 files changed, 21 insertions(+), 28 deletions(-)
^ permalink raw reply [relevance 5%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2015-09-25 2:27 5% [PATCH 0/8] nntp: more fixes and tiny speedups Eric Wong
2015-09-25 2:27 7% ` [PATCH 8/8] nntp: avoid signals for long responses Eric Wong
2016-05-30 4:54 5% [PATCH] http: yield body->getline running time Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).