* [PATCH 02/11] ds: move requeue logic over from NNTP
2019-06-29 19:59 6% [PATCH 00/11] ds: more updates Eric Wong
@ 2019-06-29 19:59 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-06-29 19:59 UTC (permalink / raw)
To: meta
We'll be reusing requeue in other places to reduce trips to
the kernel to retrieve "hot" descriptors.
---
lib/PublicInbox/DS.pm | 14 ++++++++++++++
lib/PublicInbox/NNTP.pm | 22 ++++------------------
2 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index 28240843..9f245347 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -37,6 +37,8 @@ use Errno qw(EAGAIN EINVAL EEXIST);
use Carp qw(croak confess carp);
require File::Spec;
+my $nextt; # timer for next_tick
+my $nextq = []; # queue for next_tick
our (
%DescriptorMap, # fd (num) -> PublicInbox::DS object
$Epoll, # Global epoll fd (or DSKQXS ref)
@@ -594,6 +596,18 @@ sub shutdn ($) {
}
}
+sub next_tick () {
+ $nextt = undef;
+ my $q = $nextq;
+ $nextq = [];
+ $_->event_step for @$q;
+}
+
+sub requeue ($) {
+ push @$nextq, $_[0];
+ $nextt ||= PublicInbox::EvCleanup::asap(*next_tick);
+}
+
package PublicInbox::DS::Timer;
# [$abs_float_firetime, $coderef];
sub cancel {
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 0a053627..83970309 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -38,20 +38,6 @@ my %DISABLED; # = map { $_ => 1 } qw(xover list_overview_fmt newnews xhdr);
my $EXPMAP; # fd -> [ idle_time, $self ]
my $expt;
our $EXPTIME = 180; # 3 minutes
-my $nextt;
-
-my $nextq = [];
-sub next_tick () {
- $nextt = undef;
- my $q = $nextq;
- $nextq = [];
- event_step($_) for @$q;
-}
-
-sub requeue ($) {
- push @$nextq, $_[0];
- $nextt ||= PublicInbox::EvCleanup::asap(*next_tick);
-}
sub update_idle_time ($) {
my ($self) = @_;
@@ -655,12 +641,12 @@ sub long_response ($$) {
push @$wbuf, $long_cb;
# wbuf may be populated by $cb, no need to rearm if so:
- requeue($self) if scalar(@$wbuf) == 1;
+ $self->requeue if scalar(@$wbuf) == 1;
} else { # all done!
$long_cb = undef;
res($self, '.');
out($self, " deferred[$fd] done - %0.6f", now() - $t0);
- requeue($self) unless $self->{wbuf};
+ $self->requeue unless $self->{wbuf};
}
};
$self->write($long_cb); # kick off!
@@ -915,7 +901,7 @@ sub cmd_starttls ($) {
return '580 can not initiate TLS negotiation';
res($self, '382 Continue with TLS negotiation');
$self->{sock} = IO::Socket::SSL->start_SSL($sock, %$opt);
- requeue($self) if PublicInbox::DS::accept_tls_step($self);
+ $self->requeue if PublicInbox::DS::accept_tls_step($self);
undef;
}
@@ -990,7 +976,7 @@ sub event_step {
# maybe there's more pipelined data, or we'll have
# to register it for socket-readiness notifications
- requeue($self) unless $self->{wbuf};
+ $self->requeue unless $self->{wbuf};
}
sub not_idle_long ($$) {
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 00/11] ds: more updates
@ 2019-06-29 19:59 6% Eric Wong
2019-06-29 19:59 7% ` [PATCH 02/11] ds: move requeue logic over from NNTP Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2019-06-29 19:59 UTC (permalink / raw)
To: meta
We can simplify a lot of our async logic now that we don't have
to deal with the buffer-to-heap behavior of Danga::Socket.
The biggest change is now we no longer tie git-http-backend(1)
runtime and memory use to the bandwidth of a slow HTTP client.
This increases buffering on the FS (which may be tmpfs or
a fast SSD); but it's what nginx (and varnish) would be doing,
anyways
We can further remove a lot of the EvCleanup code since that
was to workaround deferred close being deferred for too long
when no I/O events were firing.
HTTPS now works, but more work needs to be done because
Varnish is still a requirement for busy sites.
Eric Wong (11):
ds: share lazy rbuf handling between HTTP and NNTP
ds: move requeue logic over from NNTP
http: use requeue instead of watch_in1
listener: use edge-triggered notifications
ds: handle deferred DS->close after timers
ds: consolidate IO::Socket::SSL checks
http: support HTTPS (kinda)
parentpipe: document and use one-shot wakeups
parentpipe: make the ->close call more obvious
httpd/async: switch to buffering-as-fast-as-possible
http: use bigger, but shorter-lived buffers for pipes
MANIFEST | 1 +
lib/PublicInbox/DS.pm | 85 +++++++++++++++----------
lib/PublicInbox/DSKQXS.pm | 4 +-
lib/PublicInbox/Daemon.pm | 4 +-
lib/PublicInbox/EvCleanup.pm | 80 +++--------------------
lib/PublicInbox/HTTP.pm | 102 +++++++++++++++--------------
lib/PublicInbox/HTTPD/Async.pm | 55 ++++++++--------
lib/PublicInbox/Listener.pm | 7 +-
lib/PublicInbox/NNTP.pm | 47 +++-----------
lib/PublicInbox/ParentPipe.pm | 17 +++--
lib/PublicInbox/Qspawn.pm | 2 +-
lib/PublicInbox/Syscall.pm | 4 +-
lib/PublicInbox/TLS.pm | 9 +--
t/httpd-https.t | 141 +++++++++++++++++++++++++++++++++++++++++
14 files changed, 314 insertions(+), 244 deletions(-)
create mode 100644 t/httpd-https.t
--
EW
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-06-29 19:59 6% [PATCH 00/11] ds: more updates Eric Wong
2019-06-29 19:59 7% ` [PATCH 02/11] ds: move requeue logic over from NNTP Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).