* [PATCH 9/9] ds: stop caring about event flags set by epoll/poll/kqueue
2019-06-10 5:18 7% [PATCH 0/9] ds: Diet Socket Eric Wong
@ 2019-06-10 5:18 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-06-10 5:18 UTC (permalink / raw)
To: meta
If we got something to write, then write it. Otherwise, try
reading; and continue dealing with errors which normally occur
along the way.
Trying to read requests while we need to buffer in luserspace
is suicidal from a memory management standpoint.
The only adjustment needed for existing callers is EvCleanup;
where we need to ensure we're always calling the dummy
EvCleanup::event_write callback to accomplish nothing.
---
lib/PublicInbox/DS.pm | 52 +++++++++++-------------------------
lib/PublicInbox/EvCleanup.pm | 8 ++++--
2 files changed, 21 insertions(+), 39 deletions(-)
diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index 9277981b..2f028a36 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -239,15 +239,18 @@ sub RunTimers {
return $timeout;
}
-# Placeholder callback when we hit POLLERR/POLLHUP or other unrecoverable
-# errors. Shouldn't be needed in the future.
-sub event_end ($) {
+sub event_step ($) {
my ($self) = @_;
return if $self->{closed};
- $self->{wbuf} = [];
- $self->{wbuf_off} = 0;
- # we're screwed if a read handler can't handle POLLERR/POLLHUP-type errors
+ my $wbuf = $self->{wbuf};
+ if (@$wbuf) {
+ $self->event_write;
+ return if $self->{closed} || scalar(@$wbuf);
+ }
+
+ # only read more requests if we've drained the write buffer,
+ # otherwise we can be buffering infinitely w/o backpressure
$self->event_read;
}
@@ -270,19 +273,7 @@ sub EpollEventLoop {
# that ones in the front triggered unregister-interest actions. if we
# can't find the %sock entry, it's because we're no longer interested
# in that event.
- my PublicInbox::DS $pob = $DescriptorMap{$ev->[0]};
- my $code;
- my $state = $ev->[1];
-
- DebugLevel >= 1 && $class->DebugMsg("Event: fd=%d (%s), state=%d \@ %s\n",
- $ev->[0], ref($pob), $ev->[1], time);
-
- # standard non-profiling codepat
- $pob->event_read if $state & EPOLLIN && ! $pob->{closed};
- $pob->event_write if $state & EPOLLOUT && ! $pob->{closed};
- if ($state & (EPOLLERR|EPOLLHUP) && ! $pob->{closed}) {
- event_end($pob);
- }
+ event_step($DescriptorMap{$ev->[0]});
}
return unless PostEventLoop();
}
@@ -327,11 +318,7 @@ sub PollEventLoop {
my ($fd, $state) = splice(@poll, 0, 2);
next unless $state;
- $pob = $DescriptorMap{$fd};
-
- $pob->event_read if $state & POLLIN && ! $pob->{closed};
- $pob->event_write if $state & POLLOUT && ! $pob->{closed};
- event_end($pob) if $state & (POLLERR|POLLHUP) && ! $pob->{closed};
+ event_step($DescriptorMap{$fd});
}
return unless PostEventLoop();
@@ -359,16 +346,7 @@ sub KQueueEventLoop {
foreach my $kev (@ret) {
my ($fd, $filter, $flags, $fflags) = @$kev;
- my PublicInbox::DS $pob = $DescriptorMap{$fd};
-
- DebugLevel >= 1 && $class->DebugMsg("Event: fd=%d (%s), flags=%d \@ %s\n",
- $fd, ref($pob), $flags, time);
-
- $pob->event_read if $filter == IO::KQueue::EVFILT_READ() && !$pob->{closed};
- $pob->event_write if $filter == IO::KQueue::EVFILT_WRITE() && !$pob->{closed};
- if ($flags == IO::KQueue::EV_EOF() && !$pob->{closed}) {
- event_end($pob);
- }
+ event_step($DescriptorMap{$fd});
}
return unless PostEventLoop();
}
@@ -672,11 +650,11 @@ sub on_incomplete_write {
=head2 (VIRTUAL) C<< $obj->event_read() >>
Readable event handler. Concrete deriviatives of PublicInbox::DS should
-provide an implementation of this. The default implementation will die if
-called.
+provide an implementation of this. The default implementation is a noop
+if called.
=cut
-sub event_read { die "Base class event_read called for $_[0]\n"; }
+sub event_read {} # noop
=head2 C<< $obj->event_write() >>
diff --git a/lib/PublicInbox/EvCleanup.pm b/lib/PublicInbox/EvCleanup.pm
index afed24ff..f76fb681 100644
--- a/lib/PublicInbox/EvCleanup.pm
+++ b/lib/PublicInbox/EvCleanup.pm
@@ -6,7 +6,6 @@ package PublicInbox::EvCleanup;
use strict;
use warnings;
use base qw(PublicInbox::DS);
-use fields qw(rd);
my $ENABLED;
sub enabled { $ENABLED }
@@ -25,7 +24,12 @@ sub once_init () {
pipe($r, $w) or die "pipe: $!";
fcntl($w, 1031, 4096) if $^O eq 'linux'; # 1031: F_SETPIPE_SZ
$self->SUPER::new($w);
- $self->{rd} = $r; # never read, since we never write..
+
+ # always writable, since PublicInbox::EvCleanup::event_write
+ # never drains wbuf. We can avoid wasting a hash slot by
+ # stuffing the read-end of the pipe into the never-to-be-touched
+ # wbuf
+ push @{$self->{wbuf}}, $r;
$self;
}
--
EW
^ permalink raw reply related [relevance 6%]
* [PATCH 0/9] ds: Diet Socket
@ 2019-06-10 5:18 7% Eric Wong
2019-06-10 5:18 6% ` [PATCH 9/9] ds: stop caring about event flags set by epoll/poll/kqueue Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2019-06-10 5:18 UTC (permalink / raw)
To: meta
Getting more aggressive with changes to PublicInbox::DS.
I'm thinking patches 8-9 will be important to make TLS
support easier.
On a side note: while reading the code to IO::KQueue;
I've noticed it doesn't take advantage at all of the
syscall reduction kevent() can potentially provide over
over the use of epoll_ctl+epoll_wait :<
Eric Wong (9):
ds: simplify write buffer accounting
ds: cleanup Errno imports and favor constant comparisons
ds: reduce Errno imports and drop ->close reason
ds: remove {fd} field
ds: remove steal_socket method
nntp: use sysread to append to existing buffer
ds: remove read method, here, too
ds: do not distinguish between POLLHUP and POLLERR
ds: stop caring about event flags set by epoll/poll/kqueue
lib/PublicInbox/DS.pm | 235 +++++++++------------------------
lib/PublicInbox/EvCleanup.pm | 8 +-
lib/PublicInbox/HTTP.pm | 11 +-
lib/PublicInbox/HTTPD/Async.pm | 4 +-
lib/PublicInbox/NNTP.pm | 44 +++---
5 files changed, 93 insertions(+), 209 deletions(-)
--
EW
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-06-10 5:18 7% [PATCH 0/9] ds: Diet Socket Eric Wong
2019-06-10 5:18 6% ` [PATCH 9/9] ds: stop caring about event flags set by epoll/poll/kqueue Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).