* [PATCH 3/3] git+gcf2client: switch to level-triggered wakeups
@ 2023-09-30 15:20 6% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2023-09-30 15:20 UTC (permalink / raw)
To: meta
Instead of using ->requeue to emulate level-triggered wakeups in
userspace, just use level-triggered wakeups in the kernel to
save some user time at the expense of system (kernel) time. Of
course, the ready list implementation in the kernel via C is
faster than a Perl one on our end.
We must still use requeue if we've got buffered data, however.
Followup-to: 1181a7e6a853 (listener: switch to level-triggered epoll)
---
lib/PublicInbox/Gcf2Client.pm | 4 ++--
lib/PublicInbox/Git.pm | 7 ++++---
2 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/lib/PublicInbox/Gcf2Client.pm b/lib/PublicInbox/Gcf2Client.pm
index 8f442787..8ac44a5e 100644
--- a/lib/PublicInbox/Gcf2Client.pm
+++ b/lib/PublicInbox/Gcf2Client.pm
@@ -9,7 +9,7 @@ use PublicInbox::Git;
use PublicInbox::Gcf2; # fails if Inline::C or libgit2-dev isn't available
use PublicInbox::Spawn qw(spawn);
use Socket qw(AF_UNIX SOCK_STREAM);
-use PublicInbox::Syscall qw(EPOLLIN EPOLLET);
+use PublicInbox::Syscall qw(EPOLLIN);
use PublicInbox::ProcessPipe;
# fields:
@@ -35,7 +35,7 @@ sub new {
my $sock = PublicInbox::ProcessPipe->maybe_new($pid, $s1);
$self->{inflight} = [];
$self->{epwatch} = \undef; # for Git->cleanup
- $self->SUPER::new($sock, EPOLLIN|EPOLLET);
+ $self->SUPER::new($sock, EPOLLIN);
}
sub gcf2_async ($$$;$) {
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 3062f293..5003be53 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -650,15 +650,16 @@ sub event_step {
if ($inflight && @$inflight) {
$self->cat_async_step($inflight);
return $self->close unless $self->{sock};
- # more to do? requeue for fairness:
- $self->requeue if @$inflight || exists($self->{rbuf});
+ # don't loop here to keep things fair, but we must requeue
+ # if there's already-read data in rbuf
+ $self->requeue if exists($self->{rbuf});
}
}
# idempotently registers with DS epoll/kqueue/select/poll
sub watch_async ($) {
$_[0]->{epwatch} //= do {
- $_[0]->SUPER::new($_[0]->{sock}, EPOLLIN|EPOLLET);
+ $_[0]->SUPER::new($_[0]->{sock}, EPOLLIN);
\undef;
}
}
^ permalink raw reply related [relevance 6%]
* [PATCH 2/9] listener: switch to level-triggered epoll
2021-10-01 9:54 5% [PATCH 0/9] daemon-related things Eric Wong
@ 2021-10-01 9:54 7% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2021-10-01 9:54 UTC (permalink / raw)
To: meta
On second thought, the ->requeue + accept retry code path isn't
worth the userspace complexity and overhead. Level-triggered
epoll has always annoyed me since it takes an inefficient code
path in the kernel; but taking our less-efficient code path in
Perl seems even worse. We also need to take load distribution
into account for multi-worker systems.
---
lib/PublicInbox/Listener.pm | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/lib/PublicInbox/Listener.pm b/lib/PublicInbox/Listener.pm
index 64bba5b0fd7c..09f1f2e5fabd 100644
--- a/lib/PublicInbox/Listener.pm
+++ b/lib/PublicInbox/Listener.pm
@@ -7,7 +7,7 @@ use strict;
use parent 'PublicInbox::DS';
use Socket qw(SOL_SOCKET SO_KEEPALIVE IPPROTO_TCP TCP_NODELAY);
use IO::Handle;
-use PublicInbox::Syscall qw(EPOLLIN EPOLLEXCLUSIVE EPOLLET);
+use PublicInbox::Syscall qw(EPOLLIN EPOLLEXCLUSIVE);
use Errno qw(EAGAIN ECONNABORTED EPERM);
# Warn on transient errors, mostly resource limitations.
@@ -22,7 +22,7 @@ sub new ($$$) {
setsockopt($s, IPPROTO_TCP, TCP_NODELAY, 1); # ignore errors on non-TCP
listen($s, 2**31 - 1); # kernel will clamp
my $self = bless { post_accept => $cb }, $class;
- $self->SUPER::new($s, EPOLLIN|EPOLLET|EPOLLEXCLUSIVE);
+ $self->SUPER::new($s, EPOLLIN|EPOLLEXCLUSIVE);
}
sub event_step {
@@ -38,7 +38,6 @@ sub event_step {
IO::Handle::blocking($c, 0); # no accept4 :<
eval { $self->{post_accept}->($c, $addr, $sock) };
warn "E: $@\n" if $@;
- $self->requeue;
} elsif ($! == EAGAIN || $! == ECONNABORTED || $! == EPERM) {
# EAGAIN is common and likely
# ECONNABORTED is common with bad connections
^ permalink raw reply related [relevance 7%]
* [PATCH 0/9] daemon-related things
@ 2021-10-01 9:54 5% Eric Wong
2021-10-01 9:54 7% ` [PATCH 2/9] listener: switch to level-triggered epoll Eric Wong
0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2021-10-01 9:54 UTC (permalink / raw)
To: meta
5/9 has been a long-time coming, a few other small things, too.
The diff stat for lib/ alone shows a nice reduction
Eric Wong (9):
doc: lei-security: some more updates
listener: switch to level-triggered epoll
daemon: make SO_ACCEPTFILTER a shared variable
ipc: run Net::SSLeay::randomize
ds: simplify signalfd use
inbox: inline and eliminate git_cleanup
inbox: keep DB handles if git processes are live
ds: inline set_cloexec
doc: lei-daemon: new manpage
Documentation/lei-daemon.pod | 61 ++++++++++++++++
Documentation/lei-security.pod | 5 +-
Documentation/lei.pod | 9 ++-
Documentation/lei_design_notes.txt | 2 +-
MANIFEST | 1 +
Makefile.PL | 2 +-
lib/PublicInbox/ConfigIter.pm | 2 +-
lib/PublicInbox/DS.pm | 110 ++++++++++++++---------------
lib/PublicInbox/DSKQXS.pm | 10 +--
lib/PublicInbox/Daemon.pm | 20 ++----
lib/PublicInbox/ExtMsg.pm | 2 +-
lib/PublicInbox/ExtSearchIdx.pm | 12 +---
lib/PublicInbox/Gcf2Client.pm | 4 +-
lib/PublicInbox/Git.pm | 4 +-
lib/PublicInbox/IPC.pm | 5 +-
lib/PublicInbox/Inbox.pm | 41 +++++------
lib/PublicInbox/LEI.pm | 17 +----
lib/PublicInbox/Listener.pm | 5 +-
lib/PublicInbox/Qspawn.pm | 2 +-
lib/PublicInbox/Sigfd.pm | 10 +--
lib/PublicInbox/Syscall.pm | 12 ++--
lib/PublicInbox/Watch.pm | 3 +-
script/public-inbox-watch | 9 ---
t/dir_idle.t | 6 +-
t/ds-leak.t | 4 +-
t/httpd-corner.t | 4 +-
t/httpd-https.t | 6 +-
t/httpd.t | 6 +-
t/imapd-tls.t | 7 +-
t/imapd.t | 6 +-
t/nntpd-tls.t | 7 +-
t/nntpd.t | 2 +-
t/sigfd.t | 7 +-
t/watch_maildir.t | 2 +-
xt/mem-imapd-tls.t | 6 +-
xt/net_writer-imap.t | 2 +-
36 files changed, 221 insertions(+), 192 deletions(-)
create mode 100644 Documentation/lei-daemon.pod
^ permalink raw reply [relevance 5%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-10-01 9:54 5% [PATCH 0/9] daemon-related things Eric Wong
2021-10-01 9:54 7% ` [PATCH 2/9] listener: switch to level-triggered epoll Eric Wong
2023-09-30 15:20 [PATCH 0/3] git: use Unix stream sockets for --batch* Eric Wong
2023-09-30 15:20 6% ` [PATCH 3/3] git+gcf2client: switch to level-triggered wakeups Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).