Date | Commit message (Collapse) |
|
Socket::TCP_DEFER_ACCEPT() did not appear in the Socket module
distributed with Perl until 5.14, despite it being available
since Linux 2.4.
|
|
This is only needed for IO::Poll users, since users with
(signalfd || EVFILT_SIGNAL) support run with SIGPIPE (and
all other signals) blocked.
Fixes: 81a9a43fb858d197 ("daemon: use sigprocmask to block signals at startup")
|
|
There's a bunch of leftover "require" and "use" statements we no
longer need and can get rid of, along with some excessive
imports via "use".
IO::Handle usage isn't always obvious, so add comments
describing why a package loads it. Along the same lines,
document the tmpdir support as the reason we depend on
File::Temp 0.19, even though every Perl 5.10.1+ user has it.
While we're at it, favor "use" over "require", since it it gives
us extra compile-time checking.
|
|
For users not relying on socket activation via systemd (or
similar), we want to drop listeners ASAP so another process
can bind to their address. While we're at it, disable
TTIN and HUP handlers since we have no chance of starting
usable workers without listeners.
|
|
Keeping a ref to the IO::Socket handle was preventing
close(2) from being invoked on graceful shutdown of
worker.
|
|
This gets rid of the last "END{}" block in our code and cleans
up a (temporary) circular reference.
Furthermore, ensure the cleanup code still works in all
configurations by adding tests and testing both the -W1
(default, 1 worker) and -W0 (no workers) code paths.
|
|
EvCleanup only existed since Danga::Socket was a separate
component, and cleanup code belongs with the event loop.
|
|
Our attempt at using a self-pipe in signal handlers was
ineffective, since pure Perl code execution is deferred
and Perl doesn't use an internal self-pipe/eventfd. In
retrospect, I actually prefer the simplicity of Perl in
this regard...
We can use sigprocmask() from Perl, so we can introduce
signalfd(2) and EVFILT_SIGNAL support on Linux and *BSD-based
systems, respectively. These OS primitives allow us to avoid a
race where Perl checks for signals right before epoll_wait() or
kevent() puts the process to sleep.
The (few) systems nowadays without signalfd(2) or IO::KQueue
will now see wakeups every second to avoid missed signals.
|
|
While the master process has a self-pipe to avoid missing
signals, worker processes lack that aside from a pipe to
detect master death.
That pipe doesn't exist when there's no master process,
so it's possible DS::close never finishes because it
never woke up from epoll_wait. So create a pipe on
the worker_quit signal and force it into epoll/kevent
so it wakes up right away.
|
|
We need to block signals in workers during respawns
until they're ready to receive signals.
|
|
`$SIG{FOO} = "IGNORE"' will cause the daemon to miss signals
entirely. Instead, we can use sigprocmask to block signal
delivery until we have our signal handlers setup. This closes a
race where a PID file can be written for an init script and a
signal to be dropped via "IGNORE".
|
|
|
|
No point in uglifying our code since we need the POSIX
module in many places, anyways.
|
|
For users relying on socket activation via service manager (e.g.
systemd) and running multiple service instances (@1, @2),
we need to ensure configuration of the socket is NonBlocking.
Otherwise, service managers such as systemd may clear the
O_NONBLOCK flag for a small window where accept/accept4
blocks:
public-inbox-nntpd@1 |systemd |public-inbox-nntpd@2
--------------------------+----------------+--------------------
F_SETFL,O_NONBLOCK|O_RDWR | | (not running, yet)
|F_SETFL, O_RDWR |
|fork+exec @2... |
accept(...) # blocks! | |(started by systemd)
| |F_SETFL,O_NONBLOCK|O_RDWR
| |accept(...) non-blocking
It's a very small window where O_NONBLOCK can be cleared,
but it exists, and I finally hit it after many years.
|
|
We can close directly in event_step without bad side effects,
and then we also don't need to take a reason arg from worker_quit,
since we weren't logging it anywhere.
|
|
Similar to TCP_DEFER_ACCEPT on Linux, FreeBSD has a 'dataready'
accept filter which we can use to reduce wakeups when doing
TLS negotiation or plain HTTP. There's also a 'httpready'
which we can use for plain HTTP connections.
|
|
This Linux-specific option can save us some wakeups during
the TLS negotiation phase, and it can help with ordinary HTTP,
too.
Plain NNTP (and in the future, POP3) are the only things which
require the server send messages, first.
|
|
34K per idle connection adds up to large amounts of memory;
especially with the speed of malloc nowadays compared to the
cost of cache misses or worse, swapping.
|
|
I don't want to specify "--listen" in my systemd .service files,
so map 563 to NNTPS automatically (and 443 to HTTPS, but HTTPS
support doesn't work, yet).
|
|
It kinda, barely works, and I'm most happy I got it working
without any modifications to the main NNTP::event_step callback
thanks to the DS->write(CODE) support we inherited from
Danga::Socket.
|
|
All of our internal timing code should use monotonic clocks
for consistency against system clock adjustments.
This can be shared by our Daemon and NNTP packages.
|
|
It's only useful for a corner case in long-running daemons when
an admin decides to compact or vacuum a Xapian or SQLite DB.
As a result, other scripts should run slightly faster. For
instance, this saves about 80ms (2.710s => 2.630s) in t/mda.t
on my remote workstation.
While we're at it, make sure EvCleanup is properly require'd
in Daemon.pm and HTTP.pm and document our use of Devel::Peek.
|
|
These modules are unmaintained upstream at the moment, but I'll
be able to help with the intended maintainer once/if CPAN
ownership is transferred. OTOH, we've been waiting for that
transfer for several years, now...
Changes I intend to make:
* EPOLLEXCLUSIVE for Linux
* remove unused fields wasting memory
* kqueue bugfixes e.g. https://rt.cpan.org/Ticket/Display.html?id=116615
* accept4 support
And some lower priority experiments:
* switch to EV_ONESHOT / EPOLLONESHOT (incompatible changes)
* nginx-style buffering to tmpfile instead of string array
* sendfile off tmpfile buffers
* io_uring maybe?
|
|
Users on Perl 5.14+ are common, so we can try the bundled Socket
(not "Socket6") module before attempting Socket6 for IPv6.
|
|
We need to keep Unix-socket-only httpd instances working
without Socket6. This fixes t/httpd-unix.t with Socket6
uninstalled.
|
|
It looks like Net::Socket::IP comes with Perl 5.20 and
later; so we won't have to hassle users with another
package to install.
|
|
I've hit /proc/sys/fs/pipe-user-pages-* limits on some systems.
So stop hogging resources on pipes which don't benefit from
giant sizes.
Some of these can use eventfd in the future to further reduce
resource use.
|
|
I keep forgetting to run "make syntax"
|
|
We'll be using these in a more OO manner for V2Writable
(which doesn't use Danga::Socket), so lets not unnecessarily
register cleanup handlers intended for network daemons.
|
|
Using update-copyrights from gnulib
While we're at it, use the SPDX identifier for AGPL-3.0+ to
ease mechanical processing.
|
|
commit 6e238ee3396719e578d6a90e177a71ce9f8c1ca0
("nntp: respect 3 minute idle time for shutdown")
was incomplete, and needed this change to Daemon
to be effective.
In the future, there will be more common code between
NNTP.pm and HTTP.pm
|
|
As far as most process managers are concerned (e.g. systemd),
they should already start in '/'. So avoid making our daemon
more complex to run by requiring absolute paths during
development.
|
|
This allows systemd users to use SIGWINCH to temporarily
(and gracefully) stop an instance of a service without
doing a code reload to bring it back up:
# start temporary new service code
systemctl start public-inbox-nntpd@2.service
# momentarily paralyze original service
systemctl kill -s WINCH public-inbox-nntpd@1.service
if new_code_at_2_sucks
then
# restart original workers
systemctl kill -s HUP public-inbox-nntpd@1.service
else # new is better than old, replace original instance
systemctl restart public-inbox-nntpd@1.service
fi
# cleanup the temporary service
systemctl stop public-inbox-nntpd@2.service
This partially reverts commit 73d274e83b7d300f31e0cc1ceeacbf73c6c2a1e4
("daemon: disable SIGWINCH unless explicitly daemonized")
|
|
If using a master/worker setup, a careless user could be trying
to signal all processes using "killall". This may trigger bad
side-effects; but try to limit the side-effects as much as
possible.
|
|
Checking stdin/stdout/stderr is not sufficient as the daemon
without setsid can still be under the control of a terminal.
Unfortunately this means systemd users cannot use SIGWINCH,
either.
|
|
We do not need to count the httpd.async object
against our running client count, that is tied to
the socket of the actual client.
This prevents misleading sysadmins about connected
clients during shutdown.
|
|
They're effectively noops anyways, and we don't want to be
holding a reference to the read end of the parent pipe.
|
|
Users may change terminal sizes if the process is connected to a
terminal, so we can't reasonably expect SIGWINCH to work as
intended.
|
|
We shouldn't need sigprocmask unless we're running multiple
native threads or using vfork, neither of which is the case,
here.
|
|
This hopefully makes the intent of the code clearer, too.
The the HTTP use of the numeric reference for getline
caused problems in Git.pm, already.
|
|
git clones may take longer than 30s, much longer... So prepare
to wait almost indefinitely for sockets to timeout and document
the second signal behavior for immediate shutdown.
While we're at it, move parent death handling to a separate
class to avoid Danga::Socket->AddOtherFds, since that does not
allow proper handling the parent pipe being closed and would
actually misterminate a worker prematurely. t/nntpd.t is update
to illustrate the failure with workers enabled.
We will work to keep memory usage low and let clients take their
time without interrupting them.
|
|
We also require --stdout/--stderr/--pid-file to be absolute
paths for USR2 usage. However, allow PSGI files for -httpd
to be relative paths for ease-of-use.
|
|
We need to ensure $sock_pkg is preserved outside of the loop.
The variable passed to "for" or "foreach" is implicitly local
and restores the previous value when the loop exits. This is
documented in the perlsyn manpage in the "Foreach Loops"
section.
Fixes: ea1b6cbd422b ("daemon: allow using IO::Socket::IP over INET6")
|
|
IO::Socket::IP is bundled with newer versions of Perl,
so it is more likely to be available. There should
be no differences between these with our use cases.
|
|
This means we can avoid false-positives when inheriting multiple
Unix domain sockets.
|
|
Non-socket activation users will want to install Net::Server
for daemonization, pid file writing, and user/group switching.
|
|
No need to create a new sub which kill ourselves $$ when we can
invoke worker_quit directly.
|
|
Not that these subs are repeatedly created, but this makes
the code easier-to-review and these callbacks are idempotent
anyways.
|
|
We do not want to be accepting connections during graceful
shutdown because another new process is likely taking over.
This also allows us to free up the listener case another
(independent) process wants to claim it.
|
|
IO::Handle->new_from_fd has existed since at least 1996,
so it should be safe to depend on at this point.
|