Date | Commit message (Collapse) |
|
"delete local" is only in Perl v5.11.0, and we only depend on
Perl v5.10.1. We already localize and delete it as two separate
statements immediately above.
I wish this was hidden behind a "use feature" flag like other
new-fangled things :<. Oh well, I think the oldest Perl actually
in use for this project is 5.16 (CentOS 7.x).
|
|
For users not relying on socket activation via systemd (or
similar), we want to drop listeners ASAP so another process
can bind to their address. While we're at it, disable
TTIN and HUP handlers since we have no chance of starting
usable workers without listeners.
|
|
Keeping a ref to the IO::Socket handle was preventing
close(2) from being invoked on graceful shutdown of
worker.
|
|
Apparently, neither our previous address parsing code nor
Email::Address::XS recognizes local, username-only addresses
in the form of <username> (without "@host"). Without
this change, Email::Address::XS->address would return
"undef", so we need to filter it out via "grep { defined }"
It seems the cases where users email each other on the same
machine is small and public-inbox won't be able to index
addresses for those cases... Oh well :/
|
|
Email::Address::XS is a dependency of modern versions of Email::MIME,
so it's likely loaded and installed on newer systems, already;
and capable of handling more corner-cases than our pure-Perl
fallback.
We still fallback to the imperfect-but-good-enough-in-practice
pure-Perl code while avoiding the non-XS Email::Address (which
was susceptible to DoS attacks (CVE-2015-7686)). We just need
to keep "git fast-import" happy.
|
|
Some users will set their From: headers in the form of:
"<user@example.com> (A U Thor)", where their name is in
the parenthesized comment. Use that instead of the
email address, if available.
|
|
We need to use $PublicInbox::DS::in_loop instead of ::running().
The latter is not valid for systems with signalfd or kqueue and
is now gone, completely.
Not needing periodic cleanups at all to deal with unlinked pack
indices will be a tougher task...
|
|
git_unquote works in-place, and we sometimes see strange
filenames, or badly munged diffs with terminal escape
characters (for colorization) end up in emails.
|
|
This gets rid of the last "END{}" block in our code and cleans
up a (temporary) circular reference.
Furthermore, ensure the cleanup code still works in all
configurations by adding tests and testing both the -W1
(default, 1 worker) and -W0 (no workers) code paths.
|
|
We'll be supporting idle timeout for the HTTP code in the
future to deal directly with Internet-exposed clients w/o
Varnish or nginx.
|
|
EvCleanup only existed since Danga::Socket was a separate
component, and cleanup code belongs with the event loop.
|
|
The "dsyn" target needs to remain working, despite still
being dependent on GNU-isms at the moment.
Fixes: 73fe3421f1ecbdc8 ("build: support doc generation w/o GNU make")
|
|
This was causing warnings to pop up in syslogs for messages with
empty Subject headers.
|
|
Wacky dates show up in lore for valid messages. Lets ignore
them and let future generations deal with Y10K and time-travel
problems.
|
|
-mda should not be dealing with broken Date: headers
nowadays, and deprioritize it in our documentation and
internal checks.
|
|
Date::Parse is not optimized for RFC2822 dates and isn't
packaged on OpenBSD. It's still useful for historical
email when email clients were less conformant, but is
less relevant for new emails.
|
|
Changes will be coming for MsgTime to stop depending on
Date::Parse due to lack of package availability on OpenBSD
and suboptimal performance on RFC822 dates.
|
|
This is a transitionary interface which does NOT require an
event loop. It can be plugged into in current synchronous code
without major surgery.
It allows HTTP/1.1 pipelining-like functionality by taking
advantage of predictable and well-specified POSIX pipe semantics
by stuffing multiple git cat-file requests into the --batch pipe
With xt/git_async_cmp.t and GIANT_GIT_DIR=git.git, the async
interface is 10-25% faster than the synchronous interface since
it can keep the "git cat-file" process busier.
This is expected to improve performance on systems with slower
storage (but multiple cores).
|
|
We can replace the GNU-isms for building docs with Perl5
equivalents. The only downside is the resulting Makefile
gets larger, but that's the price of portability.
|
|
run_die() doesn't require an $env arg, so there's no
point passing "undef" to it.
|
|
It's unnecessary code which I'm not sure we ever used. In
retrospect, completely clearing the environment doesn't make
sense for the processes we spawn. We don't need to clobber
individual environment variables in our code, either
(and if we did for tests, we can use 'local').
|
|
We can create a stamp to avoid rerunning the check unless
NEWS.atom changes (and it will, soon, I hope :>).
|
|
SpamAssassin has used re2c (via sa-compile) for many years, now,
and it seems to work fine, there. GMime also looks promising
when combined with Inline::C since GMime can operate on mmap-ed
regions.
Given the inevitable demise of many .orgs when price rise;
supporting a URL rewriter similar to .mailmap makes sense.
And HTTP CONNECT seems like something our -httpd can support
to let firewalled users read over NNTP.
|
|
I haven't noticed this being a problem in practice, but
be consistent with the rest of the singleton stuff.
Since we always call Reset() at load time, only do
initialization in that sub and not at declaration.
|
|
We can localize changes to $0 so $0 is restored when the
"script" sub is done. This will be helpful when we encounter
a stuck/slow processes during our tests (hopefully never!)
|
|
We don't want the user's ~/.public-inbox/config to be read from
during tests. I only noticed this because I had a non-existent
pathname for one of my inboxes :x
I've also verified this change by running "inotifywait
~/.public-inbox/config -m" in another terminal while running
"make check"; (perhaps a portable solution could make it
into the test suite).
|
|
Since we give users no indication or control of how "git gc"
runs, showing its progress is confusing.
|
|
Test output can be a terminal if running as "perl -I lib t/$FOO.t",
and showing fsck progress is pointless for tests.
|
|
Tested FreeBSD 11.2. I'm starting to think I'm too conservative
with this check and it could be safely expanded to cover any OS
with UNIX sockets.
|
|
We need to force an update to Makefile (not Makefile.PL) when
MANIFEST changes. Since "Makefile" (aka. "$(FIRST_MAKEFILE)")
is already a single-colon make target; we can't create a
double-colon rule to augment it. So we'll continue using a
"Makefile.PL" rule, but have it recreate the resulting Makefile
Finally, change the "check" target to use "prove -b" instead of
"prove -l" so we test against "blib/lib", since what's in the
"blib" dir will be installed.
Fixes: 4c20de0694d06ff3 ("Makefile.PL: add dependency on MANIFEST contents")
|
|
Our attempt at using a self-pipe in signal handlers was
ineffective, since pure Perl code execution is deferred
and Perl doesn't use an internal self-pipe/eventfd. In
retrospect, I actually prefer the simplicity of Perl in
this regard...
We can use sigprocmask() from Perl, so we can introduce
signalfd(2) and EVFILT_SIGNAL support on Linux and *BSD-based
systems, respectively. These OS primitives allow us to avoid a
race where Perl checks for signals right before epoll_wait() or
kevent() puts the process to sleep.
The (few) systems nowadays without signalfd(2) or IO::KQueue
will now see wakeups every second to avoid missed signals.
|
|
Oops, IO::KQueue support was broken due to this missing
constant. Add a new ds-kqxs.t test case to ensure we
test the IO::KQueue path if IO::KQueue is available.
|
|
I'm not sure if TZ minute offsets aside from '00' or '30' exist,
but lets just deal with them properly when negative. Examples
taken from various inboxes on lore.kernel.org. These are mostly
message from spammers, but some are legitimate messages.
|
|
Broken email clients sent the darndest things, make sure
we can still support them when we make Date::Parse optional.
|
|
Otherwise it's hard to figure what fails.
|
|
xt/ is typically reserved for "eXtended tests" intended for
the maintainers and not ordinary users. Since these require
special configuration and do nothing by waste cycles
during startup, they qualify.
|
|
It's more consistent with our current terminology and
"PI_DIR" is already used to override ~/.public-inbox/
(which holds "config" and possibly other files which affect
all inboxes for a particular user, but is not an inbox itself);
so stop advertising GIANT_PI_DIR in skip messages.
|
|
Newer versions of git enable the commit graph by default.
Since we blow away our temporary directories every test,
generating graphis is a waste and clutters stderr with
"Computing commit graph generation numbers" messages.
|
|
We'll also introduce a tmpdir() API to give tempdirs
consistent names.
|
|
Since we're using Perl 5.10.1 and File::Temp 0.19+, we don't
need Xtmpdirs at all for cleaning up tempdirs on failure and
can just rely on the DESTROY handler provided by File::Temp.
|
|
Threads are officially discouraged by perl5-porters and proves
problematic with my Perl installation when using run_mode=1
to speed up tests. So just use fork() and pipes to share
results from Net::NNTP.
|
|
We can shave several hundred milliseconds off tests which spawn
daemons by preloading and avoiding startup time for common
modules which are already loaded in the parent process.
This also gives ENV{TAIL} support to all tests which support
daemons which log to stdout/stderr.
|
|
While the master process has a self-pipe to avoid missing
signals, worker processes lack that aside from a pipe to
detect master death.
That pipe doesn't exist when there's no master process,
so it's possible DS::close never finishes because it
never woke up from epoll_wait. So create a pipe on
the worker_quit signal and force it into epoll/kevent
so it wakes up right away.
|
|
We need to block signals in workers during respawns
until they're ready to receive signals.
|
|
`$SIG{FOO} = "IGNORE"' will cause the daemon to miss signals
entirely. Instead, we can use sigprocmask to block signal
delivery until we have our signal handlers setup. This closes a
race where a PID file can be written for an init script and a
signal to be dropped via "IGNORE".
|
|
As described in prove(1), .prove is storage for --state=save
and .proverc allows per-worktree customizations.
|
|
It seems caching can happen within OpenSSL or negotiation
can be delayed in some cases. In any case, don't barf on
PublicInbox::TLS::epollbit() when connect_SSL succeeds
unexpectedly.
|
|
We need to ensure the worker process is terminated before
starting a new connection, so leave a persistent HTTP/1.1
connection open and wait for the SIGKILL to take effect
and drop the client.
|
|
We did not have a test for this, and need to guard against
regressions when changing Xapcmd to use File::Temp->newdir
in future commits.
|
|
Use the "-q" flag like everywhere else.
|