Date | Commit message (Collapse) |
|
Since signalfd is often combined with our event loop, give it a
convenient API and reduce the code duplication required to use it.
EventLoop is replaced with ::event_loop to allow consistent
parameter passing and avoid needlessly passing the package name
on stack.
We also avoid exporting SFD_NONBLOCK since it's the only flag we
support. There's no sense in having the memory overhead of a
constant function when it's in cold code.
|
|
t/v2mirror.t and t/lei-mirror.t are now skipped when curl
is missing (instead of failing in appropriate places).
A bunch of which() checks are updated to use require_cmd
to avoid explicitly loading Spawn.
|
|
Outside of mirrors, we can -init with indexlevel to avoid
calling -index explicitly.
|
|
This may help track down some occasional test failures I'm
seeing.
|
|
|
|
NetReader::<imap|nntp>_each were based on the -watch
code they now replace.
v2: do not warn on EINTR if user quit to fix occasional
test failure in t/imapd.t
|
|
Tested with git 1.8.3.1 on CentOS 7.x
`plan skip_all => ...' doesn't work after some tests have run,
we have to call skip() instead.
|
|
IPv4 gets plenty of real-world coverage, and apparently there's
Debian buildd hosts which lack IPv4(*). So ensure everything
can work on IPv6 and not cause problems for odd setups.
(*) https://bugs.debian.org/979432
|
|
Using "make update-copyrights" after setting GNULIB_PATH in my
config.mak
|
|
We may need to test against development versions of Xapian,
which may rely on setting `XAPIAN_COMPACT=xapian-compact-1.5'.
Ensure it's possible to do that.
And add a missing check in t/xcpdb-reshard.t, too.
|
|
It's only in RFC 2980 (not 977 or 3977), but Net::NNTP has
supported it since 2001, at least. We'll be making changes
to avoid pathological behavior, so test it, first.
|
|
We cannot blindly use the selected newsgroup for
HEAD/ARTICLE/BODY requests using Message-ID, since
those commands look across all newsgroups; not just
the selected one (if any).
So stuff a reference to the Inbox object into $smsg.
We can reduce args passed into set_nntp_headers() and
msg_hdr_write(), too.
Fixes: 0e6ceff37fc38f28 ("nntp: support slow blob retrievals")
|
|
This is no longer limited to Maildirs now that IMAP and NNTP
support exist; so give it a shorter name.
|
|
We can reduce the need to edit the config file for NNTP group names
this way.
|
|
No need to waste resources when doing minimal work. With
PI_TEST_VERSION=2, this fixes a test failure where
Net::NNTP::DESTROY was getting called in the shard process.
We'll also get rid of an unnecessary use_ok under v2, too.
|
|
The return value of art_lookup changed but this command wasn't
updated since it wasn't tested.
Fixes: 0e6ceff37fc38f28 ("nntp: support slow blob retrievals")
|
|
SQLite already knows the filename internally, so avoid having it
as a long-lived Perl SV to save some bytes when there's many
inboxes and open DBs.
|
|
While it's even less common to experience a replaced
msgmap.sqlite3 file, BOFHs may do the darndest things. This is
another step towards reducing the number of needless wakeups
we need to do in long-lived read-only daemons.
|
|
The default (and fast) TEST_RUN_MODE=2 preloads most modules,
but TEST_RUN_MODE=0 is more realistic and can catch some
problems which may show up in real-world use.
|
|
This is similar to IMAP support, but only supports polling.
Automatic altid support is not yet supported, yet; but may
be in the future.
v2: small grammar fix by Kyle Meyer
Link: https://public-inbox.org/meta/87sgeg5nxf.fsf@kyleam.com/
|
|
Net::NNTP does not support older IO::Socket::SSL. 1.94 on
CentOS 7.x fails HTTPS and IMAPS tests, too.
cf. https://rt.cpan.org/Ticket/Display.html?id=100529
|
|
Having `git cat-file' as a separate process naturally lends
itself to asynchronous dispatch. Our event loop for -nntpd no
longer blocks on slow git storage.
Pipelining in -imapd was tricky and bugs were exposed by
mbsync(1). Update t/nntpd.t to support pipelining ARTICLE
requests to ensure we don't have the same problems -imapd
did during development.
|
|
Since IMAP IDLE users aren't expected to issue any commands, we
can terminate their connections immediately on graceful
shutdown.
Furthermore, we need to drop the inotify FD from the epoll set
to avoid warnings during global destruction. Embarassingly,
this required fixing wacky test ordering from 2a717d13f10fcdc6
("nntpd+imapd: detect replaced over.sqlite3")
|
|
For v1 inboxes (and possibly v2 in the future, for VACUUM),
public-inbox-compact replaces over.sqlite3 with a new file.
This currently doesn't need an extra inotify watch descriptor
(or FD for kevent) at the moment, so it can coexist nicely for
systems w/o IO::KQueue or Linux::Inotify2.
|
|
NNTP and IMAP both require CRLF conversions on the wire.
They're also the only components which care about
$smsg->{bytes}, so store the CRLF-adjusted value in over.sqlite3
and Xapian DBs..
This will allow us to optimize RFC822.SIZE fetch item in IMAP
without triggering size mismatch errors in some clients' default
configurations (e.g. Mail::IMAPClient), but not most others.
It could also fix hypothetical problems with NNTP clients that
report discrepancies between overview and article data.
|
|
InboxWritable should only set $v2w->{parallel} if the $parallel
flag is defined to 0 or 1. We want indexing a new inbox to
utilize SMP, just like --reindex.
-index once again allows -j0/--jobs=0 to force single-process
use, and we'll be ensuring that works in tests to maintain
performance on small systems.
Fixes: 61a2fff5b34a3e32 ("admin: move index_inbox over")
|
|
We no longer load or use Email::MIME outside of comparison
tests.
|
|
Instead, favor PublicInbox::MIME->new for non-attachment emails.
We may support alternatives to Email::MIME down the line.
We'll still keep Email::MIME->create to deal with attachments,
for now, but there's also a fair amount of test duplication
we should eliminate, later.
|
|
We need to detect FS errors and bail out on the test
if we can't open a file -nntpd was just writing to.
|
|
Since the advent of run_script(), we can rely on it to simplify
our test code. Changes like this will let us evolve the
internal API more easily while preserving stable CLI interfaces,
especially since we test the v2 path by default, now.
|
|
The `xqx' sub requires an absolute path for optional
commands.
Fixes: 6e07def560b211d9 ("testcommon: spawn-aware system() and qx[] workalikes")
|
|
Barely noticeable on Linux, but this gives a 1-2% speedup
on a FreeBSD 11.3 VM and lets us use built-in redirects
rather than relying on /bin/sh.
|
|
We don't want to propagate %SIG changes to other tests when
running multiple tests within the same process via t/run.perl.
|
|
It was implemented at some point, but it was more things to
support and the worst of both worlds: both unrealistic compared
to real-world use and slower than run_mode=2.
Noticed while looking for speling erorrs.
|
|
I didn't wait until September to do it, this year!
|
|
This cuts down on lines of code in individual test cases and
fixes some misnamed error messages by using "$0" consistently.
This will also provide us with a method of swapping out
dependencies which provide equivalent functionality (e.g
"Xapian" SWIG can replace "Search::Xapian" XS bindings).
|
|
Spawning a new Perl interpreter for every test case
means Perl has to reparse and recompile every single file
it needs, costing us performance and development time.
Now that we've modified our code to avoid global state,
we can preload everything we need.
The new "check-run" test target is now 20-30% faster
than the original "check" target.
|
|
We want to be able to use run_script with *.t files, so
t/common.perl putting subs into the top-level "main" namespace
won't work. Instead, make it a module which uses Exporter
like other libraries.
|
|
We'll also introduce a tmpdir() API to give tempdirs
consistent names.
|
|
We can shave several hundred milliseconds off tests which spawn
daemons by preloading and avoiding startup time for common
modules which are already loaded in the parent process.
This also gives ENV{TAIL} support to all tests which support
daemons which log to stdout/stderr.
|
|
There were still a few places where we used worker processes
unnecessarily in tests, causing a small amount of unnecessary
overhead.
Followup-to: ad221e9b2852f6c5 ("t/*.t: disable nntpd/httpd worker processes in most tests")
|
|
This only gives a 5% speedup or so, but anything helps.
|
|
"mainrepo" ws a bad name and artifact from the early days when I
intended for there to be a "spamrepo" (now just the
ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be
especially confusing, since v2 needs at least two git
repositories (epoch + all.git) to function and we shouldn't
confuse users by having them point to a git repository for v2.
Much of our documentation already references "INBOX_DIR" for
command-line arguments, so use "inboxdir" as the
git-config(1)-friendly variant for that.
"mainrepo" remains supported indefinitely for compatibility.
Users may need to revert to old versions, or may be referring
to old documentation and must not be forced to change config
files to account for this change.
So if you're using "mainrepo" today, I do NOT recommend changing
it right away because other bugs can lurk.
Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
|
|
|
|
IO::Socket::INET->new is rather verbose with the options hash,
extract it into a standalone sub
|
|
RFC3977 6.1.2.2 LISTGROUP allows a [range] arg after [group],
and supporting it allows NNTP support in neomutt to work again.
Tested with NeoMutt 20170113 (1.7.2) on Debian stretch
(oldstable)
|
|
RFC3977 8.4.2 mandates the order of non-standard headers
to be after the first seven standard headers/metadata;
so "Xref:" must appear after "Lines:"|":lines".
Additionally, non-required header names must be followed
by ":full".
Cc: Jonathan Corbet <corbet@lwn.net>
Reported-by: Urs Janßen
<E1hmKBw-0008Bq-8t@akw>
|
|
This is only tested so far with my patches to Net::NNTP at:
https://rt.cpan.org/Ticket/Display.html?id=129967
Memory use in C10K situations is disappointing, but that's
the nature of compression.
gzip compression over HTTPS does have the advantage of not
keeping zlib streams open when clients are idle, at the
cost of worse compression.
|
|
Net::NNTP won't attempt to use older versions of IO::Socket::SSL
because 2.007 is the "first version with default CA on most platforms"
according to comments in Net::NNTP. But then again we don't make
remote requests when testing...
|
|
Some clients may rely on this for STARTTLS support.
|