Date | Commit message (Collapse) |
|
We may need to test against development versions of Xapian,
which may rely on setting `XAPIAN_COMPACT=xapian-compact-1.5'.
Ensure it's possible to do that.
And add a missing check in t/xcpdb-reshard.t, too.
|
|
"*foo" is ambiguous in that it may refer to a bareword file handle;
so we'll use it where we can without triggering warnings.
PublicInbox::TestCommon::run_script_exit required dropping the
prototype, however. We'll also future-proof by dropping "use
warnings" in Cgit.pm and use the less-ambiguous "//=" in Inbox.pm
while we're in the area.
|
|
Existing use of the $ENV{TAIL} relied on parsing --std{out,err},
which was only usable for read-only daemons. However, -watch
doesn't use PublicInbox::Daemon code(*), so attempt to figure
out redirects.
(*) -watch won't able to run as a daemon in cases when
git-credential prompts for IMAP/NNTP passwords.
PublicInbox::Daemon is also designed for read-only
parallelism where all worker processes are the same.
Any subprocesses spawned by -watch are to do specific
tasks for a particular set of inboxes.
|
|
Since we have IMAP client support in -watch; make sure per-URL
settings are familiar to git users by taking advantage of git's
URL matching abilities.
This requires git 1.8.5+, which most users ought to have
(though base CentOS 7 is on 1.8.3).
|
|
We can avoid synchronous `waitpid(-1, 0)' and save a process
when simultaneously watching Maildirs.
One DS bug is fixed: ->Reset needs to clear the DS $in_loop flag
in forked children so dwaitpid() fails and allows git processes
to be reaped synchronously. TestCommon also calls DS->Reset
when spawning new processes, since t/imapd.t uses DS->EventLoop
while waiting on -watch to write.
|
|
Since we already use inotify and EVFILT_VNODE (kqueue)
in -imapd, we might as well use them directly in -watch,
too.
This will allow public-inbox-watch to use PublicInbox::DS
for timers to watch newsgroups/mailboxes and have saner
signal handling in future commits.
|
|
This fixes a bug on FreeBSD 11 here -nntpd + TEST_RUN_MODE=2
(default) was occasionally causing failures in t/v2writable.t
due to the kqueue descriptor being auto-closed by the OS on fork.
|
|
Net::NNTP does not support older IO::Socket::SSL. 1.94 on
CentOS 7.x fails HTTPS and IMAPS tests, too.
cf. https://rt.cpan.org/Ticket/Display.html?id=100529
|
|
IMAP requires either the Email::Address::XS or Mail::Address
package (part of perl-MailTools RPM or libmailtools-perl deb);
and Email::Address::XS is not officially packaged for some older
distros, most notably CentOS 7.x.
|
|
None of our tests rely on this failing, so just bail out
if the system is out of resources.
|
|
Somewhat recent versions of GNU tail(1) use inotify(7) on Linux;
so don't penalize hackers using TAIL='tail -F' to run their tests
with extra delays.
Ironically, we still need to busy loop on /proc/$TAIL_PID/{fd,fdinfo}
since inotify doesn't seem to support procfs.
|
|
To avoid confusing future readers and users, recommend
PublicInbox::Eml in our Import POD and refer to PublicInbox::Eml
comments at the top of PublicInbox::MIME.
mime_load() confined to t/eml.t, since we won't be using
it anywhere else in our tests.
|
|
We no longer load or use Email::MIME outside of comparison
tests.
|
|
PublicInbox::Eml has enough functionality to replace the
Email::MIME-based PublicInbox::MIME.
|
|
Email::MIME eats memory, wastes time parsing out all the
headers, and some problems can't be fixed without breaking
compatibility for other projects which depend on it.
Informal benchmarks show a ~2x improvement in general
stats gathering scripts and ~10% improvement in HTML
view rendering.
We also don't need the ability to create MIME messages, just
parse them and maybe drop an attachment.
While this isn't the zero-copy or streaming MIME parser of my
dreams; it's still an improvement in that it doesn't keep a
scalar copy of the raw body around along with subparts. It also
doesn't parse subparts up front, so it can also replace our uses
of Email::Simple.
|
|
We don't need the callback arg, anymore.
|
|
Replace them with .eml files generated with the help of
Email::MIME, but without some extraneous and unnecessary
headers, and strip mime_load down to just loading files.
This will give us more freedom to experiment with other mail
libraries which may be more correct, better maintained, use
less memory and/or be faster than Email::MIME.
|
|
We'll use this to create, memoize, and reuse .eml files. This
will be used to reduce (and eventually eliminate) our dependency
on Email::MIME in tests.
|
|
Barely noticeable on Linux, but this gives a 1-2% speedup
on a FreeBSD 11.3 VM and lets us use built-in redirects
rather than relying on /bin/sh.
|
|
Otherwise, the waitpid(-1, 0) call in Xapcmd::process_queue()
may reap it in a subsequent test when using t/run.perl to reuse
processes for testing.
While we're at it, make Xapcmd::process_queue warn about unknown
PIDs in case other PIDs leak through to us in the future.
|
|
It was implemented at some point, but it was more things to
support and the worst of both worlds: both unrealistic compared
to real-world use and slower than run_mode=2.
Noticed while looking for speling erorrs.
|
|
I didn't wait until September to do it, this year!
|
|
Most spawn and popen_rd callers die on failure to spawn,
anyways, and some are missing checks entirely. This saves
us a bunch of verbose error-checking code in callers.
This also makes popen_rd more consistent, since it already
dies on pipe creation failures.
|
|
In rare cases where Message-IDs get reused, we do not want to
hold onto the large Email::MIME objects in memory after showing
the first message. So discard each message as soon as we're
done using it so we can save memory for the next message.
The new and expensive xt/mem-msgview.t test shows a nearly 14MB
reduction for two ~7MB messages. run_script() also gets
upgraded to make it easier to pass large inputs via IO GLOBs.
|
|
PublicInbox::Search always loads DBD::SQLite, so we
can't blindly "use" it in t/xcpdb-reshard.t. We also
need to account for that in TestCommon.
|
|
We can save callers the trouble of {-hold} and {-dev_null}
refs as well as the trouble of calling fileno().
|
|
Xapian upstream is slowly phasing out the XS-based Search::Xapian
in favor of the SWIG-generated "Xapian" package. While Debian and
both FreeBSD have Search::Xapian, OpenBSD only includes the "Xapian"
binding.
More information about the status of the "Xapian" Perl module here:
https://trac.xapian.org/ticket/523
|
|
This cuts down on lines of code in individual test cases and
fixes some misnamed error messages by using "$0" consistently.
This will also provide us with a method of swapping out
dependencies which provide equivalent functionality (e.g
"Xapian" SWIG can replace "Search::Xapian" XS bindings).
|
|
require_git is no longer in the "::main" namespace, so we must
call Test::More::plan() explicitly.
|
|
Using Perl "open" to dup(2) and save the old handles is required
since "local *STDIN = *STDIN" does not work on old Perls. Even
worse, this was silently a no-op when tested with Perl 5.24.1 on
Debian 9.x and led to confusing failures in the t/httpd-corner.t
lsof(1) tests when run after t/v2mirror.t from the same worker
process using t/run.perl.
|
|
Spawning a new Perl interpreter for every test case
means Perl has to reparse and recompile every single file
it needs, costing us performance and development time.
Now that we've modified our code to avoid global state,
we can preload everything we need.
The new "check-run" test target is now 20-30% faster
than the original "check" target.
|
|
We want to be able to use run_script with *.t files, so
t/common.perl putting subs into the top-level "main" namespace
won't work. Instead, make it a module which uses Exporter
like other libraries.
|