Date | Commit message (Collapse) |
|
This cuts down on lines of code in individual test cases and
fixes some misnamed error messages by using "$0" consistently.
This will also provide us with a method of swapping out
dependencies which provide equivalent functionality (e.g
"Xapian" SWIG can replace "Search::Xapian" XS bindings).
|
|
The displays the Xapian ->get_percent value in the skeleton to
improve scanning of relevancy; irrelevant results do not display
that.
This fixes broken #anchor links introduced in the previous
commit, irrelevant messages now link to the /$INBOX/$MESSAGE_ID page.
|
|
Spawning a new Perl interpreter for every test case
means Perl has to reparse and recompile every single file
it needs, costing us performance and development time.
Now that we've modified our code to avoid global state,
we can preload everything we need.
The new "check-run" test target is now 20-30% faster
than the original "check" target.
|
|
We want to be able to use run_script with *.t files, so
t/common.perl putting subs into the top-level "main" namespace
won't work. Instead, make it a module which uses Exporter
like other libraries.
|
|
These usages of file-local global variables make the *.t files
incompatible with run_script(). Instead, use anonymous subs,
"our", or pass the parameter as appropriate.
|
|
Expose MAX_SIZE via "our" will make it possible
to use in tests, and configure, later.
Additionally, returning HTTP 500 code for big files is not an
Internal Server Error, just a memory limit... Some browsers
won't show our HTML response with the link to the raw file in
case of errors, either, so we'll return 200 to ensure users can
use the link to access the raw blob.
Finally, throw in some tests to the existing solver_git testcase,
since that was incomplete and was pointlessly loading Plack
modules without testing PSGI.
|
|
"delete local" is only in Perl v5.11.0, and we only depend on
Perl v5.10.1. We already localize and delete it as two separate
statements immediately above.
I wish this was hidden behind a "use feature" flag like other
new-fangled things :<. Oh well, I think the oldest Perl actually
in use for this project is 5.16 (CentOS 7.x).
|
|
Apparently, neither our previous address parsing code nor
Email::Address::XS recognizes local, username-only addresses
in the form of <username> (without "@host"). Without
this change, Email::Address::XS->address would return
"undef", so we need to filter it out via "grep { defined }"
It seems the cases where users email each other on the same
machine is small and public-inbox won't be able to index
addresses for those cases... Oh well :/
|
|
Email::Address::XS is a dependency of modern versions of Email::MIME,
so it's likely loaded and installed on newer systems, already;
and capable of handling more corner-cases than our pure-Perl
fallback.
We still fallback to the imperfect-but-good-enough-in-practice
pure-Perl code while avoiding the non-XS Email::Address (which
was susceptible to DoS attacks (CVE-2015-7686)). We just need
to keep "git fast-import" happy.
|
|
Some users will set their From: headers in the form of:
"<user@example.com> (A U Thor)", where their name is in
the parenthesized comment. Use that instead of the
email address, if available.
|
|
This gets rid of the last "END{}" block in our code and cleans
up a (temporary) circular reference.
Furthermore, ensure the cleanup code still works in all
configurations by adding tests and testing both the -W1
(default, 1 worker) and -W0 (no workers) code paths.
|
|
Wacky dates show up in lore for valid messages. Lets ignore
them and let future generations deal with Y10K and time-travel
problems.
|
|
Date::Parse is not optimized for RFC2822 dates and isn't
packaged on OpenBSD. It's still useful for historical
email when email clients were less conformant, but is
less relevant for new emails.
|
|
It's unnecessary code which I'm not sure we ever used. In
retrospect, completely clearing the environment doesn't make
sense for the processes we spawn. We don't need to clobber
individual environment variables in our code, either
(and if we did for tests, we can use 'local').
|
|
We can localize changes to $0 so $0 is restored when the
"script" sub is done. This will be helpful when we encounter
a stuck/slow processes during our tests (hopefully never!)
|
|
We don't want the user's ~/.public-inbox/config to be read from
during tests. I only noticed this because I had a non-existent
pathname for one of my inboxes :x
I've also verified this change by running "inotifywait
~/.public-inbox/config -m" in another terminal while running
"make check"; (perhaps a portable solution could make it
into the test suite).
|
|
Test output can be a terminal if running as "perl -I lib t/$FOO.t",
and showing fsck progress is pointless for tests.
|
|
Tested FreeBSD 11.2. I'm starting to think I'm too conservative
with this check and it could be safely expanded to cover any OS
with UNIX sockets.
|
|
Our attempt at using a self-pipe in signal handlers was
ineffective, since pure Perl code execution is deferred
and Perl doesn't use an internal self-pipe/eventfd. In
retrospect, I actually prefer the simplicity of Perl in
this regard...
We can use sigprocmask() from Perl, so we can introduce
signalfd(2) and EVFILT_SIGNAL support on Linux and *BSD-based
systems, respectively. These OS primitives allow us to avoid a
race where Perl checks for signals right before epoll_wait() or
kevent() puts the process to sleep.
The (few) systems nowadays without signalfd(2) or IO::KQueue
will now see wakeups every second to avoid missed signals.
|
|
Oops, IO::KQueue support was broken due to this missing
constant. Add a new ds-kqxs.t test case to ensure we
test the IO::KQueue path if IO::KQueue is available.
|
|
I'm not sure if TZ minute offsets aside from '00' or '30' exist,
but lets just deal with them properly when negative. Examples
taken from various inboxes on lore.kernel.org. These are mostly
message from spammers, but some are legitimate messages.
|
|
Broken email clients sent the darndest things, make sure
we can still support them when we make Date::Parse optional.
|
|
Otherwise it's hard to figure what fails.
|
|
xt/ is typically reserved for "eXtended tests" intended for
the maintainers and not ordinary users. Since these require
special configuration and do nothing by waste cycles
during startup, they qualify.
|
|
It's more consistent with our current terminology and
"PI_DIR" is already used to override ~/.public-inbox/
(which holds "config" and possibly other files which affect
all inboxes for a particular user, but is not an inbox itself);
so stop advertising GIANT_PI_DIR in skip messages.
|
|
Newer versions of git enable the commit graph by default.
Since we blow away our temporary directories every test,
generating graphis is a waste and clutters stderr with
"Computing commit graph generation numbers" messages.
|
|
We'll also introduce a tmpdir() API to give tempdirs
consistent names.
|
|
Threads are officially discouraged by perl5-porters and proves
problematic with my Perl installation when using run_mode=1
to speed up tests. So just use fork() and pipes to share
results from Net::NNTP.
|
|
We can shave several hundred milliseconds off tests which spawn
daemons by preloading and avoiding startup time for common
modules which are already loaded in the parent process.
This also gives ENV{TAIL} support to all tests which support
daemons which log to stdout/stderr.
|
|
It seems caching can happen within OpenSSL or negotiation
can be delayed in some cases. In any case, don't barf on
PublicInbox::TLS::epollbit() when connect_SSL succeeds
unexpectedly.
|
|
We need to ensure the worker process is terminated before
starting a new connection, so leave a persistent HTTP/1.1
connection open and wait for the SIGKILL to take effect
and drop the client.
|
|
We did not have a test for this, and need to guard against
regressions when changing Xapcmd to use File::Temp->newdir
in future commits.
|
|
Use the "-q" flag like everywhere else.
|
|
The "strict" pragma makes code easier to debug, and we had
undeclared variables as a result in t/watch_maildir_v2.t.
So use it everywhere to be consistent with the rest of our
code.
|
|
There were still a few places where we used worker processes
unnecessarily in tests, causing a small amount of unnecessary
overhead.
Followup-to: ad221e9b2852f6c5 ("t/*.t: disable nntpd/httpd worker processes in most tests")
|
|
This more than doubles the speed of the test, since we make
many invocations of -xcpdb.
|
|
This more than doubles the speed of these tests
|
|
This more than doubles the speed of the test.
|
|
This nets us a 20% speedup or so.
|
|
This only gives a 5% speedup or so, but anything helps.
|
|
This only gives a small 10% speedup or so, but anything helps.
|
|
This only gives a small ~10% speedup, since -httpd still
needs execve, but any speedup is welcome.
|
|
While this didn't use IPC::Run, having to reload several Perl
modules and scripts is slow and inefficient, so roughly
double the speed of this test.
|
|
We need to be careful and explicitly close FDs before doing
-index, since we can't rely on FD_CLOEXEC without execve(2)
syscalls.
|
|
This test runs more than twice as fast, now.
|
|
Another noticeable speedup, this test is roughly ~3x faster now.
|
|
Not taking advantage of faster run modes in run_script, yet
since some lifetime problems need to be sorted.
|
|
This nets us another sizeable speedup.
|
|
This gives a 2-3x speedup on the test with the default
run_mode=1.
|
|
Perl parsing is slow, and run_script default behavior allows
this to speed up t/edit.t by over 100% in my case.
|