about summary refs log tree commit homepage
path: root/t
DateCommit message (Collapse)
2019-12-24testcommon: add require_mods method and use it
This cuts down on lines of code in individual test cases and fixes some misnamed error messages by using "$0" consistently. This will also provide us with a method of swapping out dependencies which provide equivalent functionality (e.g "Xapian" SWIG can replace "Search::Xapian" XS bindings).
2019-12-20view: show percentage in search results thread skeleton
The displays the Xapian ->get_percent value in the skeleton to improve scanning of relevancy; irrelevant results do not display that. This fixes broken #anchor links introduced in the previous commit, irrelevant messages now link to the /$INBOX/$MESSAGE_ID page.
2019-12-19t/run.perl: to avoid repeated process spawning for *.t
Spawning a new Perl interpreter for every test case means Perl has to reparse and recompile every single file it needs, costing us performance and development time. Now that we've modified our code to avoid global state, we can preload everything we need. The new "check-run" test target is now 20-30% faster than the original "check" target.
2019-12-19tests: move t/common.perl to PublicInbox::TestCommon
We want to be able to use run_script with *.t files, so t/common.perl putting subs into the top-level "main" namespace won't work. Instead, make it a module which uses Exporter like other libraries.
2019-12-19t/*.t: avoid sharing "my" variables in subs
These usages of file-local global variables make the *.t files incompatible with run_script(). Instead, use anonymous subs, "our", or pass the parameter as appropriate.
2019-12-18viewvcs: flesh out some functionality and test
Expose MAX_SIZE via "our" will make it possible to use in tests, and configure, later. Additionally, returning HTTP 500 code for big files is not an Internal Server Error, just a memory limit... Some browsers won't show our HTML response with the link to the raw file in case of errors, either, so we'll return 200 to ensure users can use the link to access the raw blob. Finally, throw in some tests to the existing solver_git testcase, since that was incomplete and was pointlessly loading Plack modules without testing PSGI.
2019-12-17t/edit.t: drop redundant "delete local $ENV{...}"
"delete local" is only in Perl v5.11.0, and we only depend on Perl v5.10.1. We already localize and delete it as two separate statements immediately above. I wish this was hidden behind a "use feature" flag like other new-fangled things :<. Oh well, I think the oldest Perl actually in use for this project is 5.16 (CentOS 7.x).
2019-12-15address: explicitly reject local-only addresses
Apparently, neither our previous address parsing code nor Email::Address::XS recognizes local, username-only addresses in the form of <username> (without "@host"). Without this change, Email::Address::XS->address would return "undef", so we need to filter it out via "grep { defined }" It seems the cases where users email each other on the same machine is small and public-inbox won't be able to index addresses for those cases... Oh well :/
2019-12-15address: use Email::Address::XS if available
Email::Address::XS is a dependency of modern versions of Email::MIME, so it's likely loaded and installed on newer systems, already; and capable of handling more corner-cases than our pure-Perl fallback. We still fallback to the imperfect-but-good-enough-in-practice pure-Perl code while avoiding the non-XS Email::Address (which was susceptible to DoS attacks (CVE-2015-7686)). We just need to keep "git fast-import" happy.
2019-12-15address: use comment as name if no phrase available
Some users will set their From: headers in the form of: "<user@example.com> (A U Thor)", where their name is in the parenthesized comment. Use that instead of the email address, if available.
2019-12-14daemon: use DESTROY for unlinking --pid-file
This gets rid of the last "END{}" block in our code and cleans up a (temporary) circular reference. Furthermore, ensure the cleanup code still works in all configurations by adding tests and testing both the -W1 (default, 1 worker) and -W0 (no workers) code paths.
2019-12-12msgtime: avoid obviously out-of-range dates (for now)
Wacky dates show up in lore for valid messages. Lets ignore them and let future generations deal with Y10K and time-travel problems.
2019-12-12msgtime: drop Date::Parse for RFC2822
Date::Parse is not optimized for RFC2822 dates and isn't packaged on OpenBSD. It's still useful for historical email when email clients were less conformant, but is less relevant for new emails.
2019-12-11spawn: remove support for clearing the env
It's unnecessary code which I'm not sure we ever used. In retrospect, completely clearing the environment doesn't make sense for the processes we spawn. We don't need to clobber individual environment variables in our code, either (and if we did for tests, we can use 'local').
2019-12-11t/common: set $0 when running script w/o fork
We can localize changes to $0 so $0 is restored when the "script" sub is done. This will be helpful when we encounter a stuck/slow processes during our tests (hopefully never!)
2019-12-11t: localize the PI_CONFIG env
We don't want the user's ~/.public-inbox/config to be read from during tests. I only noticed this because I had a non-existent pathname for one of my inboxes :x I've also verified this change by running "inotifywait ~/.public-inbox/config -m" in another terminal while running "make check"; (perhaps a portable solution could make it into the test suite).
2019-11-29t/replace: quiet "git fsck" invocation
Test output can be a terminal if running as "perl -I lib t/$FOO.t", and showing fsck progress is pointless for tests.
2019-11-28t/httpd-unix: FreeBSD expects to fail with EADDRINUSE
Tested FreeBSD 11.2. I'm starting to think I'm too conservative with this check and it could be safely expanded to cover any OS with UNIX sockets.
2019-11-27httpd|nntpd: avoid missed signal wakeups
Our attempt at using a self-pipe in signal handlers was ineffective, since pure Perl code execution is deferred and Perl doesn't use an internal self-pipe/eventfd. In retrospect, I actually prefer the simplicity of Perl in this regard... We can use sigprocmask() from Perl, so we can introduce signalfd(2) and EVFILT_SIGNAL support on Linux and *BSD-based systems, respectively. These OS primitives allow us to avoid a race where Perl checks for signals right before epoll_wait() or kevent() puts the process to sleep. The (few) systems nowadays without signalfd(2) or IO::KQueue will now see wakeups every second to avoid missed signals.
2019-11-27dskqxs: fix missing EV_DISPATCH define
Oops, IO::KQueue support was broken due to this missing constant. Add a new ds-kqxs.t test case to ensure we test the IO::KQueue path if IO::KQueue is available.
2019-11-27msgtime: deal with strange minutes in TZ offsets
I'm not sure if TZ minute offsets aside from '00' or '30' exist, but lets just deal with them properly when negative. Examples taken from various inboxes on lore.kernel.org. These are mostly message from spammers, but some are legitimate messages.
2019-11-27t/msgtime: add more checks for known cases
Broken email clients sent the darndest things, make sure we can still support them when we make Date::Parse optional.
2019-11-27t/msgtime: show date in test descriptions
Otherwise it's hard to figure what fails.
2019-11-24tests: move giant inbox/git dependent tests to xt/
xt/ is typically reserved for "eXtended tests" intended for the maintainers and not ordinary users. Since these require special configuration and do nothing by waste cycles during startup, they qualify.
2019-11-24t/perf-*.t: use $ENV{GIANT_INBOX_DIR} consistently
It's more consistent with our current terminology and "PI_DIR" is already used to override ~/.public-inbox/ (which holds "config" and possibly other files which affect all inboxes for a particular user, but is not an inbox itself); so stop advertising GIANT_PI_DIR in skip messages.
2019-11-24tests: quiet down commit graph
Newer versions of git enable the commit graph by default. Since we blow away our temporary directories every test, generating graphis is a waste and clutters stderr with "Computing commit graph generation numbers" messages.
2019-11-24tests: use File::Temp->newdir instead of tempdir()
We'll also introduce a tmpdir() API to give tempdirs consistent names.
2019-11-24t/nntpd-validate: get rid of threads dependency
Threads are officially discouraged by perl5-porters and proves problematic with my Perl installation when using run_mode=1 to speed up tests. So just use fork() and pipes to share results from Net::NNTP.
2019-11-24t/common: start_script replaces spawn_listener
We can shave several hundred milliseconds off tests which spawn daemons by preloading and avoiding startup time for common modules which are already loaded in the parent process. This also gives ENV{TAIL} support to all tests which support daemons which log to stdout/stderr.
2019-11-24t/nntpd-tls: sometimes SSL_connect succeeds quickly
It seems caching can happen within OpenSSL or negotiation can be delayed in some cases. In any case, don't barf on PublicInbox::TLS::epollbit() when connect_SSL succeeds unexpectedly.
2019-11-24t/httpd-corner: wait for worker process death
We need to ensure the worker process is terminated before starting a new connection, so leave a persistent HTTP/1.1 connection open and wait for the SIGKILL to take effect and drop the client.
2019-11-24t/xcpdb-reshard: test xcpdb --compact
We did not have a test for this, and need to guard against regressions when changing Xapcmd to use File::Temp->newdir in future commits.
2019-11-24t/v1-add-remove-add: quiet down "git init"
Use the "-q" flag like everywhere else.
2019-11-24tests: use strict everywhere
The "strict" pragma makes code easier to debug, and we had undeclared variables as a result in t/watch_maildir_v2.t. So use it everywhere to be consistent with the rest of our code.
2019-11-24tests: disable daemon workers in a few more places
There were still a few places where we used worker processes unnecessarily in tests, causing a small amount of unnecessary overhead. Followup-to: ad221e9b2852f6c5 ("t/*.t: disable nntpd/httpd worker processes in most tests")
2019-11-16t/xcpdb-reshard: use run_script for -xcpdb
This more than doubles the speed of the test, since we make many invocations of -xcpdb.
2019-11-16t/indexlevels-mirror*: switch to run_script
This more than doubles the speed of these tests
2019-11-16t/v2mda: switch to run_script in many places
This more than doubles the speed of the test.
2019-11-16t/watch_filter_rubylang: run_script for -init and -index
This nets us a 20% speedup or so.
2019-11-16t/nntpd: use run_script for -init
This only gives a 5% speedup or so, but anything helps.
2019-11-16t/watch_maildir_v2: use run_script for -init
This only gives a small 10% speedup or so, but anything helps.
2019-11-16t/httpd: use run_script for -init
This only gives a small ~10% speedup, since -httpd still needs execve, but any speedup is welcome.
2019-11-16t/convert-compact: convert to run_script
While this didn't use IPC::Run, having to reload several Perl modules and scripts is slow and inefficient, so roughly double the speed of this test.
2019-11-16t/v2mirror: switch to default run_mode for speedup
We need to be careful and explicitly close FDs before doing -index, since we can't rely on FD_CLOEXEC without execve(2) syscalls.
2019-11-16t/mda_filter_rubylang: drop IPC::Run dependency
This test runs more than twice as fast, now.
2019-11-16t/mda: switch to run_script for testing
Another noticeable speedup, this test is roughly ~3x faster now.
2019-11-16t/v2mirror: get rid of IPC::Run dependency
Not taking advantage of faster run modes in run_script, yet since some lifetime problems need to be sorted.
2019-11-16t/purge: convert to run_script
This nets us another sizeable speedup.
2019-11-16t/init: convert to using run_script
This gives a 2-3x speedup on the test with the default run_mode=1.
2019-11-16t/edit: switch to use run_script
Perl parsing is slow, and run_script default behavior allows this to speed up t/edit.t by over 100% in my case.