about summary refs log tree commit homepage
path: root/t
DateCommit message (Collapse)
2019-11-24t/common: start_script replaces spawn_listener
We can shave several hundred milliseconds off tests which spawn daemons by preloading and avoiding startup time for common modules which are already loaded in the parent process. This also gives ENV{TAIL} support to all tests which support daemons which log to stdout/stderr.
2019-11-24t/nntpd-tls: sometimes SSL_connect succeeds quickly
It seems caching can happen within OpenSSL or negotiation can be delayed in some cases. In any case, don't barf on PublicInbox::TLS::epollbit() when connect_SSL succeeds unexpectedly.
2019-11-24t/httpd-corner: wait for worker process death
We need to ensure the worker process is terminated before starting a new connection, so leave a persistent HTTP/1.1 connection open and wait for the SIGKILL to take effect and drop the client.
2019-11-24t/xcpdb-reshard: test xcpdb --compact
We did not have a test for this, and need to guard against regressions when changing Xapcmd to use File::Temp->newdir in future commits.
2019-11-24t/v1-add-remove-add: quiet down "git init"
Use the "-q" flag like everywhere else.
2019-11-24tests: use strict everywhere
The "strict" pragma makes code easier to debug, and we had undeclared variables as a result in t/watch_maildir_v2.t. So use it everywhere to be consistent with the rest of our code.
2019-11-24tests: disable daemon workers in a few more places
There were still a few places where we used worker processes unnecessarily in tests, causing a small amount of unnecessary overhead. Followup-to: ad221e9b2852f6c5 ("t/*.t: disable nntpd/httpd worker processes in most tests")
2019-11-16t/xcpdb-reshard: use run_script for -xcpdb
This more than doubles the speed of the test, since we make many invocations of -xcpdb.
2019-11-16t/indexlevels-mirror*: switch to run_script
This more than doubles the speed of these tests
2019-11-16t/v2mda: switch to run_script in many places
This more than doubles the speed of the test.
2019-11-16t/watch_filter_rubylang: run_script for -init and -index
This nets us a 20% speedup or so.
2019-11-16t/nntpd: use run_script for -init
This only gives a 5% speedup or so, but anything helps.
2019-11-16t/watch_maildir_v2: use run_script for -init
This only gives a small 10% speedup or so, but anything helps.
2019-11-16t/httpd: use run_script for -init
This only gives a small ~10% speedup, since -httpd still needs execve, but any speedup is welcome.
2019-11-16t/convert-compact: convert to run_script
While this didn't use IPC::Run, having to reload several Perl modules and scripts is slow and inefficient, so roughly double the speed of this test.
2019-11-16t/v2mirror: switch to default run_mode for speedup
We need to be careful and explicitly close FDs before doing -index, since we can't rely on FD_CLOEXEC without execve(2) syscalls.
2019-11-16t/mda_filter_rubylang: drop IPC::Run dependency
This test runs more than twice as fast, now.
2019-11-16t/mda: switch to run_script for testing
Another noticeable speedup, this test is roughly ~3x faster now.
2019-11-16t/v2mirror: get rid of IPC::Run dependency
Not taking advantage of faster run modes in run_script, yet since some lifetime problems need to be sorted.
2019-11-16t/purge: convert to run_script
This nets us another sizeable speedup.
2019-11-16t/init: convert to using run_script
This gives a 2-3x speedup on the test with the default run_mode=1.
2019-11-16t/edit: switch to use run_script
Perl parsing is slow, and run_script default behavior allows this to speed up t/edit.t by over 100% in my case.
2019-11-16t/common: introduce run_script wrapper for t/cgi.t
This will give us a consistent interface for running test scripts in more performant ways while still giving us a consistent interface to recreate real-world behavior via spawn() (fork + execve), if needed. The default run_mode (1) is faster and can run within the test process with some minor adjustments to our code to avoid global state. This avoids the significante overhead of Perl code loading, parsing and compilation phases.
2019-11-15t/psgi_mount: require SearchIdx before using
We may not implicitly load it via other means in the future.
2019-11-15t/common: move unix_server to t/httpd-corner.t
unix_server() is not commonly used, only t/httpd-corner.t uses it and most HTTP tests use TCP since most HTTP libraries only support TCP.
2019-11-15t/common: inline stream_to_string into t/feed.t
We only use it in one place and have favored test_psgi in newer tests, so move it out-of-the-way to reduce startup overhead of other *.t files.
2019-11-08t/*.t: disable nntpd/httpd worker processes in most tests
And explicitly test for respawning in t/httpd-corner.t There's no need to have an extra entries in the process table for most tests we run, since that's not what we're testing.
2019-11-08t/hl_mod.t: remove IPC::Run (and File::Temp) dependency
We already load PublicInbox::Spawn for which(), so using spawn() isn't unreasonable. And rely on "skip" to log the omitted test if w3m is missing, which means we need to update the "&&" escaping test to be self-referential on the same line. File::Temp was totally unused, there; and we can use "open ...,undef" in Perl to easily create anonymous temporary files for use with spawn().
2019-11-08t/httpd-corner.t: get rid of IPC::Run for running curl
We already load PublicInbox::Spawn, so there's no need to add another dependency to make life difficult for potential contributors.
2019-11-08t/httpd-corner.t: drop unnecessary bytes:: for length()
We don't need to force byte semantics for a buffer we clearly create (via ->read) with byte semantics. Since we didn't "use bytes" in t/httpd-corner.t, it was inadvertantly made available by IPC::Run (which goes away, next).
2019-11-08t/*.t: remove IPC::Run dependency for git commands
One small step towards making tests easier-to-run. We can rely on "local $ENV{GIT_DIR}" for potentially shell-unsafe path names, and the rest of our path names are relative and don't contain characters which require escaping.
2019-11-04t/edit: use PublicInbox::Git::qx for pathname safety
Another case where spaces can be in TMPDIR and cause shell expansion with `command` to fail.
2019-11-04tests: rely on PublicInbox::Git for pathname safety
It's possible (but unlikely) a user will put spaces in TMPDIR and cause File::Temp::tempdir() to return a temporary directory with spaces in the filename, making it unsafe for shell expansion. PublicInbox::Git didn't exist when t/mda.t was written, and I just forgot about PublicInbox::Git->qx for t/plack.t :x
2019-11-04t/httpd-corner.t: check for curl(1) errors in big async test
curl(1) can fail and we need to invalidate the test in the rare case it fails.
2019-11-04index: "git log" failures are fatal
While I've never seen "git log" fail on its own, it could happen one day and we should be prepared to abort indexing when it happens. Beef up tests for t/spawn.t to ensure close() behaves on popen_rd the way we expect it to.
2019-10-31hval: replace "'" with "'" for compatibility
While testing 216light.css changes, I managed to hit some cases where dillo failed to render ' correctly, but I also can't reproduce it reliably. Anyways, it's definitely a problem with some old browsers and newer versions of highlight already work around it, but Debian 10.x has 3.41, so use "'" to maximize compatibility.
2019-10-31msgiter: do not assume UTF-8 if Email::MIME->body_str succeeds
ISO-2202-JP and other non-UTF-8 messages need to be displayed correctly. Fixes: 7d82a8bc04ce ('handle "multipart/mixed" messages which are not multipart')
2019-10-30Merge branch 'learn'
* learn: doc: add public-inbox-learn(1) manpage mda: support multiple List-ID matches mda: prepare for multiple destinations inboxwritable: add assert_usable_dir sub mda: skip MIME parsing if spam mda: hoist out mda_filter_adjust filter/base: remove MAX_MID_SIZE constant mda: hoist out List-ID handling and reuse in -learn learn: hoist out remove_or_add subroutine learn: GIT_COMMITTER_<NAME|EMAIL> may be "" or "0" learn: update usage statement learn: only map recipient list on "ham" or "rm" learn: support multiple To/Cc headers
2019-10-30mda: support multiple List-ID matches
While it's not RFC2919-conformant, mail software can theoretically set multiple List-ID headers. Deliver to all inboxes which match a given List-ID since that's likely the intended. Cc: Eric W. Biederman <ebiederm@xmission.com> Link: https://public-inbox.org/meta/87pniltscf.fsf@x220.int.ebiederm.org/
2019-10-30inboxwritable: add assert_usable_dir sub
And use it for mda, since "0" could be a usable directory if somebody insists on using relative paths...
2019-10-28index: allow search/lookups on X-Alt-Message-ID
Since we replace extra Message-ID headers with X-Alt-Message-ID to placate NNTP clients, we should allow searching and indexing on X-Alt-Message-ID just like we do with Message-ID.
2019-10-28view: move '<' and '>' outside <a>
Browsers may underline '<' and '>' in links, which may be confused with '≤' and '≥'. So have the Message-ID header display follow what we do with In-Reply-To headers and move the "&lt;" and "&gt;" outside of <a> in the HTML.
2019-10-28search: support multiple From/To/Cc/Subject headers
We can easily support searching on messages with multiple From/To/Cc/Subject headers just like we do with multiple Message-ID headers. This matches the normal mutt pager display behavior.
2019-10-21v2writable: reindex handles 3-headered monsters
And maybe 8-headered ones, too... I noticed --reindex failing on the linux-renesas-soc mirror due one 3-headed monster of a message having 3 sets of headers; while another normal message had a Message-ID that matched one of the 3 IDs of the 3-headed monster. We still try to do the majority of indexing backwards, but we defer indexing multi-Message-ID'd messages until the end to ensure we get all the "good" messages in before we process the multi-headered ones. Link: https://public-inbox.org/meta/20191016211415.GA6084@dcvr/
2019-10-17Merge remote-tracking branch 'origin/inboxdir'
* origin/inboxdir: config: remove redundant inboxdir check config: support "inboxdir" in addition to "mainrepo" examples/grok-pull.post_update_hook: use "inbox_dir"
2019-10-16config: support "inboxdir" in addition to "mainrepo"
"mainrepo" ws a bad name and artifact from the early days when I intended for there to be a "spamrepo" (now just the ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be especially confusing, since v2 needs at least two git repositories (epoch + all.git) to function and we shouldn't confuse users by having them point to a git repository for v2. Much of our documentation already references "INBOX_DIR" for command-line arguments, so use "inboxdir" as the git-config(1)-friendly variant for that. "mainrepo" remains supported indefinitely for compatibility. Users may need to revert to old versions, or may be referring to old documentation and must not be forced to change config files to account for this change. So if you're using "mainrepo" today, I do NOT recommend changing it right away because other bugs can lurk. Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
2019-10-16mda: support --no-precheck option
Since -mda now supports List-ID to better support mirroring of existing mailing lists, it probably makes sense to support disabling the precheck function to provide more accurate (though potentially spammier) mirrors of lists
2019-10-15mda, watch: wire up List-ID header support
This also adds watchheader tests for -watch, which we never had before :x
2019-10-15config: we always have {-section_order}
Rewrite a bunch of tests to use ordered input (emulating "git config -l" output) so we can always walk sections in the order they were given in the config file.
2019-10-10t/git-http-backend: disable worker processes
We want to ensure we run lsof(8) on the worker (if needed), and not the master, which doesn't serve requests. This was originally on top of a test-only patch in https://public-inbox.org/meta/20190913015043.17149-1-e@80x24.org/ In any case, no point in spawning extra processes for this test.