about summary refs log tree commit homepage
path: root/t
DateCommit message (Collapse)
2020-01-27tests: move the majority of t/view.t into t/plack.t
And some more into t/mid.t. PublicInbox::View::msg_html may change internally, so lets rely on the stable PSGI interface to test it, rather than a test which reaches deep into the internals.
2020-01-27t/plack.t: modernize and unindent
This test will be expanded, and we can take advantage of run_script to simplify our internal API use.
2020-01-23hval: from_attr: move to unit test
We don't call from_attr anywhere outside of tests, so don't bloat normal processes with it.
2020-01-23hval: to_attr: support wide characters
We need to escape wide characters when making attribute names from filename-looking things in diffstats.
2020-01-13ds: add_timer: rename from AddTimer, remove a parameter
The class parameter is pointless, especially for an internal sub which only has one external caller in a test. Add a sub prototype while we're at it to get some compile time checking.
2020-01-13use popen_rd for bidirectional pipes
popen_rd accepts arbitrary redirects, so we can reuse its code to setup the pipe end we want to read, saving each caller a few lines of code compared to calling pipe+spawn.
2020-01-13t/solver_git: avoid uninitialized warnings in hostname generation
Outside of tests, this is only relevant for non-PSGI use, which may happen someday... Fixes: cb1c874520153f5c ("inbox: use PublicInbox::Git::host_prefix_url for base_url")
2020-01-11make Plack optional for non-WWW and non-httpd users
Some users just want to run -mda, -watch, and/or -nntpd. Let them run just those without forcing them to pull in a bunch of dependencies.
2020-01-11spawn (and thus popen_rd) die on failure
Most spawn and popen_rd callers die on failure to spawn, anyways, and some are missing checks entirely. This saves us a bunch of verbose error-checking code in callers. This also makes popen_rd more consistent, since it already dies on pipe creation failures.
2020-01-11git: remove ->commit_title method
We haven't used it in SolverGit, yet, and I'll be reworking it to work with ->cat_async, instead.
2020-01-11inbox: use PublicInbox::Git::host_prefix_url for base_url
Better not to duplicate the same logic across different classes. Also, our git wrapper class is a strange place for host_prefix_url, but it needs to be usable for coderepos, so it's there, for now...
2020-01-06treewide: "require" + "use" cleanup and docs
There's a bunch of leftover "require" and "use" statements we no longer need and can get rid of, along with some excessive imports via "use". IO::Handle usage isn't always obvious, so add comments describing why a package loads it. Along the same lines, document the tmpdir support as the reason we depend on File::Temp 0.19, even though every Perl 5.10.1+ user has it. While we're at it, favor "use" over "require", since it it gives us extra compile-time checking.
2020-01-06t/nntp.t: fix parse_time test for non-GMT local time
Yes, there's actually other timezones!
2020-01-05tests: remove some "git config" calls after "git init"
Creating a hash and iterating through it just to run "git config" is ugly and slow. Just write out the text file in a human-friendly way since the git-config file format is stable and won't break randomly.
2020-01-05search: remove lookup_article
It was no longer used outside of tests, so don't penalize regular users with the extra function. Just inline it for t/search.t.
2020-01-04tests: fix running without SQLite or Xapian
PublicInbox::Search always loads DBD::SQLite, so we can't blindly "use" it in t/xcpdb-reshard.t. We also need to account for that in TestCommon.
2020-01-02config: support multi-value inbox.*.*url
Since the beginning of this project, we've implicitly supported inboxes with multiple URLs by relying on the Host: header sent by the client ($env->{HTTP_HOST}). We now offer the option to explicitly configure multiple URLs for every inbox along with the ability to do a best-effort match for matching hostnames.
2020-01-01wwwstatic: add directory listing + index.html support
It's now possible to use WwwStatic as a standalone PSGI app to serve static files and recreate the award-winning web design of https://public-inbox.org/ :>
2020-01-01nntp: handle 2-digit year "70" properly
Time::Local has the concept of a "rolling century" which is defined at 50 years on either side of the current year. Since it's now 2020 and >50 years since the Unix epoch, the year "70" gets interpreted by Time::Local as 2070-01-01 instead of 1970-01-01. Since NNTP servers are unlikely to store messages from the future, we'll feed 4-digit year to Time::Local::{timegm,timelocal} and hopefully not have to worry about things until Y10K. This fixes test failures on t/v2writable.t and t/nntpd.t since 2020-01-01.
2019-12-30spawn: allow passing GLOB handles for redirects
We can save callers the trouble of {-hold} and {-dev_null} refs as well as the trouble of calling fileno().
2019-12-28solvergit: allow passing arg to user-supplied callback
This allows us to get rid of the requirement to capture on-stack variables with an anonymous sub, as illustrated with the update to viewvcs to take advantage of this. v2: fix error handling for missing OIDs
2019-12-26qspawn: psgi_return_start: hoist out from psgi_return
Instead of just passing the rpipe to the start_cb, pass the entire qspawn ref to start_cb. Update existing callers to avoid circular refs.
2019-12-26git: allow async_cat to pass arg to callback
This allows callers to avoid allocating several KB for for every call to ->async_cat.
2019-12-26t/solver_git: test with -httpd, too
Solver uses the internal -httpd async API if available for fairness when applying large patchsets. We must test those code paths in addition to the generic PSGI code paths.
2019-12-26t/www_listing: test "all" HTML listing
We now have coverage for PublicInbox::WwwListing::list_all.
2019-12-26t/www_listing: quiet down stderr in -httpd
We need to init all.git for the v2 repo test to ensure `git --git-dir=v2/all.git rev-parse --git-path objects/info/alternates` doesn't warn or fail and clutter stderr. This is noticeable when setting TAIL="tail -F" in env before running this test.
2019-12-25t/psgi_v2: test search results Atom feed endpoint
The "x=A" search results endpoint finally gets test coverage.
2019-12-24t/search.t: update permissions check for OpenBSD
OpenBSD (tested 6.5 on amd64) seems to follow the same semantics as FreeBSD for S_ISGID, even if config.mak.uname in git.git doesn't say so.
2019-12-24search: support SWIG-generated Xapian.pm
Xapian upstream is slowly phasing out the XS-based Search::Xapian in favor of the SWIG-generated "Xapian" package. While Debian and both FreeBSD have Search::Xapian, OpenBSD only includes the "Xapian" binding. More information about the status of the "Xapian" Perl module here: https://trac.xapian.org/ticket/523
2019-12-24testcommon: add require_mods method and use it
This cuts down on lines of code in individual test cases and fixes some misnamed error messages by using "$0" consistently. This will also provide us with a method of swapping out dependencies which provide equivalent functionality (e.g "Xapian" SWIG can replace "Search::Xapian" XS bindings).
2019-12-20view: show percentage in search results thread skeleton
The displays the Xapian ->get_percent value in the skeleton to improve scanning of relevancy; irrelevant results do not display that. This fixes broken #anchor links introduced in the previous commit, irrelevant messages now link to the /$INBOX/$MESSAGE_ID page.
2019-12-19t/run.perl: to avoid repeated process spawning for *.t
Spawning a new Perl interpreter for every test case means Perl has to reparse and recompile every single file it needs, costing us performance and development time. Now that we've modified our code to avoid global state, we can preload everything we need. The new "check-run" test target is now 20-30% faster than the original "check" target.
2019-12-19tests: move t/common.perl to PublicInbox::TestCommon
We want to be able to use run_script with *.t files, so t/common.perl putting subs into the top-level "main" namespace won't work. Instead, make it a module which uses Exporter like other libraries.
2019-12-19t/*.t: avoid sharing "my" variables in subs
These usages of file-local global variables make the *.t files incompatible with run_script(). Instead, use anonymous subs, "our", or pass the parameter as appropriate.
2019-12-18viewvcs: flesh out some functionality and test
Expose MAX_SIZE via "our" will make it possible to use in tests, and configure, later. Additionally, returning HTTP 500 code for big files is not an Internal Server Error, just a memory limit... Some browsers won't show our HTML response with the link to the raw file in case of errors, either, so we'll return 200 to ensure users can use the link to access the raw blob. Finally, throw in some tests to the existing solver_git testcase, since that was incomplete and was pointlessly loading Plack modules without testing PSGI.
2019-12-17t/edit.t: drop redundant "delete local $ENV{...}"
"delete local" is only in Perl v5.11.0, and we only depend on Perl v5.10.1. We already localize and delete it as two separate statements immediately above. I wish this was hidden behind a "use feature" flag like other new-fangled things :<. Oh well, I think the oldest Perl actually in use for this project is 5.16 (CentOS 7.x).
2019-12-15address: explicitly reject local-only addresses
Apparently, neither our previous address parsing code nor Email::Address::XS recognizes local, username-only addresses in the form of <username> (without "@host"). Without this change, Email::Address::XS->address would return "undef", so we need to filter it out via "grep { defined }" It seems the cases where users email each other on the same machine is small and public-inbox won't be able to index addresses for those cases... Oh well :/
2019-12-15address: use Email::Address::XS if available
Email::Address::XS is a dependency of modern versions of Email::MIME, so it's likely loaded and installed on newer systems, already; and capable of handling more corner-cases than our pure-Perl fallback. We still fallback to the imperfect-but-good-enough-in-practice pure-Perl code while avoiding the non-XS Email::Address (which was susceptible to DoS attacks (CVE-2015-7686)). We just need to keep "git fast-import" happy.
2019-12-15address: use comment as name if no phrase available
Some users will set their From: headers in the form of: "<user@example.com> (A U Thor)", where their name is in the parenthesized comment. Use that instead of the email address, if available.
2019-12-14daemon: use DESTROY for unlinking --pid-file
This gets rid of the last "END{}" block in our code and cleans up a (temporary) circular reference. Furthermore, ensure the cleanup code still works in all configurations by adding tests and testing both the -W1 (default, 1 worker) and -W0 (no workers) code paths.
2019-12-12msgtime: avoid obviously out-of-range dates (for now)
Wacky dates show up in lore for valid messages. Lets ignore them and let future generations deal with Y10K and time-travel problems.
2019-12-12msgtime: drop Date::Parse for RFC2822
Date::Parse is not optimized for RFC2822 dates and isn't packaged on OpenBSD. It's still useful for historical email when email clients were less conformant, but is less relevant for new emails.
2019-12-11spawn: remove support for clearing the env
It's unnecessary code which I'm not sure we ever used. In retrospect, completely clearing the environment doesn't make sense for the processes we spawn. We don't need to clobber individual environment variables in our code, either (and if we did for tests, we can use 'local').
2019-12-11t/common: set $0 when running script w/o fork
We can localize changes to $0 so $0 is restored when the "script" sub is done. This will be helpful when we encounter a stuck/slow processes during our tests (hopefully never!)
2019-12-11t: localize the PI_CONFIG env
We don't want the user's ~/.public-inbox/config to be read from during tests. I only noticed this because I had a non-existent pathname for one of my inboxes :x I've also verified this change by running "inotifywait ~/.public-inbox/config -m" in another terminal while running "make check"; (perhaps a portable solution could make it into the test suite).
2019-11-29t/replace: quiet "git fsck" invocation
Test output can be a terminal if running as "perl -I lib t/$FOO.t", and showing fsck progress is pointless for tests.
2019-11-28t/httpd-unix: FreeBSD expects to fail with EADDRINUSE
Tested FreeBSD 11.2. I'm starting to think I'm too conservative with this check and it could be safely expanded to cover any OS with UNIX sockets.
2019-11-27httpd|nntpd: avoid missed signal wakeups
Our attempt at using a self-pipe in signal handlers was ineffective, since pure Perl code execution is deferred and Perl doesn't use an internal self-pipe/eventfd. In retrospect, I actually prefer the simplicity of Perl in this regard... We can use sigprocmask() from Perl, so we can introduce signalfd(2) and EVFILT_SIGNAL support on Linux and *BSD-based systems, respectively. These OS primitives allow us to avoid a race where Perl checks for signals right before epoll_wait() or kevent() puts the process to sleep. The (few) systems nowadays without signalfd(2) or IO::KQueue will now see wakeups every second to avoid missed signals.
2019-11-27dskqxs: fix missing EV_DISPATCH define
Oops, IO::KQueue support was broken due to this missing constant. Add a new ds-kqxs.t test case to ensure we test the IO::KQueue path if IO::KQueue is available.
2019-11-27msgtime: deal with strange minutes in TZ offsets
I'm not sure if TZ minute offsets aside from '00' or '30' exist, but lets just deal with them properly when negative. Examples taken from various inboxes on lore.kernel.org. These are mostly message from spammers, but some are legitimate messages.