about summary refs log tree commit homepage
path: root/t/v2mirror.t
DateCommit message (Collapse)
2019-12-24testcommon: add require_mods method and use it
This cuts down on lines of code in individual test cases and fixes some misnamed error messages by using "$0" consistently. This will also provide us with a method of swapping out dependencies which provide equivalent functionality (e.g "Xapian" SWIG can replace "Search::Xapian" XS bindings).
2019-12-19tests: move t/common.perl to PublicInbox::TestCommon
We want to be able to use run_script with *.t files, so t/common.perl putting subs into the top-level "main" namespace won't work. Instead, make it a module which uses Exporter like other libraries.
2019-12-19t/*.t: avoid sharing "my" variables in subs
These usages of file-local global variables make the *.t files incompatible with run_script(). Instead, use anonymous subs, "our", or pass the parameter as appropriate.
2019-11-24tests: quiet down commit graph
Newer versions of git enable the commit graph by default. Since we blow away our temporary directories every test, generating graphis is a waste and clutters stderr with "Computing commit graph generation numbers" messages.
2019-11-24tests: use File::Temp->newdir instead of tempdir()
We'll also introduce a tmpdir() API to give tempdirs consistent names.
2019-11-24t/common: start_script replaces spawn_listener
We can shave several hundred milliseconds off tests which spawn daemons by preloading and avoiding startup time for common modules which are already loaded in the parent process. This also gives ENV{TAIL} support to all tests which support daemons which log to stdout/stderr.
2019-11-16t/v2mirror: switch to default run_mode for speedup
We need to be careful and explicitly close FDs before doing -index, since we can't rely on FD_CLOEXEC without execve(2) syscalls.
2019-11-16t/v2mirror: get rid of IPC::Run dependency
Not taking advantage of faster run modes in run_script, yet since some lifetime problems need to be sorted.
2019-11-08t/*.t: disable nntpd/httpd worker processes in most tests
And explicitly test for respawning in t/httpd-corner.t There's no need to have an extra entries in the process table for most tests we run, since that's not what we're testing.
2019-10-16config: support "inboxdir" in addition to "mainrepo"
"mainrepo" ws a bad name and artifact from the early days when I intended for there to be a "spamrepo" (now just the ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be especially confusing, since v2 needs at least two git repositories (epoch + all.git) to function and we shouldn't confuse users by having them point to a git repository for v2. Much of our documentation already references "INBOX_DIR" for command-line arguments, so use "inboxdir" as the git-config(1)-friendly variant for that. "mainrepo" remains supported indefinitely for compatibility. Users may need to revert to old versions, or may be referring to old documentation and must not be forced to change config files to account for this change. So if you're using "mainrepo" today, I do NOT recommend changing it right away because other bugs can lurk. Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
2019-09-09run update-copyrights from gnulib for 2019
2019-06-30tests: common tcp_server and unix_server helpers
IO::Socket:*->new options are verbose and we can save a bunch of code by putting this into t/common.perl, since the related spawn_listener stuff is already there.
2019-06-09www: support $INBOX/git/$EPOCH.git for v2 cloning
And use it in manifest.js. To ease maintaining mirrors with grokmirror(1), we can accept a "git/" directory prefix before the epoch, and ".git" suffix after the epoch number. We maintain compatibility with "$INBOX/$EPOCH" cloning, of course, and it's still easier-to-type on the command-line.
2019-05-23v1writable: retire in favor of InboxWritable
In retrospect, introducing V1Writable was unnecessary and InboxWritable->importer is in a better position to abstract away differences between v1 and v2 writers. So teach InboxWritable to initialize inboxes and get rid of V1Writable.
2019-05-14tests: remove unnecessary loading of ::DS and Socket
PublicInbox::DS works for every platform we we care about, nowadays; so checking for it is a waste of time. Cleanup a few POSIX and Socket imports while we're in the area.
2019-05-04bundle Danga::Socket and Sys::Syscall
These modules are unmaintained upstream at the moment, but I'll be able to help with the intended maintainer once/if CPAN ownership is transferred. OTOH, we've been waiting for that transfer for several years, now... Changes I intend to make: * EPOLLEXCLUSIVE for Linux * remove unused fields wasting memory * kqueue bugfixes e.g. https://rt.cpan.org/Ticket/Display.html?id=116615 * accept4 support And some lower priority experiments: * switch to EV_ONESHOT / EPOLLONESHOT (incompatible changes) * nginx-style buffering to tmpfile instead of string array * sendfile off tmpfile buffers * io_uring maybe?
2019-01-10check git version requirements
This allows v1 tests to continue working on git 1.8.0 for now. This allows git 2.1.4 packaged with Debian 8 ("jessie") to run old tests, at least. I suppose it's safe to drop Debian 7 ("wheezy") due to our dependency on git 1.8.0 for "merge-base --is-ancestor". Writing V2 repositories requires git 2.6 for "get-mark" support, so mask out tests for older gits.
2018-12-29tests: consolidate process spawning code.
IPC::Run provides a nice simplification in several places; and we already use it (optionally) on a lot of tests. For the non-test code, we still rely on our vfork-capable Inline::C stuff since real-world server processes can get large enough to where vfork is an advantage. Maybe Perl5 can use CLONE_VFORK somehow, one day: https://rt.perl.org/Ticket/Display.html?id=128227 Ohg V'q engure cbeg choyvp-vaobk gb Ehol :C
2018-07-15v2writable: unindex deleted messages after incremental fetch
The normal behavior is to prevent the deleted messages from being indexed in the first place. However, when fetching incrementally via git; public-inbox-index needs to account for deleted files which were created outside of the most recent fetch/reindexing window. Reported-by: Eric W. Biederman <ebiederm@xmission.com>
2018-07-04v2: fill alternates with old epochs on init from mirrors
For v2 repositories with multiple epochs, we must not forget about earlier epochs in clones. Ensure we update the alternates file with all known epochs up to the current one. Reported-by: Eric W. Biederman <ebiederm@xmission.com> https://public-inbox.org/meta/871scj2vzi.fsf@xmission.com/
2018-05-24workaround Xapian OFD locks w/o close-on-exec
Xapian v1.2.21..v1.2.24 (inclusive) use OFD locks but failed to set the close-on-exec flag on those locks. So we must continue to work around those old versions by ensuring Xapian file descriptors aren't held any longer than necessary when in long-running git processes. Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2018-04-06ensure Xapian and SQLite are still optional for v1 tests
Xapian is size-intensive and SQLite is not strictly necessary for v1.
2018-04-04v2: support incremental indexing + purge
This is important for people running mirrors via "git fetch", as they need to be kept up-to-date. Purging is also now supported in mirrors. The short-lived "--regenerate" option is gone and is now implicitly enabled as a result. It's still cheap when article number regeneration is unnecessary, as we track the range for each git repository.