Date | Commit message (Collapse) |
|
Use the "-q" flag like everywhere else.
|
|
The "strict" pragma makes code easier to debug, and we had
undeclared variables as a result in t/watch_maildir_v2.t.
So use it everywhere to be consistent with the rest of our
code.
|
|
There were still a few places where we used worker processes
unnecessarily in tests, causing a small amount of unnecessary
overhead.
Followup-to: ad221e9b2852f6c5 ("t/*.t: disable nntpd/httpd worker processes in most tests")
|
|
This more than doubles the speed of the test, since we make
many invocations of -xcpdb.
|
|
This more than doubles the speed of these tests
|
|
This more than doubles the speed of the test.
|
|
This nets us a 20% speedup or so.
|
|
This only gives a 5% speedup or so, but anything helps.
|
|
This only gives a small 10% speedup or so, but anything helps.
|
|
This only gives a small ~10% speedup, since -httpd still
needs execve, but any speedup is welcome.
|
|
While this didn't use IPC::Run, having to reload several Perl
modules and scripts is slow and inefficient, so roughly
double the speed of this test.
|
|
We need to be careful and explicitly close FDs before doing
-index, since we can't rely on FD_CLOEXEC without execve(2)
syscalls.
|
|
This test runs more than twice as fast, now.
|
|
Another noticeable speedup, this test is roughly ~3x faster now.
|
|
Not taking advantage of faster run modes in run_script, yet
since some lifetime problems need to be sorted.
|
|
This nets us another sizeable speedup.
|
|
This gives a 2-3x speedup on the test with the default
run_mode=1.
|
|
Perl parsing is slow, and run_script default behavior allows
this to speed up t/edit.t by over 100% in my case.
|
|
This will give us a consistent interface for running
test scripts in more performant ways while still giving
us a consistent interface to recreate real-world behavior
via spawn() (fork + execve), if needed.
The default run_mode (1) is faster and can run within the test
process with some minor adjustments to our code to avoid global
state.
This avoids the significante overhead of Perl code loading,
parsing and compilation phases.
|
|
We may not implicitly load it via other means in the future.
|
|
unix_server() is not commonly used, only t/httpd-corner.t uses
it and most HTTP tests use TCP since most HTTP libraries only
support TCP.
|
|
We only use it in one place and have favored test_psgi
in newer tests, so move it out-of-the-way to reduce startup
overhead of other *.t files.
|
|
And explicitly test for respawning in t/httpd-corner.t
There's no need to have an extra entries in the process table
for most tests we run, since that's not what we're testing.
|
|
We already load PublicInbox::Spawn for which(), so using spawn()
isn't unreasonable. And rely on "skip" to log the omitted test
if w3m is missing, which means we need to update the "&&"
escaping test to be self-referential on the same line.
File::Temp was totally unused, there; and we can use "open ...,undef"
in Perl to easily create anonymous temporary files for use with
spawn().
|
|
We already load PublicInbox::Spawn, so there's no need to
add another dependency to make life difficult for potential
contributors.
|
|
We don't need to force byte semantics for a buffer we clearly
create (via ->read) with byte semantics. Since we didn't
"use bytes" in t/httpd-corner.t, it was inadvertantly made
available by IPC::Run (which goes away, next).
|
|
One small step towards making tests easier-to-run. We can rely
on "local $ENV{GIT_DIR}" for potentially shell-unsafe path
names, and the rest of our path names are relative and don't
contain characters which require escaping.
|
|
Another case where spaces can be in TMPDIR and cause
shell expansion with `command` to fail.
|
|
It's possible (but unlikely) a user will put spaces in TMPDIR
and cause File::Temp::tempdir() to return a temporary directory
with spaces in the filename, making it unsafe for shell
expansion.
PublicInbox::Git didn't exist when t/mda.t was written, and
I just forgot about PublicInbox::Git->qx for t/plack.t :x
|
|
curl(1) can fail and we need to invalidate the test in the
rare case it fails.
|
|
While I've never seen "git log" fail on its own, it could happen
one day and we should be prepared to abort indexing when it
happens.
Beef up tests for t/spawn.t to ensure close() behaves
on popen_rd the way we expect it to.
|
|
While testing 216light.css changes, I managed to hit some cases
where dillo failed to render ' correctly, but I also can't
reproduce it reliably. Anyways, it's definitely a problem with
some old browsers and newer versions of highlight already work
around it, but Debian 10.x has 3.41, so use "'" to maximize
compatibility.
|
|
ISO-2202-JP and other non-UTF-8 messages need to be displayed
correctly.
Fixes: 7d82a8bc04ce ('handle "multipart/mixed" messages which are not multipart')
|
|
* learn:
doc: add public-inbox-learn(1) manpage
mda: support multiple List-ID matches
mda: prepare for multiple destinations
inboxwritable: add assert_usable_dir sub
mda: skip MIME parsing if spam
mda: hoist out mda_filter_adjust
filter/base: remove MAX_MID_SIZE constant
mda: hoist out List-ID handling and reuse in -learn
learn: hoist out remove_or_add subroutine
learn: GIT_COMMITTER_<NAME|EMAIL> may be "" or "0"
learn: update usage statement
learn: only map recipient list on "ham" or "rm"
learn: support multiple To/Cc headers
|
|
While it's not RFC2919-conformant, mail software can
theoretically set multiple List-ID headers. Deliver to all
inboxes which match a given List-ID since that's likely the
intended.
Cc: Eric W. Biederman <ebiederm@xmission.com>
Link: https://public-inbox.org/meta/87pniltscf.fsf@x220.int.ebiederm.org/
|
|
And use it for mda, since "0" could be a usable directory
if somebody insists on using relative paths...
|
|
Since we replace extra Message-ID headers with X-Alt-Message-ID
to placate NNTP clients, we should allow searching and indexing
on X-Alt-Message-ID just like we do with Message-ID.
|
|
Browsers may underline '<' and '>' in links, which may be
confused with '≤' and '≥'. So have the Message-ID header
display follow what we do with In-Reply-To headers and move the
"<" and ">" outside of <a> in the HTML.
|
|
We can easily support searching on messages with
multiple From/To/Cc/Subject headers just like we
do with multiple Message-ID headers.
This matches the normal mutt pager display behavior.
|
|
And maybe 8-headered ones, too...
I noticed --reindex failing on the linux-renesas-soc mirror due
one 3-headed monster of a message having 3 sets of headers;
while another normal message had a Message-ID that matched one
of the 3 IDs of the 3-headed monster.
We still try to do the majority of indexing backwards, but we
defer indexing multi-Message-ID'd messages until the end to
ensure we get all the "good" messages in before we process the
multi-headered ones.
Link: https://public-inbox.org/meta/20191016211415.GA6084@dcvr/
|
|
* origin/inboxdir:
config: remove redundant inboxdir check
config: support "inboxdir" in addition to "mainrepo"
examples/grok-pull.post_update_hook: use "inbox_dir"
|
|
"mainrepo" ws a bad name and artifact from the early days when I
intended for there to be a "spamrepo" (now just the
ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be
especially confusing, since v2 needs at least two git
repositories (epoch + all.git) to function and we shouldn't
confuse users by having them point to a git repository for v2.
Much of our documentation already references "INBOX_DIR" for
command-line arguments, so use "inboxdir" as the
git-config(1)-friendly variant for that.
"mainrepo" remains supported indefinitely for compatibility.
Users may need to revert to old versions, or may be referring
to old documentation and must not be forced to change config
files to account for this change.
So if you're using "mainrepo" today, I do NOT recommend changing
it right away because other bugs can lurk.
Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
|
|
Since -mda now supports List-ID to better support mirroring of
existing mailing lists, it probably makes sense to support
disabling the precheck function to provide more accurate (though
potentially spammier) mirrors of lists
|
|
This also adds watchheader tests for -watch, which we never
had before :x
|
|
Rewrite a bunch of tests to use ordered input (emulating
"git config -l" output) so we can always walk sections in
the order they were given in the config file.
|
|
We want to ensure we run lsof(8) on the worker (if needed),
and not the master, which doesn't serve requests.
This was originally on top of a test-only patch in
https://public-inbox.org/meta/20190913015043.17149-1-e@80x24.org/
In any case, no point in spawning extra processes for this test.
|
|
First, we use flock(2) to wait on parallel public-inbox-init(1)
invocations while we make multiple changes using git-config(1).
This flock allows -init processes to wait on each other if using
reasonable POSIX filesystems.
Then, we also need a git-config(1)-compatible lock to prevent
user-invoked git-config(1) processes from clobbering our
changes while we're holding the flock.
|
|
Since I intend to add support for --skip-artnum, disambiguating
the long option name makes sense. We'll support --skip
indefinitely for compatibility.
|
|
We can save future testers some time if we bail out early
on "git init --shared" failures, since things like seccomp
or non-POSIX FSes would trigger failures.
BAIL_OUT has been in Test::Simple since Perl v5.10.0, so it's
old-enough to call for our purposes.
Thanks-to: Alyssa Ross <hi@alyssa.is>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Alyssa Ross <hi@alyssa.is>
Link: https://public-inbox.org/meta/878sq2hd08.fsf@alyssa.is/
|
|
This ought to make permissions errors on odd systems
easier to diagnose in the future.
|