Date | Commit message (Collapse) |
|
We don't want to propagate %SIG changes to other tests when
running multiple tests within the same process via t/run.perl.
|
|
It was implemented at some point, but it was more things to
support and the worst of both worlds: both unrealistic compared
to real-world use and slower than run_mode=2.
Noticed while looking for speling erorrs.
|
|
I didn't wait until September to do it, this year!
|
|
This cuts down on lines of code in individual test cases and
fixes some misnamed error messages by using "$0" consistently.
This will also provide us with a method of swapping out
dependencies which provide equivalent functionality (e.g
"Xapian" SWIG can replace "Search::Xapian" XS bindings).
|
|
Spawning a new Perl interpreter for every test case
means Perl has to reparse and recompile every single file
it needs, costing us performance and development time.
Now that we've modified our code to avoid global state,
we can preload everything we need.
The new "check-run" test target is now 20-30% faster
than the original "check" target.
|
|
We want to be able to use run_script with *.t files, so
t/common.perl putting subs into the top-level "main" namespace
won't work. Instead, make it a module which uses Exporter
like other libraries.
|
|
We'll also introduce a tmpdir() API to give tempdirs
consistent names.
|
|
We can shave several hundred milliseconds off tests which spawn
daemons by preloading and avoiding startup time for common
modules which are already loaded in the parent process.
This also gives ENV{TAIL} support to all tests which support
daemons which log to stdout/stderr.
|
|
There were still a few places where we used worker processes
unnecessarily in tests, causing a small amount of unnecessary
overhead.
Followup-to: ad221e9b2852f6c5 ("t/*.t: disable nntpd/httpd worker processes in most tests")
|
|
This only gives a 5% speedup or so, but anything helps.
|
|
"mainrepo" ws a bad name and artifact from the early days when I
intended for there to be a "spamrepo" (now just the
ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be
especially confusing, since v2 needs at least two git
repositories (epoch + all.git) to function and we shouldn't
confuse users by having them point to a git repository for v2.
Much of our documentation already references "INBOX_DIR" for
command-line arguments, so use "inboxdir" as the
git-config(1)-friendly variant for that.
"mainrepo" remains supported indefinitely for compatibility.
Users may need to revert to old versions, or may be referring
to old documentation and must not be forced to change config
files to account for this change.
So if you're using "mainrepo" today, I do NOT recommend changing
it right away because other bugs can lurk.
Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
|
|
|
|
IO::Socket::INET->new is rather verbose with the options hash,
extract it into a standalone sub
|
|
RFC3977 6.1.2.2 LISTGROUP allows a [range] arg after [group],
and supporting it allows NNTP support in neomutt to work again.
Tested with NeoMutt 20170113 (1.7.2) on Debian stretch
(oldstable)
|
|
RFC3977 8.4.2 mandates the order of non-standard headers
to be after the first seven standard headers/metadata;
so "Xref:" must appear after "Lines:"|":lines".
Additionally, non-required header names must be followed
by ":full".
Cc: Jonathan Corbet <corbet@lwn.net>
Reported-by: Urs Janßen
<E1hmKBw-0008Bq-8t@akw>
|
|
This is only tested so far with my patches to Net::NNTP at:
https://rt.cpan.org/Ticket/Display.html?id=129967
Memory use in C10K situations is disappointing, but that's
the nature of compression.
gzip compression over HTTPS does have the advantage of not
keeping zlib streams open when clients are idle, at the
cost of worse compression.
|
|
Net::NNTP won't attempt to use older versions of IO::Socket::SSL
because 2.007 is the "first version with default CA on most platforms"
according to comments in Net::NNTP. But then again we don't make
remote requests when testing...
|
|
Some clients may rely on this for STARTTLS support.
|
|
IO::Socket:*->new options are verbose and we can save
a bunch of code by putting this into t/common.perl,
since the related spawn_listener stuff is already there.
|
|
Perl prior to 5.22 did not bundle a Net::NNTP (or libnet)
capable of handling TLS.
|
|
It kinda, barely works, and I'm most happy I got it working
without any modifications to the main NNTP::event_step callback
thanks to the DS->write(CODE) support we inherited from
Danga::Socket.
|
|
PublicInbox::Inbox objects have minimal dependencies, so
drop code to support old tests which existed before the
PublicInbox::Inbox object came into existence.
|
|
It's the unfortunate reality that there are some clients which
reuse Message-IDs (in which we generate + use another) or set
multiple Message-IDs on their own. While the v2 format
addresses that, NNTP clients such as leafnode are not always
prepared to deal with that case.
So, ensure NNTP clients only see a single Message-ID, and
show the others as 'X-Alt-Message-ID'.
|
|
Leafnode cannot handle Message-ID headers which are too long and
require folding via Email::Simple::Header. Since there are
already many of these messages in git with the header already
folded, we need to handle the unfolding when emitting the
message via NNTP.
As far as we know, Leafnode is the only client software
incapable of handling this case.
|
|
The version of Test::More from Perl 5.10.1 did not support
"subtest", and the earliest version which did is Perl 5.12.0
The good news is this gives me an excuse to parallelize
the indexlevels-mirror test by splitting it into two.
(it could be further split, even).
Update t/nntpd. to use PI_TEST_VERSION consistently while
we're at it.
|
|
In retrospect, introducing V1Writable was unnecessary and
InboxWritable->importer is in a better position to abstract
away differences between v1 and v2 writers.
So teach InboxWritable to initialize inboxes and get rid
of V1Writable.
|
|
More tests work without Search::Xapian, now.
Usability issues still need to be fixed
|
|
We only need it for tests that chdir, and maybe for ENV{PATH}
portability (dash seems fine, not sure about others).
v2: revert change to solver_git.t for FreeBSD 11.2 and document
|
|
PublicInbox::DS works for every platform we we care about,
nowadays; so checking for it is a waste of time. Cleanup a
few POSIX and Socket imports while we're in the area.
|
|
These modules are unmaintained upstream at the moment, but I'll
be able to help with the intended maintainer once/if CPAN
ownership is transferred. OTOH, we've been waiting for that
transfer for several years, now...
Changes I intend to make:
* EPOLLEXCLUSIVE for Linux
* remove unused fields wasting memory
* kqueue bugfixes e.g. https://rt.cpan.org/Ticket/Display.html?id=116615
* accept4 support
And some lower priority experiments:
* switch to EV_ONESHOT / EPOLLONESHOT (incompatible changes)
* nginx-style buffering to tmpfile instead of string array
* sendfile off tmpfile buffers
* io_uring maybe?
|
|
This allows v1 tests to continue working on git 1.8.0 for
now. This allows git 2.1.4 packaged with Debian 8 ("jessie")
to run old tests, at least.
I suppose it's safe to drop Debian 7 ("wheezy") due to our
dependency on git 1.8.0 for "merge-base --is-ancestor".
Writing V2 repositories requires git 2.6 for "get-mark"
support, so mask out tests for older gits.
|
|
IPC::Run provides a nice simplification in several places; and
we already use it (optionally) on a lot of tests.
For the non-test code, we still rely on our vfork-capable
Inline::C stuff since real-world server processes can get large
enough to where vfork is an advantage. Maybe Perl5 can use
CLONE_VFORK somehow, one day:
https://rt.perl.org/Ticket/Display.html?id=128227
Ohg V'q engure cbeg choyvp-vaobk gb Ehol :C
|
|
When a client starts pipelining requests to us which trigger
long responses, we need to keep socket readiness checks disabled
and only enable them when our socket rbuf is drained.
Failure to do this caused aborted clients with
"BUG: nested long response" when Danga::Socket calls event_read
for read-readiness after our "next_tick" sub fires in the
same event loop iteration.
Reported-by: Jonathan Corbet <corbet@lwn.net>
cf. https://public-inbox.org/meta/20181013124658.23b9f9d2@lwn.net/
|
|
Putting the Xref field into xover lines allows newsreaders to mark
cross-posted messages read when catching up a group. That, in turn,
massively improves the life of crazy people who try to follow dozens of
kernel lists, where emails are often heavily cross-posted.
|
|
RFC 5536 sec 3.2.14 says that the server-name in an Xref line is "which
news server generated the header field"; indeed, that is necessary for
newsreaders like gnus to handle references properly. So pick up the server
name from the config if available (the first name if there's more than
one), from the host name otherwise, and use it rather than the domain
name of the list server.
Tests have been adjusted to match the new behavior.
|
|
For Subject/To/Cc/From headers, we squeeze them to a space (' ').
For Message-IDs (including References/In-Reply-To), '\t', '\n', '\r'
are deleted since some MUAs might screw them up:
https://public-inbox.org/git/656C30A1EFC89F6B2082D9B6@localhost/raw
|
|
* origin/master:
nntp: allow and ignore empty commands
mbox: do not barf on queries which return no results
nntp: fix NEWNEWS command
searchview: fix non-numeric comparison
Allow specification of the number of search results to return
githttpbackend: avoid infinite loop on generic PSGI servers
http: fix modification of read-only value
extmsg: use news.gmane.org for Message-ID lookups
extmsg: rework partial MID matching to favor current inbox
Update the installation instructions with Fedora package names
nntp: do not drain rbuf if there is a command pending
nntp: improve fairness during XOVER and similar commands
searchidx: do not modify Xapian DB while iterating
Don't use LIMIT in UPDATE statements
|
|
Somebody hitting "\n" into telnet shouldn't hold a client up
indefinitely and prevent shutdown.
|
|
id_batch had a an overly complicated interface, replace it
with id_batch which is simpler and takes advantage of
selectcol_arrayref in DBI. This allows simplification of
callers and the diffstat agrees with me.
|
|
I guess nobody uses this command (slrnpull does not), and
the breakage was not noticed until I started writing new
tests for multi-MID handling.
Fixes: 3fc411c772a21d8f ("search: drop pointless range processors for Unix timestamp")
|
|
I guess nobody uses this command (slrnpull does not), and
the breakage was not noticed until I started writing new
tests for multi-MID handling.
Fixes: 3fc411c772a21d8f ("search: drop pointless range processors for Unix timestamp")
|
|
A work-in-progress, but it appears the v2 UI pieces do
will not require a lot of work to do.
|
|
Using update-copyrights from gnulib
While we're at it, use the SPDX identifier for AGPL-3.0+ to
ease mechanical processing.
|
|
Oops, due to an old mistake , List-ID was set incorrectly
in the MDA. This could cause some breakage w.r.t. mail filters.
|
|
We may not always use strftime and may implement caching.
But for now, just add a test.
|
|
This avoids breaking clients on graceful shutdown since
NNTP responses should usually be quick.
|
|
Or whatever the appropriate Perl terminology, is...
And we will need to do something appropriate for other
encodings, too. I still barely understand Perl Unicode
despite attempting to understand the docs over the years..
|
|
Oops, we totally forgot to automate testing for this :x
|
|
Most of its functionality is in the PublicInbox::Inbox class.
While we're at it, we no longer auto-create newsgroup names
based on the inbox name, since newsgroup names probably deserve
some thought when it comes to hierarchy.
|
|
There's no place for them in the commands and we don't take
messages; potentially printing them into a log opened in a
terminal is too dangerous.
Hoist out read_til_dot in the test while we're at it.
|