Date | Commit message (Collapse) |
|
Bigger changes coming :>
|
|
The x32 ABI allows users to take advantage of the extra
registers on x86-64 without the bloat of 64-bit pointers and
longs.
This ought to be significant since Perl was designed when 32-bit
was prevalent; and the common structs for ops, hashes, scalars,
and arrays use longs (SSize_t/Size_t) for things which should
never need 64-bits when processing emails.
Debian's x32 port seems to work quite nicely under a chroot
on an amd64 Linux system. All tests pass under x32, now.
|
|
Oops :x
|
|
This is derived from a real-world test case where I encounterd
multiple Message-IDs in a v1 inbox causing regen problems.
Fixes: eea47b676127bcdb ("convert: preserve highwater mark from v1 msgmap")
|
|
Danga::Socket 1.62 was released a few months back and
the maintainer indicated it would be the last release.
We've diverged significantly in incompatible ways...
While most of this should've already been documented in
commit messages, putting it all into one document could
make it easier-to-digest.
It's also a strange design for anybody used to conventional
event loops. Maybe this is an unconventional project :P
|
|
In rare cases where Message-IDs get reused, we do not want to
hold onto the large Email::MIME objects in memory after showing
the first message. So discard each message as soon as we're
done using it so we can save memory for the next message.
The new and expensive xt/mem-msgview.t test shows a nearly 14MB
reduction for two ~7MB messages. run_script() also gets
upgraded to make it easier to pass large inputs via IO GLOBs.
|
|
There's a lot of test cases which we should probably
make self-contained at some point, but right now it's
easier to just mark them off in a maintainer test.
|
|
This is necessary for Filesys::Notify::Simple 0.13 using
Linux::Inotify2, since 0.13 started croaking on
inotify_add_watch failures.
|
|
Seems like a lot's happened since 1.2, but it's mostly
internal stuff...
|
|
It's now possible to use WwwStatic as a standalone PSGI
app to serve static files and recreate the award-winning
web design of https://public-inbox.org/ :>
|
|
Make it easier to share code between our GitHTTPBackend and Cgit
packages, for now, and possibly other packages in the future.
We can avoid inline_object and anonymous subs at the same
time, reducing per-request memory overhead.
|
|
Spawning a new Perl interpreter for every test case
means Perl has to reparse and recompile every single file
it needs, costing us performance and development time.
Now that we've modified our code to avoid global state,
we can preload everything we need.
The new "check-run" test target is now 20-30% faster
than the original "check" target.
|
|
We want to be able to use run_script with *.t files, so
t/common.perl putting subs into the top-level "main" namespace
won't work. Instead, make it a module which uses Exporter
like other libraries.
|
|
Email::Address::XS is a dependency of modern versions of Email::MIME,
so it's likely loaded and installed on newer systems, already;
and capable of handling more corner-cases than our pure-Perl
fallback.
We still fallback to the imperfect-but-good-enough-in-practice
pure-Perl code while avoiding the non-XS Email::Address (which
was susceptible to DoS attacks (CVE-2015-7686)). We just need
to keep "git fast-import" happy.
|
|
EvCleanup only existed since Danga::Socket was a separate
component, and cleanup code belongs with the event loop.
|
|
Changes will be coming for MsgTime to stop depending on
Date::Parse due to lack of package availability on OpenBSD
and suboptimal performance on RFC822 dates.
|
|
This is a transitionary interface which does NOT require an
event loop. It can be plugged into in current synchronous code
without major surgery.
It allows HTTP/1.1 pipelining-like functionality by taking
advantage of predictable and well-specified POSIX pipe semantics
by stuffing multiple git cat-file requests into the --batch pipe
With xt/git_async_cmp.t and GIANT_GIT_DIR=git.git, the async
interface is 10-25% faster than the synchronous interface since
it can keep the "git cat-file" process busier.
This is expected to improve performance on systems with slower
storage (but multiple cores).
|
|
Our attempt at using a self-pipe in signal handlers was
ineffective, since pure Perl code execution is deferred
and Perl doesn't use an internal self-pipe/eventfd. In
retrospect, I actually prefer the simplicity of Perl in
this regard...
We can use sigprocmask() from Perl, so we can introduce
signalfd(2) and EVFILT_SIGNAL support on Linux and *BSD-based
systems, respectively. These OS primitives allow us to avoid a
race where Perl checks for signals right before epoll_wait() or
kevent() puts the process to sleep.
The (few) systems nowadays without signalfd(2) or IO::KQueue
will now see wakeups every second to avoid missed signals.
|
|
Oops, IO::KQueue support was broken due to this missing
constant. Add a new ds-kqxs.t test case to ensure we
test the IO::KQueue path if IO::KQueue is available.
|
|
xt/ is typically reserved for "eXtended tests" intended for
the maintainers and not ordinary users. Since these require
special configuration and do nothing by waste cycles
during startup, they qualify.
|
|
Newer versions of git enable the commit graph by default.
Since we blow away our temporary directories every test,
generating graphis is a waste and clutters stderr with
"Computing commit graph generation numbers" messages.
|
|
It'll make using Compress::Raw::Zlib easier, since we
can use that and import constants more easily.
|
|
|
|
Yet another case of documenting things which should NOT be used :>
|
|
Tools intended for end users need manpages, and doubly so
to convince potential users NOT to use them :)
|
|
ISO-2202-JP and other non-UTF-8 messages need to be displayed
correctly.
Fixes: 7d82a8bc04ce ('handle "multipart/mixed" messages which are not multipart')
|
|
Tools intended for end users need manpages.
|
|
This is a plugin for SpamAssassin that happens to be quite
useful in keeping spam off lists I mirror. Hopefully more
people can find it useful now that it has a manpage.
|
|
This requires the latest (to be in 1.2) -init changes for
synchronization and has no dependencies on GNU or bash-isms
so it should run on *BSD systems without GNU tools.
It does attempt to use curl on <$INBOX_URL/_/text/config/raw>,
but curl is fairly standard nowadays, and falls back to using
an invalid address to initialize.
|
|
We'll use our Documentation/RelNotes directory and internal APIs
to generate these files for website use (the website should be
completely reproducible).
|
|
This old command was lacking a manpage, so (finally) create one.
|
|
The v1.2.0 is a work-in-progress, while the others are copied
out of our mail archives.
Eventually, a NEWS file will be generated from these emails and
distributed in the release tarball. There'll also be an Atom
feed for the website reusing our feed generation code.
|
|
NNTPS and STARTTLS seems to be working for several months
without incident on news.public-inbox.org, so consider it a
success and maybe others can try using it.
HTTPS technically works, too, but isn't documented at
the moment since I can't recommend production deployments
without varnish protecting it.
|
|
Although we always unlink temporary files, give them a
meaningful name so that we can we can still make sense
of the pre-unlink name when using lsof(8) or similar
tools on Linux.
|
|
This is only tested so far with my patches to Net::NNTP at:
https://rt.cpan.org/Ticket/Display.html?id=129967
Memory use in C10K situations is disappointing, but that's
the nature of compression.
gzip compression over HTTPS does have the advantage of not
keeping zlib streams open when clients are idle, at the
cost of worse compression.
|
|
It's barely any effort at all to support HTTPS now that we have
NNTPS support and can share all the code for writing daemons.
However, we still depend on Varnish to avoid hug-of-death
situations, so supporting reverse-proxying will be required.
|
|
At least the subset of epoll we use. EPOLLET might be
difficult to emulate if we end up using it.
|
|
We don't need to code multiple event loops or have branches in
watch() if we can easily make the IO::KQueue-based interface
look like our lower-level epoll_* API.
|
|
It kinda, barely works, and I'm most happy I got it working
without any modifications to the main NNTP::event_step callback
thanks to the DS->write(CODE) support we inherited from
Danga::Socket.
|
|
Another step towards keeping our file and package names
consistent with Xapian terminology.
|
|
* origin/reshard:
xcpdb: support resharding v2 repos
xcpdb: use destination shard as progress prefix
xapcmd: preserve indexlevel based on the destination
v2writable: use a smaller default for Xapian partitions
|
|
* origin/manifest:
git: ensure ->modified returns an integer
www: support $INBOX/git/$EPOCH.git for v2 cloning
www: wire up /$INBOX/manifest.js.gz, too
wwwlisting: generate grokmirror-compatible manifest.js.gz
wwwlisting: allow hiding entries from manifest
|
|
v2 repos are sometimes created on machines where CPU
parallelization exceeds the capability of the storage devices.
In that case, users may reshard the Xapian DB to any smaller,
positive integer to avoid excessive overhead and contention when
bottlenecked by slow storage.
Resharding can also be used to increase shard count after
hardware upgrades.
|
|
This wrapper around V2Writable->replace provides a user-interface
for editing messages as single-message mboxes (or the raw text
via $EDITOR).
|
|
Editing and purging are similar operations involving history
rewrites, so there'll be common options and code between them.
|
|
Much of the existing purge code is repurposed to a general
"replace" functionality.
->purge is simpler because it can just drop the information.
Unlike ->purge, ->replace needs to edit existing git commits (in
case of From: and Subject: headers) and reindex the modified
message.
We currently disallow editing of References:, In-Reply-To: and
Message-ID headers because it can cause bad side effects with
our threading (and our lack of rethreading support to deal with
excessive matching from incorrect/invalid References).
|
|
Support on-demand generation of "/manifest.js.gz" for inboxes.
By default, this matches inboxes with URLs matching the given
request hostname by default.
This makes it easier to create full mirrors of several inboxes
without needing to configure static file serving.
cf. https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git
|
|
Well, it could probably be moved to contrib...
|
|
The version of Test::More from Perl 5.10.1 did not support
"subtest", and the earliest version which did is Perl 5.12.0
The good news is this gives me an excuse to parallelize
the indexlevels-mirror test by splitting it into two.
(it could be further split, even).
Update t/nntpd. to use PI_TEST_VERSION consistently while
we're at it.
|
|
|