Date | Commit message (Collapse) |
|
gmane still has a NNTP server, so update links to point to it.
cf. https://lars.ingebrigtsen.no/2020/01/06/whatever-happened-to-news-gmane-org/
|
|
While both can be correct, the former seems more common,
is shorter, and is also consistent with the spelling found
in the AGPL-3.0 text.
|
|
We should support historical archives from the old days,
but I'm not sure how to best go about it, for now, given
how tricky correct handling of modern email addresses is.
We can deal with it if/when somebody decides to import some
ancient archives...
|
|
-mda should not be dealing with broken Date: headers
nowadays, and deprioritize it in our documentation and
internal checks.
|
|
SpamAssassin has used re2c (via sa-compile) for many years, now,
and it seems to work fine, there. GMime also looks promising
when combined with Inline::C since GMime can operate on mmap-ed
regions.
Given the inevitable demise of many .orgs when price rise;
supporting a URL rewriter similar to .mailmap makes sense.
And HTTP CONNECT seems like something our -httpd can support
to let firewalled users read over NNTP.
|
|
That's the only head-scratcher of the bunch remaining, since
that relies on ranges.
|
|
* regen:
v2writable: use msgmap as multi_mid queue
v2writable: move git->cleanup to the correct place
v2writable: reindex handles 3-headered monsters
v2writable: improve "num_for" API and disambiguate
v2writable: set unindexed article number
|
|
And maybe 8-headered ones, too...
I noticed --reindex failing on the linux-renesas-soc mirror due
one 3-headed monster of a message having 3 sets of headers;
while another normal message had a Message-ID that matched one
of the 3 IDs of the 3-headed monster.
We still try to do the majority of indexing backwards, but we
defer indexing multi-Message-ID'd messages until the end to
ensure we get all the "good" messages in before we process the
multi-headered ones.
Link: https://public-inbox.org/meta/20191016211415.GA6084@dcvr/
|
|
This would encourage SPOF avoidance. It would also make it
easier to do "Certificate Transparency"-style attestation of
messages.
|
|
Going where no email client/tool has gone before...
|
|
Pygments seems to be a popular highlighter and widely available,
so we'll be providing support for that at some point...
Link: https://public-inbox.org/meta/20190926131836.GB10467@chatter.i7.local/
Link: https://public-inbox.org/meta/874l0zt7sd.fsf@alyssa.is/
|
|
It'll be useful for tools and users to test and perhaps
visualize configs before reloading -httpd/-nntpd/-watch.
|
|
I forgot about this feature when I was implementing
blob-ID-based searches :x
|
|
Millions of inboxes in an instance is probably not feasible, but
dozens or even hundreds could happen and
/proc/sys/fs/pipe-user-pages-soft is only 16384 on my system,
with each "cat-file --batch" process using 16+1 pages worth
of pipes.
|
|
Inline::C seems alright, so we might use it more since it still
allows end users to quickly make changes. Our performance on
rotational disks is also terrible, and could be improved...
|
|
It never ends...
|
|
This is only tested so far with my patches to Net::NNTP at:
https://rt.cpan.org/Ticket/Display.html?id=129967
Memory use in C10K situations is disappointing, but that's
the nature of compression.
gzip compression over HTTPS does have the advantage of not
keeping zlib streams open when clients are idle, at the
cost of worse compression.
|
|
Taking one step out of setting up a performant deployment could
make setup and administration easier (at the cost of installing
an extra-but-common XS module). This can also be useful for
the day NNTP servers see hug-of-death events.
|
|
git bundles could/should make self-hosting easier.
Being able to configure synonym (and spelling) lists would make
some searches more useful.
Might as well dogfood kernel stuff, too, given the overlap and
history between this project, git and the Linux kernel. Would
be interesting to have *BSD folks throw their hat in the ring,
too.
Building/testing userspace stuff is often the most
time-consuming, but necessary to ensure future compatibility.
|
|
More tests work without Search::Xapian, now.
Usability issues still need to be fixed
|
|
IO::Kqueue seems unmaintained, so workaround a long-standing
bug where it falls over on signals:
https://rt.cpan.org/Ticket/Display.html?id=116615
|
|
Since our listen sockets are non-blocking and we may run
multiple httpd|nntpd processes; we need a way to avoid
thundering herds when there are multiple httpd|nntpd worker
processes.
EPOLLEXCLUSIVE was added just for that in Linux 4.5
|
|
These modules are unmaintained upstream at the moment, but I'll
be able to help with the intended maintainer once/if CPAN
ownership is transferred. OTOH, we've been waiting for that
transfer for several years, now...
Changes I intend to make:
* EPOLLEXCLUSIVE for Linux
* remove unused fields wasting memory
* kqueue bugfixes e.g. https://rt.cpan.org/Ticket/Display.html?id=116615
* accept4 support
And some lower priority experiments:
* switch to EV_ONESHOT / EPOLLONESHOT (incompatible changes)
* nginx-style buffering to tmpfile instead of string array
* sendfile off tmpfile buffers
* io_uring maybe?
|
|
and add a note for grokmirror support/integration, too
|
|
The git-users mailing list is on Google Groups with obfuscated
addresses and censored archives. We should allow users to
import them soon, as obfuscated/censored archives are better
than not having archives at all when Google decides to shut down
yet-another-service.
There's also some mangling that (most) instances of Mailman do
(e.g. cgit), but being able to follow such groups over NNTP
and use our search functionality is still useful and better
than what typical Mailman installations provide.
|
|
I'd like to move https://public-inbox.org/git/ to v2; but
cronjobs using "git fetch" to following should not break.
|
|
|
|
Since we now support more CSS classes for coloring,
give this feature more visibility.
|
|
Old and new versions of Mozilla-based browsers seem to support
userContent.css just fine.
cf. https://www-archive.mozilla.org/unix/customizing.html#usercss
http://kb.mozillazine.org/index.php?title=UserContent.css
|
|
We support searching on blob identifiers for a reason :>
|
|
|
|
|
|
And maybe I or somebody else interested will implement it, since
fusedav is abandoned upstream and removed from Debian testing:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840388
Yes, I have fusedav patches at https://bogomips.org/fusedav.git
as noted in the above bug report, but I think davfs2 has more
momentum at the moment.
|
|
Threads are generally discouraged in Perl5, so I won't be using
a dedicated blocking accept4() thread like I would in other
languages.
|
|
Mainly, v2 stuff is done
|
|
Inspired by interest in LKML archival:
https://public-inbox.org/meta/d5546b24-5840-4ae9-d25b-5e3e737ed73b@linuxfoundation.org
|
|
Always plenty to do while working on this...
|
|
This will allows certain feed readers to render a message thread
as described in <https://www.jwz.org/doc/threading.html>.
Feed readers with knowledge of of RFC 4685 are unknown to us at
this time, but perhaps this will encourage future implementations.
Existing feed readers I've tested (newsbeuter, feed2imap) seem
to ignore these tags gracefully without degradation.
|
|
Do not require users to have network access to know what
the link refers to.
|
|
The existing string -> number date range Xapian query is good
enough, and having too much flexibility is probably bad for
caching (as well as increasing our attack surface, because
parsing queries is tricky).
Tags-as-skiplists are probably not worth the effort given
Xapian, and we may have to import old messages after-the-fact,
anyways, and message delivery for mirrors is never orderly.
Other items are all done and need to be maintained (like the
search engine docs for the mairix-compatibility features that
just got pushed out)
|
|
Plenty more to do!
|
|
I bet there's a billion other improvements to be made elsewhere.
|
|
|
|
This should be more accessible to readers on narrow terminals
(or giant fonts) while providing a chronological view which
is also aware of message threading relationships.
|
|
"git cat-file --batch" seems expensive for big repos and
loading 70K+ tree objects in git isn't all that fast.
Ideas are cheap, time, code, and testing are not :P
|
|
It would be too much of a burden for caching system when
user-supplied CSS is more powerful.
|
|
Some readers will want to use "HTTPS Everywhere" conveniently;
and I will support it.
|
|
Unfortunately, most users still prefer their mail delivered
over SMTP; so we'll at least document mlmmj integration for now
until we can popularize pull-based reading over POP3/NNTP/ssoma.
|
|
Email addresses get out-of-date, so make sure they're mapped
properly for future readers. git and linux-kernel already have
an established convention for this, so we will follow it.
|
|
|