| Date | Commit message (Collapse) |
|---|
|
Add a few more details and (especially) the Fedora package names. I also
took the liberty of reorganizing things slightly while I was there.
|
|
Some clients pipeline requests aggressively (enough to match
LINE_MAX) and we should not read from the client socket until we
know there's no pending command in our read buffer.
Reported-and-tested-by: Sergey Organov <sorganov@gmail.com>
|
|
For other commands generating long responses, we generally want
to yield to another client after emitting 100 . However,
XOVER-based responses already query 200 lines worth of responses
at a time, so we were sending 20000 lines before yielding to
other clients. This may help avoid timeouts for some clients.
|
|
Iterating through a list of documents while modifying them does
not seem to be supported in Xapian and it can trigger
DatabaseCorruptError exceptions. This only worked with past
datasets out of dumb luck. With the work-in-progress "v2"
public-inbox layout, this problem might become more visible
as the "thread skeleton" is partitioned out to a separate,
smaller Xapian database.
I've reproduced the problem on both Debian 8.x and 9.x with
Xapian 1.2.19 (chert backend) and 1.4.3 (glass backend)
respectively.
|
|
...not all distributions build SQLite with that enabled.
[ew: LIMIT shouldn't be necessary because `key' is primary]
|
|
Might as well, this release is mostly to serve as a checkpoint
for the start of new development on v2 stuff mentioned in the
TODO.
|
|
|
|
This can be useful for tarball distributions which lack full git
history.
|
|
Using update-copyrights from gnulib
While we're at it, use the SPDX identifier for AGPL-3.0+ to
ease mechanical processing.
|
|
Sometimes, it can be desirable to jump directly to the "nested"
view when viewing a thread skeleton. This makes it possible.
While we're at it, shorten some of the text to ensure it still
fits in 80 columns.
|
|
We leave the mailto: link out when obfuscating address, so
do not stuff the "</pre>" closing tag into it. Instead,
keep the closing tag in the same context as the opening one,
making it easier to keep track of.
|
|
Namely, we do not want to obfuscate the mail address of the
site itself.
|
|
This makes the wording less confusing when showing archives
for lists where the convention is reply-to-list.
I still hate reply-to-list, but it's still better than no
archives or list at all.
|
|
This can allow streaming parsers (SAX) to work a little more
efficiently as they can handle/discard all the metadata before
the big content.
|
|
I still hate that CSS is over-used, but colors are useful
and perhaps using them for highlighting won't be too bad;
but user-supplied colors will ALWAYS be supported.
|
|
Inspired by interest in LKML archival:
https://public-inbox.org/meta/d5546b24-5840-4ae9-d25b-5e3e737ed73b@linuxfoundation.org
|
|
Obfuscating username portions of the email address leads
to having subsequent parts of the address not being obfuscated;
which could mean we show someone else's email entirely.
In other words, obfuscating "john.doe@example.com" becomes
might mean "doe@example.com" is picked up by scanners.
In other news, email address obfuscation is still a horrible
usability issue and only exists to appease misguided people.
|
|
It is usually pointless to replace a single word with a '"' character.
|
|
This should prevent crawlers (including most robots.txt ignoring
ones) from burning our CPU time without severely compromising
usability for humans.
|
|
Some search results are gigantic, and search engines are
unlikely to be able to handle gzipped mboxes anyways.
|
|
Allowing downloading of all search results as an gzipped mboxrd
file can be convenient for some users.
|
|
Perl 5.22 started warning about this.
|
|
We want to be consistent with the view change in
commit b223e6f49debb99b9132bc85d97a065ebcee00b9
|
|
This makes it easy to identify the reason for message removals.
|
|
We need to use the correct subject when doing global scanning,
too. In fact, the per-recipient spam training path is entirely
redundant at this point.
|
|
Sometimes an email is an innocent removal "rm" for a
misdirected, off-topic post, while most removed messages are
"spam". Allow anybody to look at history and easily distinguish
the reason for removing the message.
|
|
We always do threading, so perhaps it's not a good name.
"Nested" is probably more appropriate and closer to what
people are used to seeing.
|
|
This is hopefully more sensical than "raw" files from
resulting downloads.
|
|
Since we attempt to fill in threads by Subject, our thread
skeletons can cross actual thread IDs, leading to the
possibility of false ghosts showing up in the skeleton.
Try to fill in the ghosts as well as possible by performing
a message lookup.
|
|
We should not blindly join References and In-Reply-To headers
as a single string, because some messages can have an open
angle brace '<' in References: without a corresponding '>'.
|
|
Oops, we must not discard the timezone when parsing dates
for the Atom stream.
|
|
|
|
It seems Xapian is not prepared for Japanese, unfortunately.
https://public-inbox.org/meta/20170702213657.GA5312@dcvr/
|
|
This will hopefully increase visibility of some archives.
|
|
There is no need to show the same phrases over and over again
in thread skeletons, it adds to visual noise and makes things
more difficult to read.
|
|
This will be much faster and invoking -mda for every message.
|
|
Only one substitution character is necessary when obfuscating
email addresses.
|
|
It is needless bloat and doesn't seem to help with readability,
in retrospect, either.
|
|
We must not trigger future activity when initializing
a -watch shutdown.
|
|
Oops, this is needed for Perl 5.22 (tested 5.24.1) since '.'
was removed due to security problems. Fwiw, I consider this
change to Perl an overreaction and do not agree with it.
|
|
We should make changes visible sooner, even during
lengthy scans.
|
|
This should be more reliable and safer as it'll ensure
existing fast-import instances are shut down properly.
|
|
We need to ensure new messages are being processed
fairly during full rescans, so have the ->scan subroutine
yield and reschedule itself. Additionally, having a
long-running task inside the signal handler is dangerous
and subject to reentrancy bugs.
Due to the limitations of the Filesys::Notify::Simple interface,
we cannot rely on multiplexing I/O interfaces (select, IO::Poll,
Danga::Socket, etc...) for this. Forking a separate process
was considered, but it is more expensive for a mostly-idle
process.
So, we use a variant of the "self-pipe trick" via inotify (or
whatever Filesys::Notify::Simple gives us). Instead of writing
to our own pipe, we write to a file in our own temporary
directory watched by Filesys::Notify::Simple to trigger events
in signal handlers.
|
|
Signals can fire on us at any time if we're using blocking sysread.
|
|
Otherwise the old watcher may run indefinitely
|
|
Oops, due to an old mistake , List-ID was set incorrectly
in the MDA. This could cause some breakage w.r.t. mail filters.
|
|
Sometimes, URLs exist at the end of parethesized statements,
and we shouldn't unnecessarily capture that.
(example: https://public-inbox.org/ruby-core/20170623032722.GA8124@dcvr/)
|
|
We will also treat all known list addresses as non-obfuscated.
By setting publicinbox.noObfuscate in ~/.public-inbox/config,
this will allow users to disable address obfuscation on a
per-domain or per-address basis.
|
|
This should simplify the rest of our code for handling
the do-not-obfuscate list.
|
|
This is necessary to retain consistent spacing around bullet
points.
Fixes: 666844ae42b5b17f ("reply: handle address obfuscation :<")
|