Date | Commit message (Collapse) |
|
Oops, due to an old mistake , List-ID was set incorrectly
in the MDA. This could cause some breakage w.r.t. mail filters.
|
|
Sometimes, URLs exist at the end of parethesized statements,
and we shouldn't unnecessarily capture that.
(example: https://public-inbox.org/ruby-core/20170623032722.GA8124@dcvr/)
|
|
We will also treat all known list addresses as non-obfuscated.
By setting publicinbox.noObfuscate in ~/.public-inbox/config,
this will allow users to disable address obfuscation on a
per-domain or per-address basis.
|
|
This should simplify the rest of our code for handling
the do-not-obfuscate list.
|
|
This is necessary to retain consistent spacing around bullet
points.
Fixes: 666844ae42b5b17f ("reply: handle address obfuscation :<")
|
|
Sigh, yet another place to handle obfuscation for misguided
people who expect it. Maybe this will do something to prevent
spammers from getting addresses, while still allowing the
"curl $URL | git am" use case to work.
|
|
We can show users a lightly-obfuscated Bourne shell command
for invoking "git send-email" for address obfuscation. However,
I'm not sure if the mailto: arg will work effectively since
URL encoding is probably too well-known to be effective.
|
|
Yet another hiccup from reusing pre-set article numbers on
various ruby-lang.org mailing lists. This was causing messages
to not appear to NNTP readers which use XOVER.
|
|
This prevents public-inbox-watch from dying when reloading
(and thus rescanning) already-imported directories.
|
|
The RubyLang filter is strict about what messages it rejects, so
the spam learning path will not auto-train or remove messages
missing X-Mail-Count headers.
|
|
This allows users to DRY up their config a bit and avoid
specifying altid twice when reusing the NNTP-centric msgmap
for [ruby-*:\d+] serial numbers.
My current work-in-progress ~/.public-inbox/config entry
for the ruby-core list is:
------8<-------
[publicinbox "ruby-core"]
address = ruby-core@ruby-lang.org
url = //public-inbox.org/ruby-core
mainrepo = /path/to/ruby-core.git
newsgroup = inbox.comp.lang.ruby.core
watchheader = List-Id:<ruby-core.ruby-lang.org>
altid = serial:ruby-core:file=msgmap.sqlite3
watch = maildir:/path/to/Maildir/.INBOX.ruby
filter = PublicInbox::Filter::RubyLang
|
|
This will allow smoother imports as occasional Message-ID
duplicates happen and the best we can do is ignore the
second one.
|
|
Unfortunately, it appears we have to reject this and instead add
support filtering at View time(*), due to DKIM signatures in
messages from ruby-lang.org.
(*) which may not be worth it
|
|
This seems to allow weirdly-encoded "raw" emails in
blade.nagaokaut.ac.jp/ruby/ruby-core/*
to be handled without difficulties.
|
|
This is lightly-tested and seems to work. I'm still
hesitant to support this, but the alternative of receiving death
threats for displaying unobfuscated addresses seems to
be not worth it.
|
|
Reply-To is common and probably should've been supported,
since day one, but we won't omit other addresses, either.
|
|
This allows us to support centralized mailing lists (which suck,
but better than no mailing list at all).
|
|
We'll be adding more reply options for centralized mailing
lists. So split out the logic so it's easy-to-find.
Organizing code is hard :<
|
|
This fixes a bug introduced in
commit 7eeadcb62729b0efbcb53cd9b7b181897c92cf9a
("search: remove unnecessary abstractions and functionality")
|
|
This can be tied into a repository browser to browse
in-flight topics on a mailing list.
|
|
Xapian memory usage is tied to the size of the indexed
text, so take the raw message size into account when
deciding when to flush Xapian data.
More importantly, we now flush Xapian before we have it
buffer beyond our maximum; and we do it unconditionally
to prevent even high priority processes from OOM-ing.
|
|
This simplifies the code a bit and reduces the translation
overhead for looking directly at data from tools shipped
with Xapian.
While we're at it, fix thread-all.t :)
|
|
This is a high indicator of spam (but out-of-scope for this
particular module) but sometimes it is not, and people
legitimately forget to set a Subject: header at all.
|
|
This was necessary for the presence of the 0xa0 byte(*)
in the Subject: of the message at:
http://blade.nagaokaut.ac.jp/ruby/ruby-core/3220
(*) That is 0xa0, not 0x0a ("\n"), so I wonder if the
nibbles got swapped somehow.
|
|
It is possible to have double-escaped queries when copy and
pasting into browsers, so try to help users work around this
common error by automatically retrying after unescaping once.
Of course, we must inform the user when doing this results in
success, in case they really meant to search for a
double-escaped term which resulted in nothing.
Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
https://public-inbox.org/meta/CACBZZX5Gnow08r=0A1J_kt3a=zpGyMfvsqu8nAN7kacNnDm+dg@mail.gmail.com/
|
|
Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
https://public-inbox.org/meta/CACBZZX5Gnow08r=0A1J_kt3a=zpGyMfvsqu8nAN7kacNnDm+dg@mail.gmail.com/
|
|
Sometimes bots generate malformed queries with sequential
"&" and ";" characters.
|
|
It should be helpful to know what error happened.
|
|
umask should never fail and set $@, but use the cached local
to be more explicit just in case.
|
|
Due to the asynchronous nature of SMTP, it is possible for the
root message of a thread (with no References/In-Reply-To)
to arrive last in a series. We must preserve the thread_id
of the ghost message in this case, as we do when vivifiying
non-root ghosts.
Otherwise, this causes threads to be broken when the root
arrives last.
|
|
I'm not sure if people use either and it's not in mairix
(where we base our abbreviations off of). Lets go
with the shorter prefix since it's easier-to-type.
|
|
Dovecot uses 'a'..'z' (lowercase) to designate keywords
in Maildir flags. This was preventing certain messages
from being marked as spam.
https://wiki2.dovecot.org/MailboxFormat/Maildir
|
|
When displaying search results with full messages, it makes
more sense to show them in ascending chronological order when
going by date. Reverse chronological order makes more sense
for search results which only show the subject.
|
|
At least for the thread view (&x=t); this will make it
easy to link to the overview.
|
|
Apparently mid.mail-archive.com does not support HTTPS,
and the HTTP version redirects to the search query, anyways.
|
|
Otherwise funky filenames can cause HTML injection
vulnerabilities (hope you have JavaScript disabled!)
|
|
PSGI specs already require PATH_INFO to be unescaped;
so our tests were wrong, too.
|
|
It seems possible for git-send-email(1) to generate repeated
repeated instances of References and In-Reply-To headers,
as evidenced in:
https://public-inbox.org/git/20161111124541.8216-17-vascomalmeida@sapo.pt/raw
This causes a mismatch between how our search indexer threads
and how our HTML view handles threading. In the future, View.pm
will use the smsg-parsed {references} field and avoid redoing
Email::MIME header parsing.
We will still need to figure out a way to deal with messages
with repeated Message-IDs, at some point, too.
|
|
There's no need to hold everything in memory, here,
since apparently "foreach" will read everything at
once in array context
(for some reason, I thought Perl5 was smart enough
to avoid creating a temporary array, here...)
|
|
We cannot distinguish between legitimate ghosts and mis-threaded
messages before commit 83425ef12e4b65cdcecd11ddcb38175d4a91d5a0
("searchidx: deal with empty In-Reply-To and References headers")
so we must rebuild the index in parallel to fix it.
|
|
Oops, that's broken, too. I guess the only way to reindex
after fixing the thread detection is to start from scratch.
This reverts commit 5d91adedf5f33ef1cb87df2a86306ddf370b4f8d.
|
|
We cannot always reuse thread IDs since our threading
logic may change as bugs are fixed.
|
|
In some messages, these headers exist, but have empty values.
Do not let empty values throw off our search indexer to tie
threads together, as it can make non-sensical threads grouped
to a Message-Id of "" (empty string).
See
<https://public-inbox.org/git/11340844841342-git-send-email-mailing-lists.git@rawuncut.elitemail.org/raw>
for an example of such a message.
Thanks-to: Johannes Schindelin <Johannes.Schindelin@gmx.de>
<https://public-inbox.org/git/alpine.DEB.2.20.1702041206130.3496@virtualbox/>
|
|
We are in no danger of excessive buffering or OOM-ing,
the main page for every inbox already loads 200 results;
and thread page views even load 1000! Increase this to
200 for now.
|
|
Xapian can only give estimated results when a result limit is
given to it, so make clear it is an estimate to avoid showing
non-sensical ranges when no results are returned.
|
|
Some mailing lists add annoying tags into the Subject line which
discourages readers from doing proper mail organization on the
client side. They also waste precious screen space and
attention span.
Remove them from our archives to reduce clutter.
|
|
We'll want to allow some degree of configuration for
various mailing lists.
|
|
We don't want to be triggering OOM or swapping on weaker
systems when we have dozens of inboxes as potential targets.
|
|
We must call Email::Simple methods directly in our monkey patch
for Email::MIME to call the intended method. Using SUPER in our
subclass would instead hit a different, unintended method in
Email::MIME.
Reported-by: Junio C Hamano <gitster@pobox.com>
<xmqq4m0wb43w.fsf@gitster.mtv.corp.google.com>
|
|
We may need to do this even more aggressively, since the
Xapian database does not always give the latest results.
This time, we'll do it without relying on weak references,
and instead check refcounts.
|