Date | Commit message (Collapse) |
|
I didn't wait until September to do it, this year!
|
|
Avoid 'Variable "%s" will not stay shared' warnings
when the contents of this script eval'ed into a sub.
|
|
While it's not RFC2919-conformant, mail software can
theoretically set multiple List-ID headers. Deliver to all
inboxes which match a given List-ID since that's likely the
intended.
Cc: Eric W. Biederman <ebiederm@xmission.com>
Link: https://public-inbox.org/meta/87pniltscf.fsf@x220.int.ebiederm.org/
|
|
Multiple List-ID headers will be supported in the next commit
|
|
And use it for mda, since "0" could be a usable directory
if somebody insists on using relative paths...
|
|
We don't want to waste cycles parsing the message for MIME bits
if it's spam.
|
|
It makes it easier to document the default -mda behavior is
stricter than normal, including "public-inbox-learn ham"
|
|
It's now possible to inject false-positive ham into an inbox
the same way -mda does via List-ID.
|
|
* origin/inboxdir:
config: remove redundant inboxdir check
config: support "inboxdir" in addition to "mainrepo"
examples/grok-pull.post_update_hook: use "inbox_dir"
|
|
"mainrepo" ws a bad name and artifact from the early days when I
intended for there to be a "spamrepo" (now just the
ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be
especially confusing, since v2 needs at least two git
repositories (epoch + all.git) to function and we shouldn't
confuse users by having them point to a git repository for v2.
Much of our documentation already references "INBOX_DIR" for
command-line arguments, so use "inboxdir" as the
git-config(1)-friendly variant for that.
"mainrepo" remains supported indefinitely for compatibility.
Users may need to revert to old versions, or may be referring
to old documentation and must not be forced to change config
files to account for this change.
So if you're using "mainrepo" today, I do NOT recommend changing
it right away because other bugs can lurk.
Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
|
|
Since -mda now supports List-ID to better support mirroring of
existing mailing lists, it probably makes sense to support
disabling the precheck function to provide more accurate (though
potentially spammier) mirrors of lists
|
|
This also adds watchheader tests for -watch, which we never
had before :x
|
|
|
|
Clearly the AltId stuff was never tested for v2. Ensure
this tricky filter (which reuses Msgmap to avoid introducing
new serial numbers) doesn't trigger deadlocks SQLite due
to opening a DB for writing multiple times.
I went through several iterations of this change before
going with this one, which is the least intrusive I could
fine.
|
|
This reuses some of the configuration from -watch, but remains
independent since some configurations will use -watch for some
inboxes and -mda for others.
The default remains "spamc" for -mda users so nothing changes
without explicit configuration.
Per-inbox configurations may also be supported in the future.
|
|
We must not clobber the original message string, as Email::MIME(*)
still needs it for iterating through parts in SearchIdx (but not
when handing it as a raw string to git-fast-import).
I've noticed message bodies (especially dfpre/dpost) were not
getting indexed when going through -mda (no problems with
-watch). This also did not affect v1 repos, since indexing is a
separate process for v1 and requires re-reading the data from
git.
(*) tested Email::MIME 1.937 on Debian stretch
|
|
It's a convenient wrapper nowadays, so get rid of some legacy
code and minimize differences from the -watch code.
|
|
Many MTA understand these and map them to sensible SMTP error messages.
Inability to find an inbox results in "5.1.1 user unknown".
Misformatted messages are rejected with "5.6.0 data format error".
Unsupported inbox versions are reported as "5.3.5 local configuration error".
All of these are interpreted as permanent failures.
|
|
I mainly focus on -watch for mirroring busy mailing lists, but
using -mda should remain an option.
|
|
It works around some bugs in older Email::MIME which we'll
find useful.
|
|
Using update-copyrights from gnulib
While we're at it, use the SPDX identifier for AGPL-3.0+ to
ease mechanical processing.
|
|
This should fix problems with multipart messages where
text/plain parts lack a header.
cf. git clone --mirror https://github.com/rjbs/Email-MIME.git
refs/pull/28/head
In the future, we may still introduce as streaming
interface to reduce memory usage on large emails.
|
|
We don't want to leave fast_import_crash_* dumps
around on duplicates.
|
|
Oops...
While we're at it, drop blank lines before the "From ", too,
since it could happen.
|
|
This should hopefully make it easier to try other anti-spam
systems (or none at all) in the future.
|
|
fork failures are unfortunately common when Xapian has
gigabytes and gigabytes mmapped.
|
|
This prevents multiple update processes from stepping over
each other while called under the lock, and also allows the
new -watch process to update the index iff indexing was
desired.
|
|
Give users some rope to do their own filtering.
|
|
This removes the Email::Filter dependency as well as the
signature-breaking scrubber code. We now prefer to
reject unacceptable messages and grudgingly (and blindly)
mirror messages we're not the primary endpoint for.
|
|
Email::Filter doesn't offer any functionality we need, here;
and our dependency on Email::Filter will gradually be removed
since it (and Email::LocalDelivery) seem abandoned and we
can have more-fine-grained control by rolling our own Maildir
delivery which can work transactionally.
|
|
We still pull it in via Email::LocalDelivery, but that
dependency will go away, soon.
|
|
User input is imperfect, do not pollute our mail logs with
warnings we cannot fix. This is documented in the
Email::MIME::ContentType manpage so it should remain supported.
|
|
git has stricter requirements for ident names (no '<>')
which Email::Address allows.
Even in 1.908, Email::Address also has an incomplete fix for
CVE-2015-7686 with a DoS-able regexp for comments. Since we
don't care for or need all the RFC compliance of Email::Address,
avoiding it entirely may be preferable.
Email::Address will still be installed as a requirement for
Email::MIME, but it is only used by the
Email::MIME::header_str_set which we do not use
|
|
From the beginning, we've avoided objects here in favor
of faster startup time; but it may not be worth it
since a persistent httpd/nntpd is faster and -mda
isn't hit as often.
|
|
A public-inbox is NOT necessarily a mailing list, but it
could serve as an input point for zero, one, or infinite
mailing lists :D
|
|
We should update $GIT_DIR/info/refs for dumb HTTP clients
whenever we make changes to the repository. The best place
to update is immediately after making commits.
This fixes a bug where public-inbox-learn did not properly
update $GIT_DIR/info/refs after inserting or removing
messages.
|
|
By converting to using ourt git-fast-import-based Import
module. This should allow us to be more easily installed.
|
|
This seems to match more closely with what is expected of Perl
packages based on how blib is used. Hopefully makes the top-level
source tree less cluttered and things easier-to-find.
|