* [PATCH 0/7] List-ID support in mda, watch, www
@ 2019-10-15 20:36 7% Eric Wong
2019-10-15 20:36 4% ` [PATCH 6/7] mda, watch: wire up List-ID header support Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2019-10-15 20:36 UTC (permalink / raw)
To: meta; +Cc: Eric W. Biederman
Starting with Eric Biederman's support for declaring "List-ID",
-watch and -mda are now able to deliver messages using List-ID.
Benefits for -mda users:
* Extra/private addresses no longer need to be configured
Benefits for -watch users:
* (slightly?) easier-to-use than "watchheader"
* allows combining multiple lists into one inbox
There's also several Perl 5.10+-inspired cleanups to the config
while I was in the area...
Eric W. Biederman (1):
Config.pm: Add support for mailing list information
Eric Wong (6):
config: we always have {-section_order}
config: simplify lookup* methods
config: avoid unnecessary '||' use
config: allow "0" as a valid mainrepo path
mda, watch: wire up List-ID header support
wwwtext: show listid config directive(s)
Documentation/public-inbox-config.pod | 18 ++++
Documentation/public-inbox-mda.pod | 3 +
lib/PublicInbox/Config.pm | 150 ++++++++++++--------------
lib/PublicInbox/WatchMaildir.pm | 31 ++++--
lib/PublicInbox/WwwText.pm | 2 +-
script/public-inbox-mda | 17 ++-
t/config.t | 73 ++++++-------
t/config_limiter.t | 24 ++---
t/mda.t | 28 +++++
t/psgi_attach.t | 8 +-
t/psgi_bad_mids.t | 10 +-
t/psgi_mount.t | 8 +-
t/psgi_multipart_not.t | 10 +-
t/psgi_scan_all.t | 12 ++-
t/psgi_search.t | 8 +-
t/psgi_text.t | 8 +-
t/psgi_v2.t | 10 +-
t/watch_filter_rubylang.t | 20 ++--
t/watch_maildir.t | 14 +--
t/watch_maildir_v2.t | 59 +++++++---
20 files changed, 305 insertions(+), 208 deletions(-)
^ permalink raw reply [relevance 7%]
* [PATCH 6/7] mda, watch: wire up List-ID header support
2019-10-15 20:36 7% [PATCH 0/7] List-ID support in mda, watch, www Eric Wong
@ 2019-10-15 20:36 4% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-10-15 20:36 UTC (permalink / raw)
To: meta; +Cc: Eric W. Biederman
This also adds watchheader tests for -watch, which we never
had before :x
---
Documentation/public-inbox-config.pod | 18 ++++++++++++++++
Documentation/public-inbox-mda.pod | 3 +++
lib/PublicInbox/WatchMaildir.pm | 31 +++++++++++++++++++++------
script/public-inbox-mda | 17 ++++++++++++---
t/mda.t | 28 ++++++++++++++++++++++++
t/watch_maildir_v2.t | 31 +++++++++++++++++++++++++++
6 files changed, 118 insertions(+), 10 deletions(-)
diff --git a/Documentation/public-inbox-config.pod b/Documentation/public-inbox-config.pod
index 8d545f7a..6a9739f7 100644
--- a/Documentation/public-inbox-config.pod
+++ b/Documentation/public-inbox-config.pod
@@ -85,6 +85,24 @@ the given header. Multiple values are not currently supported.
Default: none; only for L<public-inbox-watch(1)> users
+=item publicinbox.<name>.listid
+
+The L<rfc2919|https://tools.ietf.org/html/rfc2919> header without
+angle brackets for L<public-inbox-mda(1)> deliveries and
+L<public-inbox-watch(1)>.
+
+For public-inbox-watch users, this is a shortcut for specifying
+C<publicinbox.$NAME.watchheader=List-Id:<foo.example.com>>
+
+For public-inbox-mda users, this may be used to avoid recipient
+matching via C<ORIGINAL_RECIPIENT> environment variable.
+
+This may be specified multiple times for merging multiple mailing
+lists into a single public-inbox, only one C<List-Id> header
+needs to match.
+
+Default: none
+
=item publicinbox.<name>.nntpmirror
This may be the full NNTP URL of an independently-run mirror.
diff --git a/Documentation/public-inbox-mda.pod b/Documentation/public-inbox-mda.pod
index 64ec690c..921b7a15 100644
--- a/Documentation/public-inbox-mda.pod
+++ b/Documentation/public-inbox-mda.pod
@@ -25,6 +25,9 @@ L<public-inbox-config(5)/publicinboxmda.spamcheck>
The original recipient email address, set by the MTA. Postfix
sets it by default, untested on other MTAs.
+This does not have to be set if relying on C<publicinbox.$NAME.listid>
+directives configured in L<public-inbox-config(5)>.
+
=item PI_CONFIG
Per-user config file parseable by L<git-config(1)>.
diff --git a/lib/PublicInbox/WatchMaildir.pm b/lib/PublicInbox/WatchMaildir.pm
index f63140c8..08b1aab4 100644
--- a/lib/PublicInbox/WatchMaildir.pm
+++ b/lib/PublicInbox/WatchMaildir.pm
@@ -59,9 +59,19 @@ sub new {
my $watch = $ibx->{watch} or return;
if (is_maildir($watch)) {
- if (my $wm = $ibx->{watchheader}) {
- my ($k, $v) = split(/:/, $wm, 2);
- $ibx->{-watchheader} = [ $k, qr/\Q$v\E/ ];
+ my $watch_hdrs = [];
+ if (my $wh = $ibx->{watchheader}) {
+ my ($k, $v) = split(/:/, $wh, 2);
+ push @$watch_hdrs, [ $k, qr/\Q$v\E/ ];
+ }
+ if (my $list_ids = $ibx->{listid}) {
+ for (@$list_ids) {
+ my $re = qr/<[ \t]*\Q$_\E[ \t]*>/;
+ push @$watch_hdrs, ['List-Id', $re ];
+ }
+ }
+ if (scalar @$watch_hdrs) {
+ $ibx->{-watchheaders} = $watch_hdrs;
}
my $new = "$watch/new";
my $cur = "$watch/cur";
@@ -159,10 +169,17 @@ sub _try_path {
my $mime = _path_to_mime($path) or next;
my $im = _importer_for($self, $ibx);
- my $wm = $ibx->{-watchheader};
- if ($wm) {
- my $v = $mime->header_obj->header_raw($wm->[0]);
- next unless ($v && $v =~ $wm->[1]);
+ # any header match means it's eligible for the inbox:
+ if (my $watch_hdrs = $ibx->{-watchheaders}) {
+ my $ok;
+ my $hdr = $mime->header_obj;
+ for my $wh (@$watch_hdrs) {
+ my $v = $hdr->header_raw($wh->[0]);
+ next unless defined($v) && $v =~ $wh->[1];
+ $ok = 1;
+ last;
+ }
+ next unless $ok;
}
if (my $scrub = $ibx->filter($im)) {
diff --git a/script/public-inbox-mda b/script/public-inbox-mda
index 4e6e04e2..2655a6c5 100755
--- a/script/public-inbox-mda
+++ b/script/public-inbox-mda
@@ -36,10 +36,21 @@ my $config = PublicInbox::Config->new;
my $key = 'publicinboxmda.spamcheck';
my $default = 'PublicInbox::Spamcheck::Spamc';
my $spamc = PublicInbox::Spamcheck::get($config, $key, $default);
+my $dst;
my $recipient = $ENV{ORIGINAL_RECIPIENT};
-defined $recipient or die "ORIGINAL_RECIPIENT not defined in ENV\n";
-my $dst = $config->lookup($recipient); # first check
-defined $dst or do_exit(67); # EX_NOUSER 5.1.1 user unknown
+if (defined $recipient) {
+ $dst = $config->lookup($recipient); # first check
+}
+if (!defined $dst) {
+ my $list_id = $simple->header('List-Id');
+ if (defined $list_id && $list_id =~ /<[ \t]*(.+)?[ \t]*>/) {
+ $dst = $config->lookup_list_id($1);
+ }
+ if (!defined $dst && !defined $recipient) {
+ die "ORIGINAL_RECIPIENT not defined in ENV\n";
+ }
+ defined $dst or do_exit(67); # EX_NOUSER 5.1.1 user unknown
+}
$dst->{mainrepo} or do_exit(67);
$dst = PublicInbox::InboxWritable->new($dst);
diff --git a/t/mda.t b/t/mda.t
index 5621b7d6..3cab590b 100644
--- a/t/mda.t
+++ b/t/mda.t
@@ -267,6 +267,34 @@ EOF
}
}
+# List-ID based delivery
+{
+ local $ENV{PI_EMERGENCY} = $faildir;
+ local $ENV{HOME} = $home;
+ local $ENV{ORIGINAL_RECIPIENT} = undef;
+ local $ENV{PATH} = $main_path;
+ my $list_id = 'foo.example.com';
+ my $mid = 'list-id-delivery@example.com';
+ my $simple = Email::Simple->new(<<EOF);
+From: user <user\@example.com>
+To: You <you\@example.com>
+Cc: $addr
+Message-ID: <$mid>
+List-Id: <$list_id>
+Subject: this message will be trained as spam
+Date: Thu, 01 Jan 1970 00:00:00 +0000
+
+EOF
+ system(qw(git config --file), $pi_config, "$cfgpfx.listid", $list_id);
+ $? == 0 or die "failed to set listid $?";
+ my $in = $simple->as_string;
+ IPC::Run::run([$mda], \$in);
+ is($?, 0, 'mda OK with List-Id match');
+ my $path = mid2path($mid);
+ my $msg = `git --git-dir=$maindir cat-file blob HEAD:$path`;
+ like($msg, qr/\Q$list_id\E/, 'delivered message w/ List-ID matches');
+}
+
done_testing();
sub fail_bad_header {
diff --git a/t/watch_maildir_v2.t b/t/watch_maildir_v2.t
index 0a5a8017..99551ceb 100644
--- a/t/watch_maildir_v2.t
+++ b/t/watch_maildir_v2.t
@@ -171,4 +171,35 @@ EOF
is($both, $$msg, 'got original message back from v2');
}
+{
+ my $want = <<'EOF';
+From: <u@example.com>
+List-Id: <i.want.you.to.want.me>
+Message-ID: <do.want@example.com>
+EOF
+ my $do_not_want = <<'EOF';
+From: <u@example.com>
+List-Id: <do.not.want>
+X-Mailing-List: no@example.com
+Message-ID: <do.not.want@example.com>
+EOF
+ my $cfg = $orig."$cfgpfx.listid=i.want.you.to.want.me\n";
+ PublicInbox::Emergency->new($maildir)->prepare(\$want);
+ PublicInbox::Emergency->new($maildir)->prepare(\$do_not_want);
+ my $config = PublicInbox::Config->new(\$cfg);
+ PublicInbox::WatchMaildir->new($config)->scan('full');
+ $ibx = $config->lookup_name('test');
+ my $num = $ibx->mm->num_for('do.want@example.com');
+ ok(defined $num, 'List-ID matched for watch');
+ $num = $ibx->mm->num_for('do.not.want@example.com');
+ is($num, undef, 'unaccepted List-ID matched for watch');
+
+ $cfg = $orig."$cfgpfx.watchheader=X-Mailing-List:no\@example.com\n";
+ $config = PublicInbox::Config->new(\$cfg);
+ PublicInbox::WatchMaildir->new($config)->scan('full');
+ $ibx = $config->lookup_name('test');
+ $num = $ibx->mm->num_for('do.not.want@example.com');
+ ok(defined $num, 'X-Mailing-List matched');
+}
+
done_testing;
^ permalink raw reply related [relevance 4%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-10-15 20:36 7% [PATCH 0/7] List-ID support in mda, watch, www Eric Wong
2019-10-15 20:36 4% ` [PATCH 6/7] mda, watch: wire up List-ID header support Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).