* [PATCH 1/2] learn: fix buggy typo on List-ID mapping
@ 2020-05-26 9:05 5% ` Eric Wong
0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2020-05-26 9:05 UTC (permalink / raw)
To: meta
There is obviously a typo here, so fix it and add a test
case to guard against future regressions.
Fixes: 74a3206babe0572a ("mda: support multiple List-ID matches")
---
script/public-inbox-learn | 2 +-
t/mda.t | 10 +++++++++-
2 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/script/public-inbox-learn b/script/public-inbox-learn
index a33d813ad01..0cb2c8e96e5 100644
--- a/script/public-inbox-learn
+++ b/script/public-inbox-learn
@@ -97,7 +97,7 @@ if ($train eq 'spam') {
}
my $dests = PublicInbox::MDA->inboxes_for_list_id($pi_config, $mime);
for my $ibx (@$dests) {
- next if !$seen{"$ibx"}++;
+ next if $seen{"$ibx"}++;
remove_or_add($ibx, $train, $mime, $ibx->{-primary_address});
}
}
diff --git a/t/mda.t b/t/mda.t
index 759c0b020ab..c7caf3e0cca 100644
--- a/t/mda.t
+++ b/t/mda.t
@@ -299,9 +299,17 @@ EOF
ok(run_script(['-mda'], undef, $rdr),
'mda OK with multiple List-Id matches');
$cur = $git->qx(qw(diff HEAD~1..HEAD));
- like($cur, qr/Message-ID: <2lids\@example>/,
+ like($cur, qr/^\+Message-ID: <2lids\@example>/sm,
'multi List-ID match delivered');
like($err, qr/multiple List-ID/, 'warned about multiple List-ID');
+
+ # ensure -learn rm works after inbox address is updated
+ ($out, $err) = ('', '');
+ xsys(qw(git config --file), $pi_config, "$cfgpfx.address",
+ 'updated-address@example.com');
+ ok(run_script(['-learn', 'rm'], undef, $rdr), 'rm-ed via -learn');
+ $cur = $git->qx(qw(diff HEAD~1..HEAD));
+ like($cur, qr/^-Message-ID: <2lids\@example>/sm, 'changed in git');
}
done_testing();
^ permalink raw reply related [relevance 5%]
* Re: [PATCH 12/14] mda: support multiple List-ID matches
2019-10-28 18:05 7% ` Eric W. Biederman
@ 2019-10-30 21:32 7% ` Eric Wong
0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2019-10-30 21:32 UTC (permalink / raw)
To: Eric W. Biederman; +Cc: meta
"Eric W. Biederman" <ebiederm@xmission.com> wrote:
> Eric Wong <e@80x24.org> writes:
>
> > While it's not RFC2919-conformant, mail software can
> > theoretically set multiple List-ID headers. Deliver to all
> > inboxes which match a given List-ID since that's likely the
> > intended.
>
> There is a todo line you can kill, noted below.
Done and pushed
> There should probably be a warning about List-ID's you can't
> look up.
>
> In case of misconfiguration or you subscribe to an extra mail-box and
> have not yet configured the List-ID for the list. I don't know how to
> find the List-ID ahead of time so it seems inevitiable that there will
> be a couple messages with an uncofigured List-ID.
I'm not so sure about that... We don't warn on existing cases
involving ORIGINAL_RECIPIENT/To/Cc. Instead, it goes to
~/.public-inbox/emergency/ (or whatever PI_EMERGENCY is set to).
> If you are not receiving from a mailling list you might get spam or
> other unsolicited email from someone's list server. Knowing the List-ID
> of that email is probably also useful. Knowing that this kind of
> non-sense exists guarantees that there will be email whose List-ID won't
> be configured.
Given we already toss undeliverables into an emergency/ Maildir;
I don't think training users to look for warnings in noisy (and
potentially access-limited) mail logs is necessary.
> > Cc: Eric W. Biederman <ebiederm@xmission.com>
> > Link: https://public-inbox.org/meta/87pniltscf.fsf@x220.int.ebiederm.org/
> > ---
> > diff --git a/lib/PublicInbox/MDA.pm b/lib/PublicInbox/MDA.pm
> > index ce2c870f..933d82a8 100644
> > --- a/lib/PublicInbox/MDA.pm
> > +++ b/lib/PublicInbox/MDA.pm
> > @@ -84,18 +84,25 @@ sub set_list_headers {
> > }
> >
> > # TODO: deal with multiple List-ID headers?
> ^^^^^^^^^^^^^^^^^^------ You can kill this line now.
Yup. I also have lots of TODO comments throughout which
need to be updated/removed :x
^ permalink raw reply [relevance 7%]
* Re: [PATCH 12/14] mda: support multiple List-ID matches
2019-10-28 10:45 5% ` [PATCH 12/14] mda: support multiple List-ID matches Eric Wong
@ 2019-10-28 18:05 7% ` Eric W. Biederman
2019-10-30 21:32 7% ` Eric Wong
0 siblings, 1 reply; 5+ results
From: Eric W. Biederman @ 2019-10-28 18:05 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
Eric Wong <e@80x24.org> writes:
> While it's not RFC2919-conformant, mail software can
> theoretically set multiple List-ID headers. Deliver to all
> inboxes which match a given List-ID since that's likely the
> intended.
There is a todo line you can kill, noted below.
There should probably be a warning about List-ID's you can't
look up.
In case of misconfiguration or you subscribe to an extra mail-box and
have not yet configured the List-ID for the list. I don't know how to
find the List-ID ahead of time so it seems inevitiable that there will
be a couple messages with an uncofigured List-ID.
If you are not receiving from a mailling list you might get spam or
other unsolicited email from someone's list server. Knowing the List-ID
of that email is probably also useful. Knowing that this kind of
non-sense exists guarantees that there will be email whose List-ID won't
be configured.
> Cc: Eric W. Biederman <ebiederm@xmission.com>
> Link: https://public-inbox.org/meta/87pniltscf.fsf@x220.int.ebiederm.org/
> ---
> lib/PublicInbox/MDA.pm | 19 +++++++++++++------
> script/public-inbox-learn | 5 +++--
> script/public-inbox-mda | 7 +++----
> t/mda.t | 19 +++++++++++++++++++
> 4 files changed, 38 insertions(+), 12 deletions(-)
>
> diff --git a/lib/PublicInbox/MDA.pm b/lib/PublicInbox/MDA.pm
> index ce2c870f..933d82a8 100644
> --- a/lib/PublicInbox/MDA.pm
> +++ b/lib/PublicInbox/MDA.pm
> @@ -84,18 +84,25 @@ sub set_list_headers {
> }
>
> # TODO: deal with multiple List-ID headers?
^^^^^^^^^^^^^^^^^^------ You can kill this line now.
> -sub inbox_for_list_id ($$) {
> +sub inboxes_for_list_id ($$) {
> my ($klass, $config, $simple) = @_;
>
> # newer Email::Simple allows header_raw, as does Email::MIME:
> - my $list_id = $simple->can('header_raw') ?
> + my @list_ids = $simple->can('header_raw') ?
> $simple->header_raw('List-Id') :
> $simple->header('List-Id');
> - my $ibx;
> - if (defined $list_id && $list_id =~ /<[ \t]*(.+)?[ \t]*>/) {
> - $ibx = $config->lookup_list_id($1);
> + my @dests;
> + for my $list_id (@list_ids) {
> + $list_id =~ /<[ \t]*(.+)?[ \t]*>/ or next;
> + if (my $ibx = $config->lookup_list_id($1)) {
> + push @dests, $ibx;
> + }
> + }
> + if (scalar(@list_ids) > 1) {
> + warn "W: multiple List-IDs in message:\n";
> + warn "W: List-ID: $_\n" for @list_ids
> }
> - $ibx;
> + \@dests;
> }
>
> 1;
> diff --git a/script/public-inbox-learn b/script/public-inbox-learn
> index 79f3ead5..3073294a 100644
> --- a/script/public-inbox-learn
> +++ b/script/public-inbox-learn
> @@ -95,8 +95,9 @@ if ($train eq 'spam') {
> next if $seen{"$ibx"}++;
> remove_or_add($ibx, $train, $addr);
> }
> - my $ibx = PublicInbox::MDA->inbox_for_list_id($pi_config, $mime);
> - if ($ibx && !$seen{"$ibx"}) {
> + my $dests = PublicInbox::MDA->inboxes_for_list_id($pi_config, $mime);
> + for my $ibx (@$dests) {
> + next if !$seen{"$ibx"}++;
> remove_or_add($ibx, $train, $ibx->{-primary_address});
> }
> }
> diff --git a/script/public-inbox-mda b/script/public-inbox-mda
> index 821bd9cc..dca8a0ea 100755
> --- a/script/public-inbox-mda
> +++ b/script/public-inbox-mda
> @@ -44,12 +44,11 @@ if (defined $recipient) {
> push @$dests, $ibx if $ibx;
> }
> if (!scalar(@$dests)) {
> - my $ibx = PublicInbox::MDA->inbox_for_list_id($config, $simple);
> - if (!defined($ibx) && !defined($recipient)) {
> + $dests = PublicInbox::MDA->inboxes_for_list_id($config, $simple);
> + if (!scalar(@$dests) && !defined($recipient)) {
> die "ORIGINAL_RECIPIENT not defined in ENV\n";
> }
> - defined($ibx) or do_exit(67); # EX_NOUSER 5.1.1 user unknown
> - push @$dests, $ibx;
> + scalar(@$dests) or do_exit(67); # EX_NOUSER 5.1.1 user unknown
> }
>
> my $err;
> diff --git a/t/mda.t b/t/mda.t
> index 99592b2d..35811ac6 100644
> --- a/t/mda.t
> +++ b/t/mda.t
> @@ -308,6 +308,25 @@ EOF
> my $cur = `git --git-dir=$maindir diff HEAD~1..HEAD`;
> like($cur, qr/this message would not be accepted without --no-precheck/,
> '--no-precheck delivered message anyways');
> +
> + # try a message with multiple List-ID headers
> + $in = <<EOF;
> +List-ID: <foo.bar>
> +List-ID: <$list_id>
> +Message-ID: <2lids\@example>
> +Subject: two List-IDs
> +From: user <user\@example.com>
> +To: $addr
> +Date: Fri, 02 Oct 1993 00:00:00 +0000
> +
> +EOF
> + ($out, $err) = ('', '');
> + IPC::Run::run([$mda], \$in, \$out, \$err);
> + is($?, 0, 'mda OK with multiple List-Id matches');
> + $cur = `git --git-dir=$maindir diff HEAD~1..HEAD`;
> + like($cur, qr/Message-ID: <2lids\@example>/,
> + 'multi List-ID match delivered');
> + like($err, qr/multiple List-ID/, 'warned about multiple List-ID');
> }
>
> done_testing();
^ permalink raw reply [relevance 7%]
* [PATCH 12/14] mda: support multiple List-ID matches
2019-10-28 10:45 6% [PATCH 00/14] learn: sync w/ -mda changes and add manpage Eric Wong
@ 2019-10-28 10:45 5% ` Eric Wong
2019-10-28 18:05 7% ` Eric W. Biederman
0 siblings, 1 reply; 5+ results
From: Eric Wong @ 2019-10-28 10:45 UTC (permalink / raw)
To: meta; +Cc: Eric W . Biederman
While it's not RFC2919-conformant, mail software can
theoretically set multiple List-ID headers. Deliver to all
inboxes which match a given List-ID since that's likely the
intended.
Cc: Eric W. Biederman <ebiederm@xmission.com>
Link: https://public-inbox.org/meta/87pniltscf.fsf@x220.int.ebiederm.org/
---
lib/PublicInbox/MDA.pm | 19 +++++++++++++------
script/public-inbox-learn | 5 +++--
script/public-inbox-mda | 7 +++----
t/mda.t | 19 +++++++++++++++++++
4 files changed, 38 insertions(+), 12 deletions(-)
diff --git a/lib/PublicInbox/MDA.pm b/lib/PublicInbox/MDA.pm
index ce2c870f..933d82a8 100644
--- a/lib/PublicInbox/MDA.pm
+++ b/lib/PublicInbox/MDA.pm
@@ -84,18 +84,25 @@ sub set_list_headers {
}
# TODO: deal with multiple List-ID headers?
-sub inbox_for_list_id ($$) {
+sub inboxes_for_list_id ($$) {
my ($klass, $config, $simple) = @_;
# newer Email::Simple allows header_raw, as does Email::MIME:
- my $list_id = $simple->can('header_raw') ?
+ my @list_ids = $simple->can('header_raw') ?
$simple->header_raw('List-Id') :
$simple->header('List-Id');
- my $ibx;
- if (defined $list_id && $list_id =~ /<[ \t]*(.+)?[ \t]*>/) {
- $ibx = $config->lookup_list_id($1);
+ my @dests;
+ for my $list_id (@list_ids) {
+ $list_id =~ /<[ \t]*(.+)?[ \t]*>/ or next;
+ if (my $ibx = $config->lookup_list_id($1)) {
+ push @dests, $ibx;
+ }
+ }
+ if (scalar(@list_ids) > 1) {
+ warn "W: multiple List-IDs in message:\n";
+ warn "W: List-ID: $_\n" for @list_ids
}
- $ibx;
+ \@dests;
}
1;
diff --git a/script/public-inbox-learn b/script/public-inbox-learn
index 79f3ead5..3073294a 100644
--- a/script/public-inbox-learn
+++ b/script/public-inbox-learn
@@ -95,8 +95,9 @@ if ($train eq 'spam') {
next if $seen{"$ibx"}++;
remove_or_add($ibx, $train, $addr);
}
- my $ibx = PublicInbox::MDA->inbox_for_list_id($pi_config, $mime);
- if ($ibx && !$seen{"$ibx"}) {
+ my $dests = PublicInbox::MDA->inboxes_for_list_id($pi_config, $mime);
+ for my $ibx (@$dests) {
+ next if !$seen{"$ibx"}++;
remove_or_add($ibx, $train, $ibx->{-primary_address});
}
}
diff --git a/script/public-inbox-mda b/script/public-inbox-mda
index 821bd9cc..dca8a0ea 100755
--- a/script/public-inbox-mda
+++ b/script/public-inbox-mda
@@ -44,12 +44,11 @@ if (defined $recipient) {
push @$dests, $ibx if $ibx;
}
if (!scalar(@$dests)) {
- my $ibx = PublicInbox::MDA->inbox_for_list_id($config, $simple);
- if (!defined($ibx) && !defined($recipient)) {
+ $dests = PublicInbox::MDA->inboxes_for_list_id($config, $simple);
+ if (!scalar(@$dests) && !defined($recipient)) {
die "ORIGINAL_RECIPIENT not defined in ENV\n";
}
- defined($ibx) or do_exit(67); # EX_NOUSER 5.1.1 user unknown
- push @$dests, $ibx;
+ scalar(@$dests) or do_exit(67); # EX_NOUSER 5.1.1 user unknown
}
my $err;
diff --git a/t/mda.t b/t/mda.t
index 99592b2d..35811ac6 100644
--- a/t/mda.t
+++ b/t/mda.t
@@ -308,6 +308,25 @@ EOF
my $cur = `git --git-dir=$maindir diff HEAD~1..HEAD`;
like($cur, qr/this message would not be accepted without --no-precheck/,
'--no-precheck delivered message anyways');
+
+ # try a message with multiple List-ID headers
+ $in = <<EOF;
+List-ID: <foo.bar>
+List-ID: <$list_id>
+Message-ID: <2lids\@example>
+Subject: two List-IDs
+From: user <user\@example.com>
+To: $addr
+Date: Fri, 02 Oct 1993 00:00:00 +0000
+
+EOF
+ ($out, $err) = ('', '');
+ IPC::Run::run([$mda], \$in, \$out, \$err);
+ is($?, 0, 'mda OK with multiple List-Id matches');
+ $cur = `git --git-dir=$maindir diff HEAD~1..HEAD`;
+ like($cur, qr/Message-ID: <2lids\@example>/,
+ 'multi List-ID match delivered');
+ like($err, qr/multiple List-ID/, 'warned about multiple List-ID');
}
done_testing();
^ permalink raw reply related [relevance 5%]
* [PATCH 00/14] learn: sync w/ -mda changes and add manpage
@ 2019-10-28 10:45 6% Eric Wong
2019-10-28 10:45 5% ` [PATCH 12/14] mda: support multiple List-ID matches Eric Wong
0 siblings, 1 reply; 5+ results
From: Eric Wong @ 2019-10-28 10:45 UTC (permalink / raw)
To: meta
What started with adding a manpage for public-inbox-learn,
ended up being a bunch of fixes and improvements to catch
up to -mda changes.
-mda also learned to deal with multiple List-ID headers in the
meantime.
Eric Wong (14):
learn: support multiple To/Cc headers
learn: only map recipient list on "ham" or "rm"
learn: update usage statement
learn: GIT_COMMITTER_<NAME|EMAIL> may be "" or "0"
learn: hoist out remove_or_add subroutine
mda: hoist out List-ID handling and reuse in -learn
filter/base: remove MAX_MID_SIZE constant
mda: hoist out mda_filter_adjust
mda: skip MIME parsing if spam
inboxwritable: add assert_usable_dir sub
mda: prepare for multiple destinations
mda: support multiple List-ID matches
learn: allow running without spamc
doc: add public-inbox-learn(1) manpage
Documentation/include.mk | 1 +
Documentation/public-inbox-learn.pod | 86 +++++++++++++++++++++
MANIFEST | 1 +
lib/PublicInbox/Filter/Base.pm | 1 -
lib/PublicInbox/InboxWritable.pm | 9 ++-
lib/PublicInbox/MDA.pm | 22 ++++++
lib/PublicInbox/V2Writable.pm | 5 +-
script/public-inbox-learn | 84 +++++++++++---------
script/public-inbox-mda | 110 ++++++++++++++++-----------
t/import.t | 8 ++
t/mda.t | 19 +++++
t/v2writable.t | 12 +++
12 files changed, 275 insertions(+), 83 deletions(-)
create mode 100644 Documentation/public-inbox-learn.pod
mode change 100755 => 100644 script/public-inbox-learn
^ permalink raw reply [relevance 6%]
Results 1-5 of 5 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-10-28 10:45 6% [PATCH 00/14] learn: sync w/ -mda changes and add manpage Eric Wong
2019-10-28 10:45 5% ` [PATCH 12/14] mda: support multiple List-ID matches Eric Wong
2019-10-28 18:05 7% ` Eric W. Biederman
2019-10-30 21:32 7% ` Eric Wong
2020-05-26 9:05 [PATCH 0/2] -learn fixes and updates Eric Wong
2020-05-26 9:05 5% ` [PATCH 1/2] learn: fix buggy typo on List-ID mapping Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).