From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS6315 166.70.0.0/16 X-Spam-Status: No, score=-3.6 required=3.0 tests=AWL,BAYES_00, RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from out01.mta.xmission.com (out01.mta.xmission.com [166.70.13.231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id A11891F4C0; Mon, 28 Oct 2019 18:07:03 +0000 (UTC) Received: from in01.mta.xmission.com ([166.70.13.51]) by out01.mta.xmission.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1iP9Py-0007t3-2f; Mon, 28 Oct 2019 12:07:02 -0600 Received: from ip68-227-160-95.om.om.cox.net ([68.227.160.95] helo=x220.xmission.com) by in01.mta.xmission.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.87) (envelope-from ) id 1iP9Pv-0007xM-SC; Mon, 28 Oct 2019 12:07:01 -0600 From: ebiederm@xmission.com (Eric W. Biederman) To: Eric Wong Cc: meta@public-inbox.org References: <20191028104528.10140-1-e@80x24.org> <20191028104528.10140-13-e@80x24.org> Date: Mon, 28 Oct 2019 13:05:51 -0500 In-Reply-To: <20191028104528.10140-13-e@80x24.org> (Eric Wong's message of "Mon, 28 Oct 2019 10:45:26 +0000") Message-ID: <874kzssq2o.fsf@x220.int.ebiederm.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-SPF: eid=1iP9Pv-0007xM-SC;;;mid=<874kzssq2o.fsf@x220.int.ebiederm.org>;;;hst=in01.mta.xmission.com;;;ip=68.227.160.95;;;frm=ebiederm@xmission.com;;;spf=neutral X-XM-AID: U2FsdGVkX181mLoRLjcwkJ8OdRbLNKX5JZUrHwwynhI= X-SA-Exim-Connect-IP: 68.227.160.95 X-SA-Exim-Mail-From: ebiederm@xmission.com Subject: Re: [PATCH 12/14] mda: support multiple List-ID matches X-SA-Exim-Version: 4.2.1 (built Thu, 05 May 2016 13:38:54 -0600) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) List-Id: Eric Wong writes: > While it's not RFC2919-conformant, mail software can > theoretically set multiple List-ID headers. Deliver to all > inboxes which match a given List-ID since that's likely the > intended. There is a todo line you can kill, noted below. There should probably be a warning about List-ID's you can't look up. In case of misconfiguration or you subscribe to an extra mail-box and have not yet configured the List-ID for the list. I don't know how to find the List-ID ahead of time so it seems inevitiable that there will be a couple messages with an uncofigured List-ID. If you are not receiving from a mailling list you might get spam or other unsolicited email from someone's list server. Knowing the List-ID of that email is probably also useful. Knowing that this kind of non-sense exists guarantees that there will be email whose List-ID won't be configured. > Cc: Eric W. Biederman > Link: https://public-inbox.org/meta/87pniltscf.fsf@x220.int.ebiederm.org/ > --- > lib/PublicInbox/MDA.pm | 19 +++++++++++++------ > script/public-inbox-learn | 5 +++-- > script/public-inbox-mda | 7 +++---- > t/mda.t | 19 +++++++++++++++++++ > 4 files changed, 38 insertions(+), 12 deletions(-) > > diff --git a/lib/PublicInbox/MDA.pm b/lib/PublicInbox/MDA.pm > index ce2c870f..933d82a8 100644 > --- a/lib/PublicInbox/MDA.pm > +++ b/lib/PublicInbox/MDA.pm > @@ -84,18 +84,25 @@ sub set_list_headers { > } > > # TODO: deal with multiple List-ID headers? ^^^^^^^^^^^^^^^^^^------ You can kill this line now. > -sub inbox_for_list_id ($$) { > +sub inboxes_for_list_id ($$) { > my ($klass, $config, $simple) = @_; > > # newer Email::Simple allows header_raw, as does Email::MIME: > - my $list_id = $simple->can('header_raw') ? > + my @list_ids = $simple->can('header_raw') ? > $simple->header_raw('List-Id') : > $simple->header('List-Id'); > - my $ibx; > - if (defined $list_id && $list_id =~ /<[ \t]*(.+)?[ \t]*>/) { > - $ibx = $config->lookup_list_id($1); > + my @dests; > + for my $list_id (@list_ids) { > + $list_id =~ /<[ \t]*(.+)?[ \t]*>/ or next; > + if (my $ibx = $config->lookup_list_id($1)) { > + push @dests, $ibx; > + } > + } > + if (scalar(@list_ids) > 1) { > + warn "W: multiple List-IDs in message:\n"; > + warn "W: List-ID: $_\n" for @list_ids > } > - $ibx; > + \@dests; > } > > 1; > diff --git a/script/public-inbox-learn b/script/public-inbox-learn > index 79f3ead5..3073294a 100644 > --- a/script/public-inbox-learn > +++ b/script/public-inbox-learn > @@ -95,8 +95,9 @@ if ($train eq 'spam') { > next if $seen{"$ibx"}++; > remove_or_add($ibx, $train, $addr); > } > - my $ibx = PublicInbox::MDA->inbox_for_list_id($pi_config, $mime); > - if ($ibx && !$seen{"$ibx"}) { > + my $dests = PublicInbox::MDA->inboxes_for_list_id($pi_config, $mime); > + for my $ibx (@$dests) { > + next if !$seen{"$ibx"}++; > remove_or_add($ibx, $train, $ibx->{-primary_address}); > } > } > diff --git a/script/public-inbox-mda b/script/public-inbox-mda > index 821bd9cc..dca8a0ea 100755 > --- a/script/public-inbox-mda > +++ b/script/public-inbox-mda > @@ -44,12 +44,11 @@ if (defined $recipient) { > push @$dests, $ibx if $ibx; > } > if (!scalar(@$dests)) { > - my $ibx = PublicInbox::MDA->inbox_for_list_id($config, $simple); > - if (!defined($ibx) && !defined($recipient)) { > + $dests = PublicInbox::MDA->inboxes_for_list_id($config, $simple); > + if (!scalar(@$dests) && !defined($recipient)) { > die "ORIGINAL_RECIPIENT not defined in ENV\n"; > } > - defined($ibx) or do_exit(67); # EX_NOUSER 5.1.1 user unknown > - push @$dests, $ibx; > + scalar(@$dests) or do_exit(67); # EX_NOUSER 5.1.1 user unknown > } > > my $err; > diff --git a/t/mda.t b/t/mda.t > index 99592b2d..35811ac6 100644 > --- a/t/mda.t > +++ b/t/mda.t > @@ -308,6 +308,25 @@ EOF > my $cur = `git --git-dir=$maindir diff HEAD~1..HEAD`; > like($cur, qr/this message would not be accepted without --no-precheck/, > '--no-precheck delivered message anyways'); > + > + # try a message with multiple List-ID headers > + $in = < +List-ID: > +List-ID: <$list_id> > +Message-ID: <2lids\@example> > +Subject: two List-IDs > +From: user > +To: $addr > +Date: Fri, 02 Oct 1993 00:00:00 +0000 > + > +EOF > + ($out, $err) = ('', ''); > + IPC::Run::run([$mda], \$in, \$out, \$err); > + is($?, 0, 'mda OK with multiple List-Id matches'); > + $cur = `git --git-dir=$maindir diff HEAD~1..HEAD`; > + like($cur, qr/Message-ID: <2lids\@example>/, > + 'multi List-ID match delivered'); > + like($err, qr/multiple List-ID/, 'warned about multiple List-ID'); > } > > done_testing();