user/dev discussion of public-inbox itself
 help / color / mirror / Atom feed
* 2 problems with listid matching
@ 2020-09-21 18:01 Konstantin Ryabitsev
  2020-09-21 20:58 ` [PATCH] mda: match List-Id insensitively Eric Wong
  0 siblings, 1 reply; 2+ messages in thread
From: Konstantin Ryabitsev @ 2020-09-21 18:01 UTC (permalink / raw)
  To: meta

Hello:

Attempting to subscribe radiotap@radiotap.org has highlighted two 
problems with list-id matching. When the email comes in from the mailing 
list, the header is set as:

  List-Id: radiotap.NetBSD.org

Public-inbox doesn't find this because the above list-id header is not 
compliant with the RFC (it should be inside angle brackets). However, 
even when <> are added, the match still fails due to capitalization: the 
List-Id value from the email header is lc'd first before it is compared 
with the listid= value in the config file (which isn't lc'd). So, if the 
config file value is using capitalization, the match will never succeed.

I think public-inbox should recognize this list-id header even though 
it's not compliant, and it should lc both values before comparing them, 
since the canonical value uses capitalization.

Best regards,
-K

^ permalink raw reply	[flat|nested] 2+ messages in thread

* [PATCH] mda: match List-Id insensitively
  2020-09-21 18:01 2 problems with listid matching Konstantin Ryabitsev
@ 2020-09-21 20:58 ` Eric Wong
  0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2020-09-21 20:58 UTC (permalink / raw)
  To: Konstantin Ryabitsev; +Cc: meta

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> Hello:
> 
> Attempting to subscribe radiotap@radiotap.org has highlighted two 
> problems with list-id matching. When the email comes in from the mailing 
> list, the header is set as:
> 
>   List-Id: radiotap.NetBSD.org
> 
> Public-inbox doesn't find this because the above list-id header is not 
> compliant with the RFC (it should be inside angle brackets). However, 
> even when <> are added, the match still fails due to capitalization: the 
> List-Id value from the email header is lc'd first before it is compared 
> with the listid= value in the config file (which isn't lc'd). So, if the 
> config file value is using capitalization, the match will never succeed.

The lack of lc for the -mda code path is definitely a bug and
also inconsistent with -watch behavior.  The patch below fixes
it, thanks.

> I think public-inbox should recognize this list-id header even though 
> it's not compliant, and it should lc both values before comparing them, 
> since the canonical value uses capitalization.

We'll have to think about that one...  It probably needs to be
an case-insensitive match of the entire header contents to avoid
inadvertant substring matching.  RFC 2919 allows a phrase
element before the list-id.

With -watch, using "watchheader=List-Id:radiotap.NetBSD.org"
(case-sensitive) can workaround it, but -mda doesn't support
watchheader or anything like it right now...

-------8<-------
Subject: [PATCH] mda: match List-Id insensitively

This follows -watch commit b70473ab8296d31ebb600adb4fa8fe0ac5935ca8
to match List-Id headers case-insensitively.

Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20200921180152.uyqluod7qxbwqubo@chatter.i7.local/
---
 lib/PublicInbox/Config.pm | 3 ++-
 t/mda.t                   | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm
index abc525db..d57c361a 100644
--- a/lib/PublicInbox/Config.pm
+++ b/lib/PublicInbox/Config.pm
@@ -424,8 +424,9 @@ EOF
 		$self->{-no_obfuscate}->{$lc_addr} = 1;
 	}
 	if (my $listids = $ibx->{listid}) {
+		# RFC2919 section 6 stipulates "case insensitive equality"
 		foreach my $list_id (@$listids) {
-			$self->{-by_list_id}->{$list_id} = $ibx;
+			$self->{-by_list_id}->{lc($list_id)} = $ibx;
 		}
 	}
 	if (my $ng = $ibx->{newsgroup}) {
diff --git a/t/mda.t b/t/mda.t
index c7caf3e0..c5b35eec 100644
--- a/t/mda.t
+++ b/t/mda.t
@@ -261,7 +261,7 @@ Subject: this message will be trained as spam
 Date: Thu, 01 Jan 1970 00:00:00 +0000
 
 EOF
-	xsys(qw(git config --file), $pi_config, "$cfgpfx.listid", $list_id);
+	xsys(qw(git config --file), $pi_config, "$cfgpfx.listid", uc $list_id);
 	$? == 0 or die "failed to set listid $?";
 	ok(run_script(['-mda'], undef, { 0 => \$in }),
 		'mda OK with List-Id match');

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2020-09-21 20:58 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-21 18:01 2 problems with listid matching Konstantin Ryabitsev
2020-09-21 20:58 ` [PATCH] mda: match List-Id insensitively Eric Wong

user/dev discussion of public-inbox itself

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 meta meta/ https://public-inbox.org/meta \
		meta@public-inbox.org
	public-inbox-index meta

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git