[PATCH 0/5] www: coderepo updates

[PATCH 0/5] www: coderepo updates

[Eric Wong]

1+2 are old cleanups I finally got around to...

3-5 fix and support globbing in URIs for to make it easier to
filter projects by basename or username/projectname.

In other words, https://yhbt.net/lore/**/vfs.git can
quickly find all the vfs.git repos mirrored from git.k.o.
Apparently, '*' is fine unescaped in URLs :o

I'll probably rely on the basename to make solver and
inbox <=> coderepo mappings display more nicely in WWW
(but I keep getting distracted playing w/ Parse::RecDescent :x)

Eric Wong (5):
  tests: clarify Email::MIME is only for development
  examples/unsubscribe-milter@.service: use KillMode=process
  config: don't vivify invalid fields for coderepos
  config: glob2re: fix over-matching /**/foo
  www: repolist: support globbing in URL

 examples/unsubscribe-milter@.service | 6 ++++++
 lib/PublicInbox/Config.pm            | 6 +++---
 lib/PublicInbox/RepoList.pm          | 4 ++--
 lib/PublicInbox/TestCommon.pm        | 1 +
 lib/PublicInbox/WwwCoderepo.pm       | 6 +++++-
 t/config.t                           | 3 ++-
 t/eml.t                              | 6 +++---
 t/import.t                           | 3 ++-
 t/mime.t                             | 4 ++--
 9 files changed, 26 insertions(+), 13 deletions(-)

[PATCH 1/5] tests: clarify Email::MIME is only for development

[Eric Wong]

We moved to PublicInbox::Eml a while back and have no plans
to go back to using Email::MIME, so don't tempt users and
packagers to waste disk space on Email::MIME.
---
 lib/PublicInbox/TestCommon.pm | 1 +
 t/eml.t                       | 6 +++---
 t/import.t                    | 3 ++-
 t/mime.t                      | 4 ++--
 4 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/PublicInbox/TestCommon.pm b/lib/PublicInbox/TestCommon.pm
index 25caaaa9..5f159683 100644
--- a/lib/PublicInbox/TestCommon.pm
+++ b/lib/PublicInbox/TestCommon.pm
@@ -266,6 +266,7 @@ sub require_mods {
 	}
 	return unless @need;
 	my $m = join(', ', @need)." missing for $0";
+	$m =~ s/\bEmail::MIME\b/Email::MIME (development purposes only)/;
 	skip($m, $maybe) if $maybe;
 	plan(skip_all => $m)
 }
diff --git a/t/eml.t b/t/eml.t
index 2e6a441f..690ada57 100644
--- a/t/eml.t
+++ b/t/eml.t
@@ -1,8 +1,8 @@
 #!perl -w
-# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+use v5.10.1; # TODO: check unicode_strings w/ 5.12
 use strict;
-use Test::More;
 use PublicInbox::TestCommon;
 use PublicInbox::MsgIter qw(msg_part_text);
 my @classes = qw(PublicInbox::Eml);
@@ -355,7 +355,7 @@ if ('maxparts is a feature unique to us') {
 }
 
 SKIP: {
-	require_mods('PublicInbox::MIME', 1);
+	require_mods('Email::MIME', 1);
 	my $eml = eml_load 't/utf8.eml';
 	my $mime = mime_load 't/utf8.eml';
 	for my $h (qw(Subject From To)) {
diff --git a/t/import.t b/t/import.t
index f1d61dae..7e2432e7 100644
--- a/t/import.t
+++ b/t/import.t
@@ -98,7 +98,8 @@ ok($@, 'Import->add fails on non-existent dir');
 
 my @cls = qw(PublicInbox::Eml);
 SKIP: {
-	require_mods('PublicInbox::MIME', 1);
+	require_mods('Email::MIME', 1);
+	require PublicInbox::MIME;
 	push @cls, 'PublicInbox::MIME';
 };
 
diff --git a/t/mime.t b/t/mime.t
index 471f0efa..bf54118a 100644
--- a/t/mime.t
+++ b/t/mime.t
@@ -1,10 +1,10 @@
 #!perl -w
-# Copyright (C) 2017-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # This library is free software; you can redistribute it and/or modify
 # it under the same terms as Perl itself.
 # Artistic or GPL-1+ <https://www.gnu.org/licenses/gpl-1.0.txt>
+use v5.10.1; # TODO: check unicode_strings w/ v5.12
 use strict;
-use Test::More;
 use PublicInbox::TestCommon;
 use PublicInbox::MsgIter;
 my @classes = qw(PublicInbox::Eml);

[PATCH 2/5] examples/unsubscribe-milter@.service: use KillMode=process

[Eric Wong]

This can be a multi-process daemon, but systemd should only kill
the top-level one.  And also finish a comment about the User
having access to the shared private key.
---
 examples/unsubscribe-milter@.service | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/examples/unsubscribe-milter@.service b/examples/unsubscribe-milter@.service
index eb5dcbe4..a68e6e81 100644
--- a/examples/unsubscribe-milter@.service
+++ b/examples/unsubscribe-milter@.service
@@ -24,7 +24,13 @@ Sockets = unsubscribe-milter.socket
 
 # the corresponding PSGI app needs permissions to modify the
 # mlmmj spool, so we might as well use the same user since
+# they both need to read /home/mlmmj/.unsubscribe.key
 User = mlmmj
 
+# only kill the parent process when using the default Sendmail::PMilter
+# postfork dispatcher, children will die naturally when they're done
+# with a given message.
+KillMode = process
+
 [Install]
 WantedBy = multi-user.target

[PATCH 3/5] config: don't vivify invalid fields for coderepos

[Eric Wong]

We don't need 404s for non-existent coderepos creating fake
(and invalid) entries.  I noticed this while working on
subsequent changes to support globbing in URLs.
---
 lib/PublicInbox/Config.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm
index 6bebf790..33c59ec7 100644
--- a/lib/PublicInbox/Config.pm
+++ b/lib/PublicInbox/Config.pm
@@ -699,8 +699,8 @@ sub get_coderepo {
 	$self->{-coderepos}->{$nick} // do {
 		defined($self->{-cgit_scan_path}) ? do {
 			apply_cgit_scan_path($self);
-			$self->{-coderepos}->{$nick} =
-					fill_coderepo($self, $nick);
+			my $cr = fill_coderepo($self, $nick);
+			$cr ? ($self->{-coderepos}->{$nick} = $cr) : undef;
 		} : undef;
 	};
 }

[PATCH 4/5] config: glob2re: fix over-matching /**/foo

[Eric Wong]

Noticed while adding wildcard support to WwwCoderepo...
---
 lib/PublicInbox/Config.pm | 2 +-
 t/config.t                | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm
index 33c59ec7..b8d3c485 100644
--- a/lib/PublicInbox/Config.pm
+++ b/lib/PublicInbox/Config.pm
@@ -662,7 +662,7 @@ sub squote_maybe ($) {
 }
 
 my %re_map = ( '*' => '[^/]*?', '?' => '[^/]',
-		'/**' => '/.*', '**/' => '.*/', '/**/' => '/.*?',
+		'/**' => '/.*', '**/' => '.*/', '/**/' => '(?:/.*?/|/)',
 		'[' => '[', ']' => ']', ',' => ',' );
 
 sub glob2re ($) {
diff --git a/t/config.t b/t/config.t
index 975cf836..c41a42d3 100644
--- a/t/config.t
+++ b/t/config.t
@@ -296,8 +296,9 @@ is_deeply($glob2re->('\\\\{a,b}'), '\\\\\\\\(a|b)', 'fake escape brace');
 is_deeply($glob2re->('**/foo'), '.*/foo', 'double asterisk start');
 is_deeply($glob2re->('foo/**'), 'foo/.*', 'double asterisk end');
 my $re = $glob2re->('a/**/b');
-is_deeply($re, 'a/.*?b', 'double asterisk middle');
+is_deeply($re, 'a(?:/.*?/|/)b', 'double asterisk middle');
 like($_, qr!$re!, "a/**/b matches $_") for ('a/b', 'a/c/b', 'a/c/a/b');
+unlike($_, qr!$re!, "a/**/b doesn't match $_") for ('a/ab');
 
 {
 	my $w = '';

[PATCH 5/5] www: repolist: support globbing in URL

[Eric Wong]

This can make it easier to find deeply-nested repositories on my
mirror of git.kernel.org.  It's not perfect, since projects like
Linux use several completely different basenames (e.g. linux.git
vs vfs.git vs net.git), but it can still help find significant
matches further up a tree.

I don't expect glob characters to conflict with actual git
repositories used by reasonable people, but direct (non-glob)
hits are still tried first.
---
 lib/PublicInbox/RepoList.pm    | 4 ++--
 lib/PublicInbox/WwwCoderepo.pm | 6 +++++-
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/lib/PublicInbox/RepoList.pm b/lib/PublicInbox/RepoList.pm
index 4b313ed6..39dc9c0b 100644
--- a/lib/PublicInbox/RepoList.pm
+++ b/lib/PublicInbox/RepoList.pm
@@ -16,9 +16,9 @@ sub html_top_fallback { # WwwStream->html_repo_top
 }
 
 sub html ($$$) {
-	my ($wcr, $ctx, $pfx) = @_;
+	my ($wcr, $ctx, $re) = @_;
 	my $cr = $wcr->{pi_cfg}->{-coderepos};
-	my @nicks = grep(m!\A\Q$pfx\E/!, keys %$cr) or return; # 404
+	my @nicks = grep(m!$re!, keys %$cr) or return; # 404
 	__PACKAGE__->html_init($ctx);
 	my $zfh = $ctx->zfh;
 	print $zfh "<pre>matching coderepos\n";
diff --git a/lib/PublicInbox/WwwCoderepo.pm b/lib/PublicInbox/WwwCoderepo.pm
index 4ab9a77c..61aa7862 100644
--- a/lib/PublicInbox/WwwCoderepo.pm
+++ b/lib/PublicInbox/WwwCoderepo.pm
@@ -356,8 +356,12 @@ sub srv { # endpoint called by PublicInbox::WWW
 	} elsif ($path_info =~ m!\A/(.+?)/(refs/(?:heads|tags))/\z! and
 			($ctx->{git} = $pi_cfg->get_coderepo($1))) {
 		refs_foo($self, $ctx, $2);
+	} elsif ($path_info =~ m!\A/(.*?\*.*?)/*\z!) {
+		my $re = PublicInbox::Config::glob2re($1);
+		PublicInbox::RepoList::html($self, $ctx, qr!$re\z!) // r(404);
 	} elsif ($path_info =~ m!\A/(.+?)/\z!) {
-		PublicInbox::RepoList::html($self, $ctx, $1) // r(404);
+		my $re = qr!\A\Q$1\E/!;
+		PublicInbox::RepoList::html($self, $ctx, $re) // r(404);
 	} elsif ($path_info =~ m!\A/(.+?)\z! and
 			($git = $pi_cfg->get_coderepo($1))) {
 		my $qs = $ctx->{env}->{QUERY_STRING};