user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [PATCH] t/spawn: fix with unusual env locations
@ 2019-09-15 13:48 14% Alyssa Ross
  2019-09-15 18:55  4% ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-09-15 13:48 UTC (permalink / raw)
  To: meta; +Cc: Alyssa Ross

The call to spawn clears the environment, including PATH.  This means
that an env in a non-default location wouldn't be found, while all the
other tests work because they use PATH.  We can fix this by looking up
which PATH to use beforehand.

I ran into this when packaging public-inbox for Nixpkgs.  We build in
a chroot, and in this case the env I wanted to use was at
/nix/store/7rmjki86923bw1inx0czpp4wgy0kk687-coreutils-8.31/bin/env.
---
 t/spawn.t | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/spawn.t b/t/spawn.t
index aba1a26..5549ca8 100644
--- a/t/spawn.t
+++ b/t/spawn.t
@@ -41,7 +41,8 @@ use PublicInbox::Spawn qw(which spawn popen_rd);
 {
 	my ($r, $w);
 	pipe $r, $w or die "pipe failed: $!";
-	my $pid = spawn(['env'], {}, { -env => 1, 1 => fileno($w) });
+	my $env = which('env');
+	my $pid = spawn([$env], {}, { -env => 1, 1 => fileno($w) });
 	close $w or die "close pipe[1] failed: $!";
 	ok(!defined(<$r>), 'read stdout of spawned from pipe');
 	is(waitpid($pid, 0), $pid, 'waitpid succeeds on spawned process');
-- 
2.22.1


^ permalink raw reply related	[relevance 14%]

* Re: [PATCH] t/spawn: fix with unusual env locations
  2019-09-15 13:48 14% [PATCH] t/spawn: fix with unusual env locations Alyssa Ross
@ 2019-09-15 18:55  4% ` Eric Wong
  2019-09-24  1:11  6%   ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-09-15 18:55 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> The call to spawn clears the environment, including PATH.  This means
> that an env in a non-default location wouldn't be found, while all the
> other tests work because they use PATH.  We can fix this by looking up
> which PATH to use beforehand.

Thanks for reporting this issue, but... is that it?
We also use spawn() for 'git', among other things...

> I ran into this when packaging public-inbox for Nixpkgs.  We build in
> a chroot, and in this case the env I wanted to use was at
> /nix/store/7rmjki86923bw1inx0czpp4wgy0kk687-coreutils-8.31/bin/env.

The Inline::C version of spawn (enabled by setting
PERL_INLINE_DIRECTORY) should already be using the result of
which() for every execve(2) call it does.

For the pure Perl codepath, this ought to be a more encompassing
alternative to your patch:

diff --git a/lib/PublicInbox/SpawnPP.pm b/lib/PublicInbox/SpawnPP.pm
index 25c8c87f..29b13371 100644
--- a/lib/PublicInbox/SpawnPP.pm
+++ b/lib/PublicInbox/SpawnPP.pm
@@ -38,11 +38,13 @@ sub pi_fork_exec ($$$$$$) {
 		}
 
 		if ($ENV{MOD_PERL}) {
-			exec qw(env -i), @$env, @$cmd;
+			exec which('env'), '-i', @$env, @$cmd;
 			die "exec env -i ... $cmd->[0] failed: $!\n";
 		} else {
 			local %ENV = map { split(/=/, $_, 2) } @$env;
-			exec @$cmd;
+			my @cmd = @$cmd;
+			$cmd[0] = $f;
+			exec @cmd;
 			die "exec $cmd->[0] failed: $!\n";
 		}
 	}

Can you confirm?  Thanks.

^ permalink raw reply related	[relevance 4%]

* Re: [PATCH] t/spawn: fix with unusual env locations
  2019-09-15 18:55  4% ` Eric Wong
@ 2019-09-24  1:11  6%   ` Alyssa Ross
  2019-09-24  4:01  4%     ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-09-24  1:11 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 1065 bytes --]

>> The call to spawn clears the environment, including PATH.  This means
>> that an env in a non-default location wouldn't be found, while all the
>> other tests work because they use PATH.  We can fix this by looking up
>> which PATH to use beforehand.
>
> Thanks for reporting this issue, but... is that it?
> We also use spawn() for 'git', among other things...

Yes, I can confirm that this was the only failing test case in my build.
(It may be of note however that I did not have Xapian enabled.  Trying
to build with that caused a host of other failures that were beyond my
ability to troubleshoot, so I decided to just give up on running the
tests for now, since they seem very dependent on the environment and the
environment inside our build sandboxes is so different from what
public-inbox will have on a running system anyway.)

> For the pure Perl codepath, this ought to be a more encompassing
> alternative to your patch:
>
> [ ... snip ... ]
>
> Can you confirm?  Thanks.

I can confirm that this patch also solves my problem.  Thank you very
much.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: [PATCH] t/spawn: fix with unusual env locations
  2019-09-24  1:11  6%   ` Alyssa Ross
@ 2019-09-24  4:01  4%     ` Eric Wong
  2019-09-24 22:12  5%       ` Test failures in build sandbox Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-09-24  4:01 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> >> The call to spawn clears the environment, including PATH.  This means
> >> that an env in a non-default location wouldn't be found, while all the
> >> other tests work because they use PATH.  We can fix this by looking up
> >> which PATH to use beforehand.
> >
> > Thanks for reporting this issue, but... is that it?
> > We also use spawn() for 'git', among other things...
> 
> Yes, I can confirm that this was the only failing test case in my build.

Thanks for confirming, pushed my patch as
4b3e278d9c4e5d1f55c1781aa0a7c9f0453f3852 to master.

In my original reply, I missed the test was using the "-env"
option to clear %ENV.  It's probably safe to drop the "-env"
option for spawn() to clear %ENV since we don't use it in real
code...

> (It may be of note however that I did not have Xapian enabled.  Trying
> to build with that caused a host of other failures that were beyond my
> ability to troubleshoot, so I decided to just give up on running the
> tests for now, since they seem very dependent on the environment and the
> environment inside our build sandboxes is so different from what
> public-inbox will have on a running system anyway.)

I don't know much about Nix, but I hope/expect public-inbox
tests to work on every reasonably modern Unix-like OS which
Perl5, git, and SQLite run on.

If you post logs of the failures you encountered, it would
improve the chances that somebody here (maybe not me) can
help fix them.  Thanks.

^ permalink raw reply	[relevance 4%]

* [PATCH] hlmod: update for highlight 3.51 API change
@ 2019-09-24 20:05 14% Alyssa Ross
  2019-09-26  1:59  4% ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-09-24 20:05 UTC (permalink / raw)
  To: meta; +Cc: Alyssa Ross, Amitai Schleier

Quoting Amitai Schleier, who made this same change in ikiwiki[1],
where lots of the public-inbox highlight code comes from:

> As of 3.51, searchFile() is no longer provided in highlight's Perl
> bindings (at least on NetBSD and OS X, as built from pkgsrc). This
> leaves us falling through to getConfDir(), which has been gone
> rather longer.
>
> From highlight git, it appears searchFile() and getFiletypesConfPath()
> both originated in the 3.14 release. The latter is still available in
> 3.51, and returns the same result searchFile() used to. Switch to it.

So, this should still be compatible with the version of highlight.pm in
Debian, but add support for newer versions as well.

[1]: commit 4d06df9583e6c4145f8c6fc2fd51d7894c0b85ce

Cc: Amitai Schleier <schmonz-web-ikiwiki@schmonz.com>
---
 lib/PublicInbox/HlMod.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/PublicInbox/HlMod.pm b/lib/PublicInbox/HlMod.pm
index 36e3110..1d7a460 100644
--- a/lib/PublicInbox/HlMod.pm
+++ b/lib/PublicInbox/HlMod.pm
@@ -20,7 +20,7 @@ use PublicInbox::Hval qw(src_escape ascii_html);
 my $hl;

 sub _parse_filetypes ($) {
-	my $ft_conf = $_[0]->searchFile('filetypes.conf') or
+	my $ft_conf = $_[0]->getFiletypesConfPath('filetypes') or
 				die 'filetypes.conf not found by highlight';
 	open my $fh, '<', $ft_conf or die "failed to open($ft_conf): $!";
 	local $/;
--
2.22.1


^ permalink raw reply related	[relevance 14%]

* [PATCH] #!/usr/bin/perl -> #!/usr/bin/env perl
@ 2019-09-24 20:43  6% Alyssa Ross
  2019-09-26  2:49  3% ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-09-24 20:43 UTC (permalink / raw)
  To: meta; +Cc: Alyssa Ross

It's much more reasonable to assume that the Perl a user wants to use
is the one they have in their path, rather than whatever one has been
installed system-wide by their distribution (if any!).
---
 Documentation/extman.perl     | 3 ++-
 Documentation/standards.perl  | 3 ++-
 Makefile.PL                   | 3 ++-
 certs/create-certs.perl       | 2 +-
 ci/deps.perl                  | 3 ++-
 examples/cgit.psgi            | 2 +-
 examples/highlight.psgi       | 2 +-
 examples/newswww.psgi         | 2 +-
 examples/public-inbox.psgi    | 2 +-
 examples/unsubscribe.milter   | 2 +-
 examples/unsubscribe.psgi     | 3 ++-
 script/public-inbox-compact   | 2 +-
 script/public-inbox-convert   | 2 +-
 script/public-inbox-edit      | 2 +-
 script/public-inbox-httpd     | 2 +-
 script/public-inbox-index     | 2 +-
 script/public-inbox-init      | 2 +-
 script/public-inbox-learn     | 2 +-
 script/public-inbox-mda       | 2 +-
 script/public-inbox-nntpd     | 2 +-
 script/public-inbox-purge     | 2 +-
 script/public-inbox-watch     | 2 +-
 script/public-inbox-xcpdb     | 3 ++-
 script/public-inbox.cgi       | 2 +-
 scripts/dupe-finder           | 2 +-
 scripts/import_maildir        | 2 +-
 scripts/import_slrnspool      | 2 +-
 scripts/import_vger_from_mbox | 2 +-
 scripts/slrnspool2maildir     | 2 +-
 scripts/ssoma-replay          | 3 ++-
 scripts/xhdr-num2mid          | 2 +-
 t/check-www-inbox.perl        | 2 +-
 t/git-http-backend.psgi       | 2 +-
 t/hl_mod.t                    | 2 +-
 34 files changed, 41 insertions(+), 34 deletions(-)

diff --git a/Documentation/extman.perl b/Documentation/extman.perl
index a72caca..cba689f 100755
--- a/Documentation/extman.perl
+++ b/Documentation/extman.perl
@@ -1,8 +1,9 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 # prints a manpage to stdout
 use strict;
+use warnings;
 my $xapmsg = 'See https://xapian.org/ for more information on Xapian';
 my $usage = "$0 /path/to/manpage.SECTION.txt";
 my $manpage = shift or die $usage;
diff --git a/Documentation/standards.perl b/Documentation/standards.perl
index f75c412..92edbc9 100755
--- a/Documentation/standards.perl
+++ b/Documentation/standards.perl
@@ -1,5 +1,6 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 use strict;
+use warnings;
 # Copyright 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 
diff --git a/Makefile.PL b/Makefile.PL
index 8daeb4f..eb6f23c 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -1,7 +1,8 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2013-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
+use warnings;
 use ExtUtils::MakeMaker;
 open my $m, '<', 'MANIFEST' or die "open(MANIFEST): $!\n";
 chomp(my @manifest = (<$m>));
diff --git a/certs/create-certs.perl b/certs/create-certs.perl
index b287cec..607cac5 100755
--- a/certs/create-certs.perl
+++ b/certs/create-certs.perl
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # License: GPL-1.0+ or Artistic-1.0-Perl
 # from IO::Socket::SSL 2.063 / https://github.com/noxxi/p5-io-socket-ssl
 use strict;
diff --git a/ci/deps.perl b/ci/deps.perl
index 62870c1..d797a31 100755
--- a/ci/deps.perl
+++ b/ci/deps.perl
@@ -1,10 +1,11 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 # Helper script for installing/uninstalling packages for CI use
 # Intended for use on non-production chroots or VMs since it
 # changes installed packages
 use strict;
+use warnings;
 my $usage = "$0 PKG_FMT PROFILE [PROFILE_MOD]";
 my $pkg_fmt = shift;
 @ARGV or die $usage, "\n";
diff --git a/examples/cgit.psgi b/examples/cgit.psgi
index ca93f92..8c8a904 100644
--- a/examples/cgit.psgi
+++ b/examples/cgit.psgi
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
 #
diff --git a/examples/highlight.psgi b/examples/highlight.psgi
index 244b128..b07181f 100644
--- a/examples/highlight.psgi
+++ b/examples/highlight.psgi
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/examples/newswww.psgi b/examples/newswww.psgi
index 0f66782..64c1ad6 100644
--- a/examples/newswww.psgi
+++ b/examples/newswww.psgi
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
 #
diff --git a/examples/public-inbox.psgi b/examples/public-inbox.psgi
index 53a5970..92f525e 100644
--- a/examples/public-inbox.psgi
+++ b/examples/public-inbox.psgi
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2014-2019 all contributors <meta@public-inbox.org>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
 # Note: this is part of our test suite, update t/plack.t if this changes
diff --git a/examples/unsubscribe.milter b/examples/unsubscribe.milter
index d6db766..a0f87e0 100644
--- a/examples/unsubscribe.milter
+++ b/examples/unsubscribe.milter
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
diff --git a/examples/unsubscribe.psgi b/examples/unsubscribe.psgi
index 7020af2..a8968ce 100644
--- a/examples/unsubscribe.psgi
+++ b/examples/unsubscribe.psgi
@@ -1,10 +1,11 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
 # This should not require any other PublicInbox code, but may use
 # PublicInbox::Config if ~/.public-inbox/config exists or
 # PI_CONFIG is pointed to an appropriate location
 use strict;
+use warnings;
 use Plack::Builder;
 use PublicInbox::Unsubscribe;
 my $app = PublicInbox::Unsubscribe->new(
diff --git a/script/public-inbox-compact b/script/public-inbox-compact
index e8bf31e..ded81aa 100755
--- a/script/public-inbox-compact
+++ b/script/public-inbox-compact
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2018-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
diff --git a/script/public-inbox-convert b/script/public-inbox-convert
index 9d2d251..09e0be0 100755
--- a/script/public-inbox-convert
+++ b/script/public-inbox-convert
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2018-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <http://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
diff --git a/script/public-inbox-edit b/script/public-inbox-edit
index 6884fd0..e9c206e 100755
--- a/script/public-inbox-edit
+++ b/script/public-inbox-edit
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/script/public-inbox-httpd b/script/public-inbox-httpd
index b2464f4..da786de 100755
--- a/script/public-inbox-httpd
+++ b/script/public-inbox-httpd
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/script/public-inbox-index b/script/public-inbox-index
index 439da15..4da9dca 100755
--- a/script/public-inbox-index
+++ b/script/public-inbox-index
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2015-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 # Basic tool to create a Xapian search index for a git repository
diff --git a/script/public-inbox-init b/script/public-inbox-init
index 98aea4b..5ec74e9 100755
--- a/script/public-inbox-init
+++ b/script/public-inbox-init
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2014-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/script/public-inbox-learn b/script/public-inbox-learn
index c4c4d4b..6362a83 100755
--- a/script/public-inbox-learn
+++ b/script/public-inbox-learn
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2014-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/script/public-inbox-mda b/script/public-inbox-mda
index 4e6e04e..81f7d0c 100755
--- a/script/public-inbox-mda
+++ b/script/public-inbox-mda
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2013-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/script/public-inbox-nntpd b/script/public-inbox-nntpd
index 3c4be19..2476780 100755
--- a/script/public-inbox-nntpd
+++ b/script/public-inbox-nntpd
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2015-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/script/public-inbox-purge b/script/public-inbox-purge
index 0705d17..e5cd602 100755
--- a/script/public-inbox-purge
+++ b/script/public-inbox-purge
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/script/public-inbox-watch b/script/public-inbox-watch
index 5cbf761..fc220a1 100755
--- a/script/public-inbox-watch
+++ b/script/public-inbox-watch
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
diff --git a/script/public-inbox-xcpdb b/script/public-inbox-xcpdb
index b41c683..a043b7c 100755
--- a/script/public-inbox-xcpdb
+++ b/script/public-inbox-xcpdb
@@ -1,7 +1,8 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 # xcpdb: Xapian copy database, a wrapper around Xapian's copydatabase(1)
+use warnings;
 use Getopt::Long qw(:config gnu_getopt no_ignore_case auto_abbrev);
 use PublicInbox::InboxWritable;
 use PublicInbox::Xapcmd;
diff --git a/script/public-inbox.cgi b/script/public-inbox.cgi
index c0e8e6c..0bf9a1a 100755
--- a/script/public-inbox.cgi
+++ b/script/public-inbox.cgi
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2014-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ or later <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/scripts/dupe-finder b/scripts/dupe-finder
index 1e98093..afb3610 100644
--- a/scripts/dupe-finder
+++ b/scripts/dupe-finder
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2018-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/scripts/import_maildir b/scripts/import_maildir
index fbf3f64..bd6b1de 100755
--- a/scripts/import_maildir
+++ b/scripts/import_maildir
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2014, Eric Wong <e@80x24.org> and all contributors
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/scripts/import_slrnspool b/scripts/import_slrnspool
index 1dccb8d..7e85d26 100755
--- a/scripts/import_slrnspool
+++ b/scripts/import_slrnspool
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2015-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/scripts/import_vger_from_mbox b/scripts/import_vger_from_mbox
index b2dd5b3..413936a 100644
--- a/scripts/import_vger_from_mbox
+++ b/scripts/import_vger_from_mbox
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
diff --git a/scripts/slrnspool2maildir b/scripts/slrnspool2maildir
index b321db1..f5599c1 100755
--- a/scripts/slrnspool2maildir
+++ b/scripts/slrnspool2maildir
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2013-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
diff --git a/scripts/ssoma-replay b/scripts/ssoma-replay
index f20e027..c59215c 100755
--- a/scripts/ssoma-replay
+++ b/scripts/ssoma-replay
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2015-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
@@ -26,6 +26,7 @@ ssoma add $NAME $URL "command:/path/to/ssoma-replay -L /path/to/spool/$NAME"
 	command = /path/to/ssoma-replay -L /path/to/spool/meta
 =cut
 use strict;
+use warnings;
 use Email::Simple;
 use URI::Escape qw/uri_escape_utf8/;
 use File::Temp qw/tempfile/;
diff --git a/scripts/xhdr-num2mid b/scripts/xhdr-num2mid
index 717fa46..79a94a4 100755
--- a/scripts/xhdr-num2mid
+++ b/scripts/xhdr-num2mid
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 # Useful for mapping article IDs from existing NNTP servers to MIDs
diff --git a/t/check-www-inbox.perl b/t/check-www-inbox.perl
index db292c5..77569d9 100644
--- a/t/check-www-inbox.perl
+++ b/t/check-www-inbox.perl
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 # Parallel WWW checker
diff --git a/t/git-http-backend.psgi b/t/git-http-backend.psgi
index ed70d50..dc4967d 100644
--- a/t/git-http-backend.psgi
+++ b/t/git-http-backend.psgi
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2016-2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
diff --git a/t/hl_mod.t b/t/hl_mod.t
index fc7b712..827b232 100644
--- a/t/hl_mod.t
+++ b/t/hl_mod.t
@@ -1,4 +1,4 @@
-#!/usr/bin/perl -w
+#!/usr/bin/env perl
 # Copyright (C) 2019 all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict;
-- 
2.22.1


^ permalink raw reply related	[relevance 6%]

* Re: Test failures in build sandbox
  2019-09-24  4:01  4%     ` Eric Wong
@ 2019-09-24 22:12  5%       ` Alyssa Ross
  2019-09-26  8:44  4%         ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-09-24 22:12 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 4516 bytes --]

Eric Wong <e@80x24.org> writes:

>> (It may be of note however that I did not have Xapian enabled.  Trying
>> to build with that caused a host of other failures that were beyond my
>> ability to troubleshoot, so I decided to just give up on running the
>> tests for now, since they seem very dependent on the environment and the
>> environment inside our build sandboxes is so different from what
>> public-inbox will have on a running system anyway.)
>
> I don't know much about Nix, but I hope/expect public-inbox
> tests to work on every reasonably modern Unix-like OS which
> Perl5, git, and SQLite run on.
>
> If you post logs of the failures you encountered, it would
> improve the chances that somebody here (maybe not me) can
> help fix them.  Thanks.

As requested, following a test log, edited down to just failures /
successes with error output:

PERL_DL_NONLAZY=1 "/nix/store/5r85g4zzk0djffzs47zaimvkxfmbpp3q-perl-5.30.0/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/convert-compact.t ........ 1/? error: unable to create temporary file: Operation not permitted
fatal: failed to write object
t/search.t ................. fatal: Could not make /build/pi-search-i646IK/a.git/branches/ writable by group
t/search.t ................. 1/?
#   Failed test 'git init (main)'
#   at t/search.t line 20.
#          got: '0'
#     expected: '32768'

#   Failed test 'undefined permission is group'
#   at t/search.t line 40.
#          got: '0'
#     expected: '432'
t/search.t ................. 92/?
#   Failed test 'sharedRepository respected for msgmap.sqlite3'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for public-inbox'
#   at t/search.t line 455.
#          got: '493'
#     expected: '1528'

#   Failed test 'sharedRepository respected for xapian15'
#   at t/search.t line 455.
#          got: '493'
#     expected: '1528'

#   Failed test 'sharedRepository respected for docdata.glass'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for flintlock'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for iamglass'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for over.sqlite3'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for over.sqlite3-journal'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for position.glass'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for postlist.glass'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for spelling.glass'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for synonym.glass'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'

#   Failed test 'sharedRepository respected for termlist.glass'
#   at t/search.t line 455.
#          got: '420'
#     expected: '432'
# Looks like you failed 15 tests of 114.
t/search.t ................. Dubious, test returned 15 (wstat 3840, 0xf00)
Failed 15/114 subtests
t/v2writable.t ............. 3/? error: unable to create temporary file: Operation not permitted
fatal: failed to write object
t/v2writable.t ............. 48/? error: unable to create temporary file: Operation not permitted
fatal: failed to write object
t/v2writable.t ............. 97/? error: unable to create temporary file: Operation not permitted
fatal: failed to write object
error: unable to create temporary file: Operation not permitted
fatal: failed to write object
t/v2writable.t ............. 101/? error: unable to create temporary file: Operation not permitted
fatal: failed to write object
t/v2writable.t ............. ok

Test Summary Report
-------------------
t/search.t               (Wstat: 3840 Tests: 114 Failed: 15)
  Failed tests:  1, 3, 93-105
  Non-zero exit status: 15
Files=84, Tests=2967, 574 wallclock secs ( 1.03 usr  0.21 sys + 85.43 cusr 102.57 csys = 189.24 CPU)
Result: FAIL
Failed 1/84 test programs. 15/2967 subtests failed.
make: *** [Makefile:1050: test_dynamic] Error 255

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] hlmod: update for highlight 3.51 API change
  2019-09-24 20:05 14% [PATCH] hlmod: update for highlight 3.51 API change Alyssa Ross
@ 2019-09-26  1:59  4% ` Eric Wong
    2019-09-26 15:08  6%   ` Alyssa Ross
  0 siblings, 2 replies; 49+ results
From: Eric Wong @ 2019-09-26  1:59 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta, Amitai Schleier

Alyssa Ross <hi@alyssa.is> wrote:
> Quoting Amitai Schleier, who made this same change in ikiwiki[1],
> where lots of the public-inbox highlight code comes from:
> 
> > As of 3.51, searchFile() is no longer provided in highlight's Perl
> > bindings (at least on NetBSD and OS X, as built from pkgsrc). This
> > leaves us falling through to getConfDir(), which has been gone
> > rather longer.
> >
> > From highlight git, it appears searchFile() and getFiletypesConfPath()
> > both originated in the 3.14 release. The latter is still available in
> > 3.51, and returns the same result searchFile() used to. Switch to it.
> 
> So, this should still be compatible with the version of highlight.pm in
> Debian, but add support for newer versions as well.
> 
> [1]: commit 4d06df9583e6c4145f8c6fc2fd51d7894c0b85ce

Thanks both, pushed as dadaf7fd96301f1737b2eca2171a742daef8befd

I had my reservations about relying on highlight.pm; and this
confirms them, unfortunately :<  Oh well...

I wonder if pygments or any other code highlighting packages are
more stable.  We'd need to use a pipe or socket to interact with
pygments or anything w/o Perl bindings.

^ permalink raw reply	[relevance 4%]

* Re: [PATCH] #!/usr/bin/perl -> #!/usr/bin/env perl
  2019-09-24 20:43  6% [PATCH] #!/usr/bin/perl -> #!/usr/bin/env perl Alyssa Ross
@ 2019-09-26  2:49  3% ` Eric Wong
  2019-10-01 17:47  6%   ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-09-26  2:49 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> It's much more reasonable to assume that the Perl a user wants to use
> is the one they have in their path, rather than whatever one has been
> installed system-wide by their distribution (if any!).

I used to think that, too...
Nowadays, I would only use "/usr/bin/env perl" for standalone
or standard-library-only scripts; but nothing with
non-standard-library dependencies.

Some general notes:

* Perl will parse the "-w" in a shebang even if it's invoked as
  "perl /path/to/script" without the "-w"

* "-w" is stronger than "use warnings", in that it applies to
  the whole program rather than the scope of "use warnings"

* Different Perl installations can have different modules
  installed for it, so relying on /usr/bin/env can lead
  to surprises + failures when libraries are missing.

* "/usr/bin/env perl" won't take into account "perl5.xx" names
  that dev versions of Perl get installed as by default, so shebang
  rewriting is necessary for anything end users would use, anyways.

* I've been looking into startup time of some scripts, and
  "use warnings" adds more open() syscalls which affects
  performance a miniscule amount (maybe more for Intel CPUs with
  vulnerabilities).  So I might get rid of some "use warnings"
  in those paths rely on "-w" down the line...

Reordering the diffstat slightly...

>  Makefile.PL                   | 3 ++-

Makefile.PL doesn't have the executable bit set, and all the
documentation says to run it as "perl Makefile.PL"

The reason for the shebang is for external tools,
highlighters and stuff like file(1).

>  Documentation/extman.perl     | 3 ++-
>  Documentation/standards.perl  | 3 ++-

Makefile.PL writes a Makefile which sets $(PERL), with an
absolute path to run those files.

>  certs/create-certs.perl       | 2 +-

Documentation in corresponding t/*.t files could be better
about running that and instead recommend "$^X"

>  ci/deps.perl                  | 3 ++-

That's only run via $PERL (which defaults to 'perl')

>  examples/cgit.psgi            | 2 +-
>  examples/highlight.psgi       | 2 +-
>  examples/newswww.psgi         | 2 +-
>  examples/public-inbox.psgi    | 2 +-
>  examples/unsubscribe.psgi     | 3 ++-

>  t/git-http-backend.psgi       | 2 +-

I only had the shebang there for external tools.
Running a .psgi directly via Perl wouldn't work.

"#!/usr/bin/plackup" might work if it was executable,
but I'm not sure if people do that w/o command-line
options.

>  examples/unsubscribe.milter   | 2 +-

That could probably be installed via Makefile using
$(FIXIN) like the rest of the stuff in script/ below
(but not by default).

>  script/public-inbox-compact   | 2 +-
>  script/public-inbox-convert   | 2 +-
>  script/public-inbox-edit      | 2 +-
>  script/public-inbox-httpd     | 2 +-
>  script/public-inbox-index     | 2 +-
>  script/public-inbox-init      | 2 +-
>  script/public-inbox-learn     | 2 +-
>  script/public-inbox-mda       | 2 +-
>  script/public-inbox-nntpd     | 2 +-
>  script/public-inbox-purge     | 2 +-
>  script/public-inbox-watch     | 2 +-
>  script/public-inbox-xcpdb     | 3 ++-
>  script/public-inbox.cgi       | 2 +-

All of these are handled by $(FIXIN) in the Makefile.
I install these on FreeBSD which puts Perl in /usr/local/bin
and never had a problem.

>  scripts/dupe-finder           | 2 +-
>  scripts/import_maildir        | 2 +-
>  scripts/import_slrnspool      | 2 +-
>  scripts/import_vger_from_mbox | 2 +-
>  scripts/slrnspool2maildir     | 2 +-
>  scripts/ssoma-replay          | 3 ++-
>  scripts/xhdr-num2mid          | 2 +-
>  t/check-www-inbox.perl        | 2 +-

None of these are really serious scripts, and if they're run;
it's always via "perl /path/to/script"

>  t/hl_mod.t                    | 2 +-

Oops, there probably shouldn't have a shebang there at all
since none of the other *.t files have it.  I've also seen
"#!perl -w" (no '/' at all) in other projects, but I don't think
it's worth the effort unless some external tools/highlighters
work better with it. I expect "#!/usr/bin/perl -w" to be most
compatible with highlighters and stuff like file(1).

^ permalink raw reply	[relevance 3%]

* Re: Test failures in build sandbox
  2019-09-24 22:12  5%       ` Test failures in build sandbox Alyssa Ross
@ 2019-09-26  8:44  4%         ` Eric Wong
  2019-10-02 23:23  6%           ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-09-26  8:44 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> Eric Wong <e@80x24.org> writes:
> 
> >> (It may be of note however that I did not have Xapian enabled.  Trying
> >> to build with that caused a host of other failures that were beyond my
> >> ability to troubleshoot, so I decided to just give up on running the
> >> tests for now, since they seem very dependent on the environment and the
> >> environment inside our build sandboxes is so different from what
> >> public-inbox will have on a running system anyway.)
> >
> > I don't know much about Nix, but I hope/expect public-inbox
> > tests to work on every reasonably modern Unix-like OS which
> > Perl5, git, and SQLite run on.
> >
> > If you post logs of the failures you encountered, it would
> > improve the chances that somebody here (maybe not me) can
> > help fix them.  Thanks.
> 
> As requested, following a test log, edited down to just failures /
> successes with error output:

Thanks.

> PERL_DL_NONLAZY=1 "/nix/store/5r85g4zzk0djffzs47zaimvkxfmbpp3q-perl-5.30.0/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
> t/convert-compact.t ........ 1/? error: unable to create temporary file: Operation not permitted
> fatal: failed to write object

What kind of filesystem and mount options was the code and
(if set, TMPDIR) on?

Since Nix seems to use odd paths, what's the default temporary
directory? ("/tmp" on my systems)

AFAIK, it should be possible to run tests with the
source/worktree on a read-only FS (only TMPDIR needs to be
writable); but it's not something that's fun/quick to test.

> t/search.t ................. fatal: Could not make /build/pi-search-i646IK/a.git/branches/ writable by group
> t/search.t ................. 1/?
> #   Failed test 'git init (main)'
> #   at t/search.t line 20.
> #          got: '0'
> #     expected: '32768'
> 
> #   Failed test 'undefined permission is group'
> #   at t/search.t line 40.
> #          got: '0'
> #     expected: '432'
> t/search.t ................. 92/?
> #   Failed test 'sharedRepository respected for msgmap.sqlite3'
> #   at t/search.t line 455.
> #          got: '420'
> #     expected: '432'
> 
> #   Failed test 'sharedRepository respected for public-inbox'
> #   at t/search.t line 455.
> #          got: '493'
> #     expected: '1528

<snip>

Sidenote, it would be easier to read if Test::More output those
as octal numbers.  I use iprint ("i") from Debian oldstable
installs, but it's no longer in Debian buster :<

printf '%0o\n' 493
printf '%0o\n' 1528
...
works, too, but it's more typing

^ permalink raw reply	[relevance 4%]

* Re: [PATCH] hlmod: update for highlight 3.51 API change
  @ 2019-09-26 15:07  6%     ` Alyssa Ross
  0 siblings, 0 replies; 49+ results
From: Alyssa Ross @ 2019-09-26 15:07 UTC (permalink / raw)
  To: Konstantin Ryabitsev, Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 571 bytes --]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

> On Thu, Sep 26, 2019 at 01:59:53AM +0000, Eric Wong wrote:
>>I had my reservations about relying on highlight.pm; and this
>>confirms them, unfortunately :<  Oh well...
>
> I was also unable to find highlight.pm anywhere outside of Debian 
> packages.

I was, at least, able to package it for Nixpkgs relatively easily.  I
suspect other distributions might just include it with the highlight
package -- that's probably what I'll be doing.  I note that there's also
a package for it in pkgsrc.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: [PATCH] hlmod: update for highlight 3.51 API change
  2019-09-26  1:59  4% ` Eric Wong
  @ 2019-09-26 15:08  6%   ` Alyssa Ross
  1 sibling, 0 replies; 49+ results
From: Alyssa Ross @ 2019-09-26 15:08 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 584 bytes --]

Eric Wong <e@80x24.org> writes:

> I wonder if pygments or any other code highlighting packages are
> more stable.  We'd need to use a pipe or socket to interact with
> pygments or anything w/o Perl bindings.

In that case, it would also be possible to interact with highlight
without using its perl bindings -- the CLI appears to be extremely
widely packaged.

But Pygments is the one I've seen used all over the place, and I so bet
(though haven't checked) that it has better language support, etc.  So
there's probably no reason to prefer highlight if not using the Perl
bindings.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: [PATCH] #!/usr/bin/perl -> #!/usr/bin/env perl
  2019-09-26  2:49  3% ` Eric Wong
@ 2019-10-01 17:47  6%   ` Alyssa Ross
  2019-10-01 20:58  4%     ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-01 17:47 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]

>>  certs/create-certs.perl       | 2 +-
>
> Documentation in corresponding t/*.t files could be better
> about running that and instead recommend "$^X"

Makes sense.  This was the one that was giving me problems in the first
place.

>>  t/hl_mod.t                    | 2 +-
>
> Oops, there probably shouldn't have a shebang there at all
> since none of the other *.t files have it.  I've also seen
> "#!perl -w" (no '/' at all) in other projects, but I don't think
> it's worth the effort unless some external tools/highlighters
> work better with it. I expect "#!/usr/bin/perl -w" to be most
> compatible with highlighters and stuff like file(1).

#!perl has some nice characteristics -- it's understood by the
interpreter for setting flags, and for -x, but at the same time it's
clear that a file with #!perl is not supposed to be directly executed.

Assuming you don't want to change to #!perl, do you want a patch to
recommend running certs/create-certs.perl and remove the shebang from
t/hl_mod.t?  And should the executable bits still be set on the other
files?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: [PATCH] #!/usr/bin/perl -> #!/usr/bin/env perl
  2019-10-01 17:47  6%   ` Alyssa Ross
@ 2019-10-01 20:58  4%     ` Eric Wong
  2019-10-02 21:36 13%       ` [PATCH] tests: recommend running create-certs.pl with $^X Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-01 20:58 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> >>  certs/create-certs.perl       | 2 +-
> >
> > Documentation in corresponding t/*.t files could be better
> > about running that and instead recommend "$^X"
> 
> Makes sense.  This was the one that was giving me problems in the first
> place.
> 
> >>  t/hl_mod.t                    | 2 +-
> >
> > Oops, there probably shouldn't have a shebang there at all
> > since none of the other *.t files have it.  I've also seen
> > "#!perl -w" (no '/' at all) in other projects, but I don't think
> > it's worth the effort unless some external tools/highlighters
> > work better with it. I expect "#!/usr/bin/perl -w" to be most
> > compatible with highlighters and stuff like file(1).
> 
> #!perl has some nice characteristics -- it's understood by the
> interpreter for setting flags, and for -x, but at the same time it's
> clear that a file with #!perl is not supposed to be directly executed.

Agreed.  However, I'm not sure if compatibility with
highlighters would be as good as a full path.

> Assuming you don't want to change to #!perl, do you want a patch to
> recommend running certs/create-certs.perl and remove the shebang from
> t/hl_mod.t?  And should the executable bits still be set on the other
> files?

Just the messages in t/*.t for certs/create-certs.perl is fine.
Thanks.

I don't think the other changes would affect users/testers.
Also, t/hl_mod.t as-is could be a good comparison for syntax
highlighters in the future (and I am leaning towards dropping
"use warnings").

Not sure if there's executable bits worth changing, now; I think
it's a good way to inform users that a given file can be
executed (rather than loaded via `require'); but the '.t' files
might be a special case...

^ permalink raw reply	[relevance 4%]

* [PATCH] tests: recommend running create-certs.pl with $^X
  2019-10-01 20:58  4%     ` Eric Wong
@ 2019-10-02 21:36 13%       ` Alyssa Ross
  2019-10-02 23:19  4%         ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-02 21:36 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta, Alyssa Ross

This is better than recommending running the script directly because
it will ensure the correct version of perl is used.
---
 t/httpd-https.t    | 2 +-
 t/nntpd-tls.t      | 2 +-
 t/nntpd-validate.t | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/httpd-https.t b/t/httpd-https.t
index 410ae65..22c62bf 100644
--- a/t/httpd-https.t
+++ b/t/httpd-https.t
@@ -14,7 +14,7 @@ my $cert = 'certs/server-cert.pem';
 my $key = 'certs/server-key.pem';
 unless (-r $key && -r $cert) {
 	plan skip_all =>
-		"certs/ missing for $0, run ./create-certs.perl in certs/";
+		"certs/ missing for $0, run $^X ./create-certs.perl in certs/";
 }
 use_ok 'PublicInbox::TLS';
 use_ok 'IO::Socket::SSL';
diff --git a/t/nntpd-tls.t b/t/nntpd-tls.t
index e961965..1a74924 100644
--- a/t/nntpd-tls.t
+++ b/t/nntpd-tls.t
@@ -20,7 +20,7 @@ my $cert = 'certs/server-cert.pem';
 my $key = 'certs/server-key.pem';
 unless (-r $key && -r $cert) {
 	plan skip_all =>
-		"certs/ missing for $0, run ./create-certs.perl in certs/";
+		"certs/ missing for $0, run $^X ./create-certs.perl in certs/";
 }
 
 use_ok 'PublicInbox::TLS';
diff --git a/t/nntpd-validate.t b/t/nntpd-validate.t
index 532ef72..87829b0 100644
--- a/t/nntpd-validate.t
+++ b/t/nntpd-validate.t
@@ -32,7 +32,7 @@ my $test_tls = $ENV{TEST_SKIP_TLS} ? 0 : eval { require IO::Socket::SSL };
 my $cert = 'certs/server-cert.pem';
 my $key = 'certs/server-key.pem';
 if ($test_tls && !-r $key || !-r $cert) {
-	plan skip_all => "certs/ missing for $0, run ./certs/create-certs.perl";
+	plan skip_all => "certs/ missing for $0, run $^X ./certs/create-certs.perl";
 }
 require './t/common.perl';
 my $keep_tmp = !!$ENV{TEST_KEEP_TMP};
-- 
2.23.0


^ permalink raw reply related	[relevance 13%]

* Re: [PATCH] tests: recommend running create-certs.pl with $^X
  2019-10-02 21:36 13%       ` [PATCH] tests: recommend running create-certs.pl with $^X Alyssa Ross
@ 2019-10-02 23:19  4%         ` Eric Wong
  0 siblings, 0 replies; 49+ results
From: Eric Wong @ 2019-10-02 23:19 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> This is better than recommending running the script directly because
> it will ensure the correct version of perl is used.

Applied and pushed, thanks!

^ permalink raw reply	[relevance 4%]

* Re: Test failures in build sandbox
  2019-09-26  8:44  4%         ` Eric Wong
@ 2019-10-02 23:23  6%           ` Alyssa Ross
  2019-10-03  0:46 11%             ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-02 23:23 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 940 bytes --]

> What kind of filesystem and mount options was the code and
> (if set, TMPDIR) on?
> Since Nix seems to use odd paths, what's the default temporary
> directory? ("/tmp" on my systems)

TMPDIR is /build.  The code is extracted to /build/public-inbox.
Relevant mount output is:

    tmp on /build type tmpfs (rw,relatime)

>> t/search.t ................. fatal: Could not make /build/pi-search-i646IK/a.git/branches/ writable by group
>> t/search.t ................. 1/?
>> #   Failed test 'git init (main)'
>> #   at t/search.t line 20.
>> #          got: '0'
>> #     expected: '32768'

After some rather extensive debugging, I've determined that most of
these are to do with not being able to set the setgid bit.  If I mask it
off in git's adjust_shared_perm function, I get down to two failures.  I
don't know enough about filesystem permissions to know why this wouldn't
be allowed inside the sandbox if it is allowed normally, though.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: Test failures in build sandbox
  2019-10-02 23:23  6%           ` Alyssa Ross
@ 2019-10-03  0:46 11%             ` Alyssa Ross
  2019-10-03  1:28  4%               ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-03  0:46 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 944 bytes --]

Alyssa Ross <hi@alyssa.is> writes:

> After some rather extensive debugging, I've determined that most of
> these are to do with not being able to set the setgid bit.  If I mask it
> off in git's adjust_shared_perm function, I get down to two failures.  I
> don't know enough about filesystem permissions to know why this wouldn't
> be allowed inside the sandbox if it is allowed normally, though.

Aha!  setuid and setgid permissions are disallowed in Nix builds using
seccomp[1].

So, I suppose it would be reasonable for us to either disable the test
suite, or try to disable just the tests that try to create setgid
things, unless you're open to adding a check for such a restriction and
skipping tests if so.  More than reasonable not to, though -- seccomp is
a nightmare and next to impossible to reasonably support as upstream.

[1]: https://github.com/NixOS/nix/blob/5038e1bec43a71c97ae7f8be07218a8a2edbb6a1/src/libstore/build.cc#L2678

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 11%]

* Re: Test failures in build sandbox
  2019-10-03  0:46 11%             ` Alyssa Ross
@ 2019-10-03  1:28  4%               ` Eric Wong
  2019-10-03  1:43  6%                 ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-03  1:28 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> Alyssa Ross <hi@alyssa.is> writes:
> 
> > After some rather extensive debugging, I've determined that most of
> > these are to do with not being able to set the setgid bit.  If I mask it
> > off in git's adjust_shared_perm function, I get down to two failures.  I
> > don't know enough about filesystem permissions to know why this wouldn't
> > be allowed inside the sandbox if it is allowed normally, though.
> 
> Aha!  setuid and setgid permissions are disallowed in Nix builds using
> seccomp[1].
> 
> So, I suppose it would be reasonable for us to either disable the test
> suite, or try to disable just the tests that try to create setgid
> things, unless you're open to adding a check for such a restriction and
> skipping tests if so.  More than reasonable not to, though -- seccomp is
> a nightmare and next to impossible to reasonably support as upstream.

How do "git init --shared" tests work for the Nix git package?  We could
try to follow that as an example...

> 
> [1]: https://github.com/NixOS/nix/blob/5038e1bec43a71c97ae7f8be07218a8a2edbb6a1/src/libstore/build.cc#L2678



^ permalink raw reply	[relevance 4%]

* Re: Test failures in build sandbox
  2019-10-03  1:28  4%               ` Eric Wong
@ 2019-10-03  1:43  6%                 ` Alyssa Ross
  2019-10-03  7:57  5%                   ` [PATCH] t/search: bail out on `git init --shared' failures Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-03  1:43 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 258 bytes --]

> How do "git init --shared" tests work for the Nix git package?  We could
> try to follow that as an example...

They're disabled, hehe:

<https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/version-management/git-and-tools/git/default.nix#L274>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* [PATCH] t/search: bail out on `git init --shared' failures
  2019-10-03  1:43  6%                 ` Alyssa Ross
@ 2019-10-03  7:57  5%                   ` Eric Wong
  2019-10-03 10:15 12%                     ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-03  7:57 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> > How do "git init --shared" tests work for the Nix git package?  We could
> > try to follow that as an example...
> 
> They're disabled, hehe:
> 
> <https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/version-management/git-and-tools/git/default.nix#L274>

Yeah, seems reasonable to disable here, too.

Thanks for all your time debugging and documenting this!  Using
BAIL_OUT could save future testers a lot of time.  And I also
made another patch to make mode failures more obvious:
https://public-inbox.org/meta/20191003073817.13259-1-e@80x24.org/

----------8<-----------
Subject: [PATCH] t/search: bail out on `git init --shared' failures

We can save future testers some time if we bail out early
on "git init --shared" failures, since things like seccomp
or non-POSIX FSes would trigger failures.

BAIL_OUT has been in Test::Simple since Perl v5.10.0, so it's
old-enough to call for our purposes.

Thanks-to: Alyssa Ross <hi@alyssa.is>
Link: https://public-inbox.org/meta/878sq2hd08.fsf@alyssa.is/
---
 t/search.t | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/search.t b/t/search.t
index 830c668e..a728d79f 100644
--- a/t/search.t
+++ b/t/search.t
@@ -17,7 +17,8 @@ my $git_dir = "$tmpdir/a.git";
 my $ibx = PublicInbox::Inbox->new({ mainrepo => $git_dir });
 my ($root_id, $last_id);
 
-is(0, system(qw(git init --shared -q --bare), $git_dir), "git init (main)");
+is(0, system(qw(git init --shared -q --bare), $git_dir), "git init (main)")
+	or BAIL_OUT("`git init --shared' failed, weird FS or seccomp?");
 eval { PublicInbox::Search->new($ibx)->xdb };
 ok($@, "exception raised on non-existent DB");
 
-- 
EW

^ permalink raw reply related	[relevance 5%]

* Re: [PATCH] t/search: bail out on `git init --shared' failures
  2019-10-03  7:57  5%                   ` [PATCH] t/search: bail out on `git init --shared' failures Eric Wong
@ 2019-10-03 10:15 12%                     ` Alyssa Ross
  0 siblings, 0 replies; 49+ results
From: Alyssa Ross @ 2019-10-03 10:15 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 189 bytes --]

> Subject: [PATCH] t/search: bail out on `git init --shared' failures

Nice one!  If you want to, consider it

Reviewed-by: Alyssa Ross <hi@alyssa.is>
Tested-by: Alyssa Ross <hi@alyssa.is>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 12%]

* public-inbox-init with minimal info
@ 2019-10-03 11:16  6% Alyssa Ross
  2019-10-04  2:45  4% ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-03 11:16 UTC (permalink / raw)
  To: meta

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

In NixOS, the best way for us to provide a public-inbox module would be
to generate the configuration file ahead of time, and then initialize
inboxes that don't already exist at some reasonable time during boot.

public-inbox-init tries to write a config file in addition to
initializing an inbox.  My initial idea was to just eschew
public-inbox-init for doing git init --bare myself, which works great
for V1 repositories, but I'd really like to be generating V2 ones.

Since the V2 initialization isn't encapsulated in one easy command, I'm
wondering what the best way to accomplish initialization without writing
a config file or asking for unnecessary information is.  I could just
run public-inbox-init with PI_CONFIG=/dev/null, but then it's still not
clear to me what information about the mailbox the script requires to be
able to initialize the mailbox.  Looking at the code, I see that at
least the primary address is passed to PublicInbox::Inbox, but I'm not
sure what that would actually be used for inside the inbox.

So, what would the best thing for me to do here be?  To summarise, I'd
like to generate V2 inboxes while providing as little information about
the inbox as possible, and without writing a config file.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: public-inbox-init with minimal info
  2019-10-03 11:16  6% public-inbox-init with minimal info Alyssa Ross
@ 2019-10-04  2:45  4% ` Eric Wong
  2019-10-04 11:18  6%   ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-04  2:45 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> In NixOS, the best way for us to provide a public-inbox module would be
> to generate the configuration file ahead of time, and then initialize
> inboxes that don't already exist at some reasonable time during boot.
> 
> public-inbox-init tries to write a config file in addition to
> initializing an inbox.  My initial idea was to just eschew
> public-inbox-init for doing git init --bare myself, which works great
> for V1 repositories, but I'd really like to be generating V2 ones.
> 
> Since the V2 initialization isn't encapsulated in one easy command, I'm
> wondering what the best way to accomplish initialization without writing
> a config file or asking for unnecessary information is.  I could just
> run public-inbox-init with PI_CONFIG=/dev/null, but then it's still not
> clear to me what information about the mailbox the script requires to be
> able to initialize the mailbox.  Looking at the code, I see that at
> least the primary address is passed to PublicInbox::Inbox, but I'm not
> sure what that would actually be used for inside the inbox.

That address is used for making commits using the
public-inbox-{learn,edit,purge} commands, and also matching
with public-inbox-mda for delivery.

> So, what would the best thing for me to do here be?  To summarise, I'd
> like to generate V2 inboxes while providing as little information about
> the inbox as possible, and without writing a config file.

You want to provide that inbox during the post-install?

I'm not sure if I understand what's going on (but it's been a
long day :x).  I'm not sure if providing an inbox on package
installation is necessary or even a good thing...

Using git itself as an analogy: I wouldn't expect installing git
to auto-generate an empty git repo for me.  So same with
public-inbox, and stuff like cgit/gitweb...

Perhaps examples/public-inbox-config could add some newish
features such as nntpserver and css support and packaged
up for users, though:

diff --git a/examples/public-inbox-config b/examples/public-inbox-config
index 7fcbe0ba..a6785a7c 100644
--- a/examples/public-inbox-config
+++ b/examples/public-inbox-config
@@ -1,5 +1,13 @@
 # this usually in ~/.public-inbox/config and parseable with git-config(1)
 # update t/config.t if changing this, that test relies on this
+[publicinbox]
+	nntpserver = news.example.com
+	css = /path/to/share/public-inbox/216dark.css media=screen
+	css = /path/to/share/public-inbox/216light.css media=print
+	css = /path/to/share/public-inbox/216light.css \
+		media='screen AND (prefers-color-scheme:light)'
+[publicinboxwatch]
+	spamcheck = spamc
 [publicinbox "test"]
 	address = try@public-inbox.org
 	address = sandbox@public-inbox.org

^ permalink raw reply related	[relevance 4%]

* Re: public-inbox-init with minimal info
  2019-10-04  2:45  4% ` Eric Wong
@ 2019-10-04 11:18  6%   ` Alyssa Ross
  2019-10-05  5:14  3%     ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-04 11:18 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 3318 bytes --]

>> Since the V2 initialization isn't encapsulated in one easy command, I'm
>> wondering what the best way to accomplish initialization without writing
>> a config file or asking for unnecessary information is.  I could just
>> run public-inbox-init with PI_CONFIG=/dev/null, but then it's still not
>> clear to me what information about the mailbox the script requires to be
>> able to initialize the mailbox.  Looking at the code, I see that at
>> least the primary address is passed to PublicInbox::Inbox, but I'm not
>> sure what that would actually be used for inside the inbox.
>
> That address is used for making commits using the
> public-inbox-{learn,edit,purge} commands, and also matching
> with public-inbox-mda for delivery.

I'm guessing those read from the config file, though?  I'm trying to
figure out what configuration is stored in the inbox directory as
opposed to the config file.

> You want to provide that inbox during the post-install?
>
> I'm not sure if I understand what's going on (but it's been a
> long day :x).  I'm not sure if providing an inbox on package
> installation is necessary or even a good thing...
>
> Using git itself as an analogy: I wouldn't expect installing git
> to auto-generate an empty git repo for me.  So same with
> public-inbox, and stuff like cgit/gitweb...

I agree -- creating an inbox on package installation would be a terrible
idea, and is not what I'm proposing to do.

Some background: in the Nix ecosystem, we have a package repository,
Nixpkgs.  These packages are fairly typical except for unusual paths
(containing checksums of the "inputs" -- think dependencies -- of the
package) and the functional language they're written in.  We also have
NixOS, which is a GNU/Linux distribution, that uses those packages but
also adds the concept of "modules", written in the same language as the
packages.  These modules do things like configuring the users on your
system, setting up config files, etc.  The idea being that you can
generate a whole Linux system from pure (in the functional programming
sense) configuration, reproducibly, and have the system stuff be read
only at runtime to the maximum extent possible.  I'm working on both a
package _and_ a module for public-inbox.  The package will do exactly
what you'd expect a package to do, but the module will let you express
global and per-inbox configuration in the Nix language, generate a
read-only public-inbox config file and systemd units from those, and, at
boot time or configuration change time, initialize the inboxes defined
by the user.

As an example, Tor in NixOS looks like this:

    { ... }:

    {
      services.tor.enable = true;
      services.tor.hiddenServices = [
        {
          name = "public-inbox";
          map = [ { port = 80; destination = 8000; } ];
          version = 3;
        };
      ];
    };

This will generate a static tor.service and tor config file -- we do as
much as possible staticly and purely because then we know that if a
configuration is rolled back, we can remove the service etc.  For state,
however, like the hidden service private key (in this case -- I could
have used a static one here if I'd wanted), it will be generated either
at boot time or in Tor's ExecStartPre.  This is the same mechanism I
would use to run public-inbox-init.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: public-inbox-init with minimal info
  2019-10-04 11:18  6%   ` Alyssa Ross
@ 2019-10-05  5:14  3%     ` Eric Wong
  2019-10-05 13:05  6%       ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-05  5:14 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> >> Since the V2 initialization isn't encapsulated in one easy command, I'm
> >> wondering what the best way to accomplish initialization without writing
> >> a config file or asking for unnecessary information is.  I could just
> >> run public-inbox-init with PI_CONFIG=/dev/null, but then it's still not
> >> clear to me what information about the mailbox the script requires to be
> >> able to initialize the mailbox.  Looking at the code, I see that at
> >> least the primary address is passed to PublicInbox::Inbox, but I'm not
> >> sure what that would actually be used for inside the inbox.
> >
> > That address is used for making commits using the
> > public-inbox-{learn,edit,purge} commands, and also matching
> > with public-inbox-mda for delivery.
> 
> I'm guessing those read from the config file, though?  I'm trying to
> figure out what configuration is stored in the inbox directory as
> opposed to the config file.

Yes, they all read from the config file because they need to
operate on inboxes.   public-inbox-{edit,purge,index,xcpdb,compact}
can also operate on inboxes outside of the config file, but -mda
and -learn needs the config.

There isn't any config stored in the inbox directories
themselves, although there's some metadata in SQLite for
incremental indexing and indexlevel for Xapian.

> > You want to provide that inbox during the post-install?
> >
> > I'm not sure if I understand what's going on (but it's been a
> > long day :x).  I'm not sure if providing an inbox on package
> > installation is necessary or even a good thing...
> >
> > Using git itself as an analogy: I wouldn't expect installing git
> > to auto-generate an empty git repo for me.  So same with
> > public-inbox, and stuff like cgit/gitweb...
> 
> I agree -- creating an inbox on package installation would be a terrible
> idea, and is not what I'm proposing to do.

Good to know :>

> Some background: in the Nix ecosystem, we have a package repository,
> Nixpkgs.  These packages are fairly typical except for unusual paths
> (containing checksums of the "inputs" -- think dependencies -- of the
> package) and the functional language they're written in.  We also have
> NixOS, which is a GNU/Linux distribution, that uses those packages but
> also adds the concept of "modules", written in the same language as the
> packages.  These modules do things like configuring the users on your
> system, setting up config files, etc.  The idea being that you can
> generate a whole Linux system from pure (in the functional programming
> sense) configuration, reproducibly, and have the system stuff be read
> only at runtime to the maximum extent possible.  I'm working on both a
> package _and_ a module for public-inbox.  The package will do exactly
> what you'd expect a package to do, but the module will let you express
> global and per-inbox configuration in the Nix language, generate a
> read-only public-inbox config file and systemd units from those, and, at
> boot time or configuration change time, initialize the inboxes defined
> by the user.

Thanks for that clarification, NixOS modules are a new concept
to me.

I'm not sure how the public-inbox config file can/should remain
read-only.  It's analogous to a config file for cgit or gitweb,
so maybe modules for those can offer some inspiration...

> As an example, Tor in NixOS looks like this:
> 
>     { ... }:
> 
>     {
>       services.tor.enable = true;
>       services.tor.hiddenServices = [
>         {
>           name = "public-inbox";
>           map = [ { port = 80; destination = 8000; } ];
>           version = 3;
>         };
>       ];
>     };
> 
> This will generate a static tor.service and tor config file -- we do as
> much as possible staticly and purely because then we know that if a
> configuration is rolled back, we can remove the service etc.  For state,
> however, like the hidden service private key (in this case -- I could
> have used a static one here if I'd wanted), it will be generated either
> at boot time or in Tor's ExecStartPre.  This is the same mechanism I
> would use to run public-inbox-init.

So that means a new tor process is spawned for every hidden
service?  (instead of one tor process running multiple
services).  It works, but it's not memory-efficient (but
could be more secure if tor has bugs).

Like gitweb and cgit, public-inbox-{httpd,nntpd} usually expects
to serve multiple inboxes off one instance.  It's possible to
start an independent -httpd/-nntpd instance for every inbox,
each with it's own config, but that's not efficient at all.

Both the WWW and NNTP code are able to scan Message-IDs across
multiple inboxes in the same process in case of cross-posts
between related inboxes.  At some point, I think it'll be useful
to define groups/relationships between inboxes for the WWW UI.

Likewise, public-inbox-watch would be expected to watch multiple
Maildirs and write to multiple inboxes, if needed.  Maybe
modules for mail downloading/notification tools such as
offlineimap/mbsync(isync) could serve as inspiration, there.

I tried to keep most of the daemon-specific config knobs in the
command-line, so it goes into .service files, which can be
read-only (well, unless somebody wants to change worker
processes via "-W $NUM").

But yeah, packaging services/modules for different systems and
use-cases is hard, everybody seems to do it differently and
want different things.  So I'm really not sure how packaging
a module would work, it seems like that's something each user
would want to manage on their own.

^ permalink raw reply	[relevance 3%]

* Re: public-inbox-init with minimal info
  2019-10-05  5:14  3%     ` Eric Wong
@ 2019-10-05 13:05  6%       ` Alyssa Ross
  2019-10-05 19:58  4%         ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-05 13:05 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 2251 bytes --]

> There isn't any config stored in the inbox directories
> themselves, although there's some metadata in SQLite for
> incremental indexing and indexlevel for Xapian.

Cool, okay, I think this is what I wanted to know.  I'll generate an
inbox and see what's in sqlite, and omit anything that isn't used there.

> I'm not sure how the public-inbox config file can/should remain
> read-only.  It's analogous to a config file for cgit or gitweb,
> so maybe modules for those can offer some inspiration...

I've been using cgit fine with a readonly config for ages, fwiw.

>> As an example, Tor in NixOS looks like this:
>> 
>>     { ... }:
>> 
>>     {
>>       services.tor.enable = true;
>>       services.tor.hiddenServices = [
>>         {
>>           name = "public-inbox";
>>           map = [ { port = 80; destination = 8000; } ];
>>           version = 3;
>>         };
>>       ];
>>     };
>> 
>> This will generate a static tor.service and tor config file -- we do as
>> much as possible staticly and purely because then we know that if a
>> configuration is rolled back, we can remove the service etc.  For state,
>> however, like the hidden service private key (in this case -- I could
>> have used a static one here if I'd wanted), it will be generated either
>> at boot time or in Tor's ExecStartPre.  This is the same mechanism I
>> would use to run public-inbox-init.
>
> So that means a new tor process is spawned for every hidden
> service?  (instead of one tor process running multiple
> services).  It works, but it's not memory-efficient (but
> could be more secure if tor has bugs).

No.  If I had added multiple hidden services above, Nix would still
generate one config file, one systemd service, etc.

> But yeah, packaging services/modules for different systems and
> use-cases is hard, everybody seems to do it differently and
> want different things.  So I'm really not sure how packaging
> a module would work, it seems like that's something each user
> would want to manage on their own.

Well, the idea is to provide sufficient configuration options that it
should be sufficient for almost everybody's needs.  Tor has way more
options than I showed above, for example.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: public-inbox-init with minimal info
  2019-10-05 13:05  6%       ` Alyssa Ross
@ 2019-10-05 19:58  4%         ` Eric Wong
  2019-10-06  9:52  6%           ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-05 19:58 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> > There isn't any config stored in the inbox directories
> > themselves, although there's some metadata in SQLite for
> > incremental indexing and indexlevel for Xapian.
> 
> Cool, okay, I think this is what I wanted to know.  I'll generate an
> inbox and see what's in sqlite, and omit anything that isn't used there.
> 
> > I'm not sure how the public-inbox config file can/should remain
> > read-only.  It's analogous to a config file for cgit or gitweb,
> > so maybe modules for those can offer some inspiration...
> 
> I've been using cgit fine with a readonly config for ages, fwiw.

Is that possible without scan-path/project-list options in cgitrc?
public-inbox has nothing analogous to those options, right now.

> >> As an example, Tor in NixOS looks like this:
> >> 
> >>     { ... }:
> >> 
> >>     {
> >>       services.tor.enable = true;
> >>       services.tor.hiddenServices = [
> >>         {
> >>           name = "public-inbox";
> >>           map = [ { port = 80; destination = 8000; } ];
> >>           version = 3;
> >>         };
> >>       ];
> >>     };
> >> 
> >> This will generate a static tor.service and tor config file -- we do as
> >> much as possible staticly and purely because then we know that if a
> >> configuration is rolled back, we can remove the service etc.  For state,
> >> however, like the hidden service private key (in this case -- I could
> >> have used a static one here if I'd wanted), it will be generated either
> >> at boot time or in Tor's ExecStartPre.  This is the same mechanism I
> >> would use to run public-inbox-init.
> >
> > So that means a new tor process is spawned for every hidden
> > service?  (instead of one tor process running multiple
> > services).  It works, but it's not memory-efficient (but
> > could be more secure if tor has bugs).
> 
> No.  If I had added multiple hidden services above, Nix would still
> generate one config file, one systemd service, etc.

one systemd service == one tor process, right?  I haven't looked
too deeply into systemd, though, so maybe there's some way to
add services to an existing tor process...

> > But yeah, packaging services/modules for different systems and
> > use-cases is hard, everybody seems to do it differently and
> > want different things.  So I'm really not sure how packaging
> > a module would work, it seems like that's something each user
> > would want to manage on their own.
> 
> Well, the idea is to provide sufficient configuration options that it
> should be sufficient for almost everybody's needs.  Tor has way more
> options than I showed above, for example.

I tried to make the defaults reasonable, so I don't think any
config is needed outside of what's required to map
inboxes/addresses to directories (which public-inbox-init does)

^ permalink raw reply	[relevance 4%]

* Re: public-inbox-init with minimal info
  2019-10-05 19:58  4%         ` Eric Wong
@ 2019-10-06  9:52  6%           ` Alyssa Ross
  2019-10-06 12:01  4%             ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-06  9:52 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 2640 bytes --]

>> I've been using cgit fine with a readonly config for ages, fwiw.
>
> Is that possible without scan-path/project-list options in cgitrc?
> public-inbox has nothing analogous to those options, right now.

Yes, because I can generate my cgit configuration file with Nix.
Theer's no static config file or anything -- it's generated from the
configuration options provided by the user.

So, as a user of cgit in NixOS, I can define my repositories, and the
paths to them in Nix, and when I "rebuild"[1] my system the cgit
configuration file is generated based on that.

[1]: conceptually an update, but works by generating a new system
     configuration without referring to the current one, so it's more
     like rebuilding from scratch

> one systemd service == one tor process, right?  I haven't looked
> too deeply into systemd, though, so maybe there's some way to
> add services to an existing tor process...

At least in this case, yes, one systemd service == one tor process.  We
don't support more than one, AFAIK.  That would have to be done in a
container or something.

> I tried to make the defaults reasonable, so I don't think any
> config is needed outside of what's required to map
> inboxes/addresses to directories (which public-inbox-init does)

The only one I've really added so far that affects the public-inbox
config is whether to enable spam checking or not, but I suspect there
might be more.  There are also options for things like whether a service
should be generated to run public-inbox-httpd, etc.

Here's what my configuration looks like so far, using the module I'm
writing:

    services.public-inbox.enable = true;

    # Add spamassassin to the PATH of public-inbox-mda,
    # public-inbox-watch, etc.
    services.public-inbox.path = with pkgs; [ spamassassin ];
    
    services.public-inbox.mda.spamCheck = "spamc";
    services.public-inbox.http.mount = "/lists/archives/";
    services.public-inbox.inboxes = { [...] };

As you can see, it's in some ways like just writing a public-inbox
configuration file, but it can go beyond that too -- there can be
options like services.public-inbox.path that are more of a packaging but
that can be delegated to a user (by default, services on NixOS have
almost nothing in their PATH to ensure purity).  I'm probably not the
best person to explain why NixOS modules are good, or the benefits of
expressing all system configuration in a single functional programming
language, so suffice it to say that doing these things are the
fundamental goals of the distribution, and that it works extremely well.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: public-inbox-init with minimal info
  2019-10-06  9:52  6%           ` Alyssa Ross
@ 2019-10-06 12:01  4%             ` Eric Wong
  2019-10-07 20:52  5%               ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-06 12:01 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> >> I've been using cgit fine with a readonly config for ages, fwiw.
> >
> > Is that possible without scan-path/project-list options in cgitrc?
> > public-inbox has nothing analogous to those options, right now.
> 
> Yes, because I can generate my cgit configuration file with Nix.
> Theer's no static config file or anything -- it's generated from the
> configuration options provided by the user.
> 
> So, as a user of cgit in NixOS, I can define my repositories, and the
> paths to them in Nix, and when I "rebuild"[1] my system the cgit
> configuration file is generated based on that.
> 
> [1]: conceptually an update, but works by generating a new system
>      configuration without referring to the current one, so it's more
>      like rebuilding from scratch

OK, it seems like you can "build" the full public-inbox config
file the same way you'd build your cgitrc.

> > one systemd service == one tor process, right?  I haven't looked
> > too deeply into systemd, though, so maybe there's some way to
> > add services to an existing tor process...
> 
> At least in this case, yes, one systemd service == one tor process.  We
> don't support more than one, AFAIK.  That would have to be done in a
> container or something.
> 
> > I tried to make the defaults reasonable, so I don't think any
> > config is needed outside of what's required to map
> > inboxes/addresses to directories (which public-inbox-init does)
> 
> The only one I've really added so far that affects the public-inbox
> config is whether to enable spam checking or not, but I suspect there
> might be more.  There are also options for things like whether a service
> should be generated to run public-inbox-httpd, etc.

Yeah, I run -httpd, -nntpd, and -watch as services (see examples/ dir);
so it makes sense to have those in a package.  -httpd/-nntpd
generally run on the same machines, but they don't need -watch
on the same server (they can be running off git-clone/fetch &&
public-inbox-index)

> Here's what my configuration looks like so far, using the module I'm
> writing:
> 
>     services.public-inbox.enable = true;
> 
>     # Add spamassassin to the PATH of public-inbox-mda,
>     # public-inbox-watch, etc.
>     services.public-inbox.path = with pkgs; [ spamassassin ];

They'll all need git, too (unless that's in the default path).
Also, httpd/nntpd don't need spamassassin.

>     services.public-inbox.mda.spamCheck = "spamc";

Note: one slight oddity is there's also a "publicinboxwatch.spamc"
in addition to "publicinboxmda.spamc"...  I figure some folks will
want differently-configured spamcheckers depending on whether the
mail hits -mda or -watch (so -watch defaults to not having a
spamchecker at all).

Does Nix allow users to set things in the config file directly?
(instead of going through the functional language).

I'm also not sure if you need to have
"services.public-inbox.mda.spamCheck" at all, since "spamc"
is the default value.

>     services.public-inbox.http.mount = "/lists/archives/";

I think all the services would want access to the same
directories, not just httpd (if I'm understanding that config
correctly).  Also, httpd/nntpd only need read-only access to their
mount points, in case that affects things...

>     services.public-inbox.inboxes = { [...] };
> 
> As you can see, it's in some ways like just writing a public-inbox
> configuration file, but it can go beyond that too -- there can be
> options like services.public-inbox.path that are more of a packaging but
> that can be delegated to a user (by default, services on NixOS have
> almost nothing in their PATH to ensure purity).  I'm probably not the
> best person to explain why NixOS modules are good, or the benefits of
> expressing all system configuration in a single functional programming
> language, so suffice it to say that doing these things are the
> fundamental goals of the distribution, and that it works extremely well.

The purity and rollback parts definitely sound good :)

However, I'm wondering what level of customization can be
supported by editing the public-inbox config directly? (instead
of using the Nix language)

Having both an upstream and distro-specific ways to configure the
same thing could be confusing to both users and people trying to
help them.

I can agree with things like PATH, environment variables,
services-to-enable and mount-points being environment and/or
distro-specific.  The rest (everything in public-inbox-config(5)),
I'm not sure about; it would increase support/doc overhead.

^ permalink raw reply	[relevance 4%]

* [PATCH] public-inbox-v2-format(5): fix formatting
@ 2019-10-07 20:27 13% Alyssa Ross
  2019-10-07 20:52  4% ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-07 20:27 UTC (permalink / raw)
  To: meta; +Cc: Alyssa Ross

This was being rendered as a paragraph, so line breaks weren't
preserved and it was unreadable in man.
---
 Documentation/public-inbox-v2-format.pod | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/Documentation/public-inbox-v2-format.pod b/Documentation/public-inbox-v2-format.pod
index 28d3550..58e6b6d 100644
--- a/Documentation/public-inbox-v2-format.pod
+++ b/Documentation/public-inbox-v2-format.pod
@@ -24,13 +24,13 @@ $EPOCH - Integer starting with 0 based on time
 $SCHEMA_VERSION - PublicInbox::Search::SCHEMA_VERSION used by Xapian
 $PART - Integer (0..NPROCESSORS)
 
-foo/ # assuming "foo" is the name of the list
-- inbox.lock                 # lock file (flock) to protect global state
-- git/$EPOCH.git             # normal git repositories
-- all.git                    # empty git repo, alternates to git/$EPOCH.git
-- xap$SCHEMA_VERSION/$SHARD  # per-shard Xapian DB
-- xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP and threading
-- msgmap.sqlite3             # same the v1 msgmap
+	foo/                              # assuming "foo" is the name of the list
+	- inbox.lock                      # lock file (flock) to protect global state
+	- git/$EPOCH.git                  # normal git repositories
+	- all.git                         # empty git repo, alternates to git/$EPOCH.git
+	- xap$SCHEMA_VERSION/$SHARD       # per-shard Xapian DB
+	- xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP and threading
+	- msgmap.sqlite3                  # same the v1 msgmap
 
 For blob lookups, the reader only needs to open the "all.git"
 repository with $GIT_DIR/objects/info/alternates which references
-- 
2.23.0


^ permalink raw reply related	[relevance 13%]

* Re: [PATCH] public-inbox-v2-format(5): fix formatting
  2019-10-07 20:27 13% [PATCH] public-inbox-v2-format(5): fix formatting Alyssa Ross
@ 2019-10-07 20:52  4% ` Eric Wong
  2019-10-07 21:13  6%   ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-07 20:52 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> This was being rendered as a paragraph, so line breaks weren't
> preserved and it was unreadable in man.

Thanks for noticing that, it's obvious I never really looked at
the result :x  (and I'm writing a new .pod right now, too)

> diff --git a/Documentation/public-inbox-v2-format.pod b/Documentation/public-inbox-v2-format.pod
> index 28d3550..58e6b6d 100644
> --- a/Documentation/public-inbox-v2-format.pod
> +++ b/Documentation/public-inbox-v2-format.pod
> @@ -24,13 +24,13 @@ $EPOCH - Integer starting with 0 based on time
>  $SCHEMA_VERSION - PublicInbox::Search::SCHEMA_VERSION used by Xapian
>  $PART - Integer (0..NPROCESSORS)

Looks like the lines preceding this patch need to be indented to
preserve newlines, too.

> -foo/ # assuming "foo" is the name of the list
> -- inbox.lock                 # lock file (flock) to protect global state
> -- git/$EPOCH.git             # normal git repositories
> -- all.git                    # empty git repo, alternates to git/$EPOCH.git
> -- xap$SCHEMA_VERSION/$SHARD  # per-shard Xapian DB
> -- xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP and threading
> -- msgmap.sqlite3             # same the v1 msgmap
> +	foo/                              # assuming "foo" is the name of the list
> +	- inbox.lock                      # lock file (flock) to protect global state
> +	- git/$EPOCH.git                  # normal git repositories
> +	- all.git                         # empty git repo, alternates to git/$EPOCH.git
> +	- xap$SCHEMA_VERSION/$SHARD       # per-shard Xapian DB
> +	- xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP and threading
> +	- msgmap.sqlite3                  # same the v1 msgmap

I noticed both the original and post-patch manpage exceeds the
limits of an 80-column terminal.  I think it's worth it to
reflow and perhaps reword to ensure folks who are limited to 80
columns can viwe (but I can take care of it, too, if you're busy)

Even fitting 80 columns on my screen is a challenge with the
giant fonts I need to workaround poor eyesight :<

^ permalink raw reply	[relevance 4%]

* Re: public-inbox-init with minimal info
  2019-10-06 12:01  4%             ` Eric Wong
@ 2019-10-07 20:52  5%               ` Alyssa Ross
  2019-10-08  7:11  4%                 ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-07 20:52 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 5970 bytes --]

> OK, it seems like you can "build" the full public-inbox config
> file the same way you'd build your cgitrc.

Yep!  I've had that working for a couple of weeks now and it's working
great!

> Yeah, I run -httpd, -nntpd, and -watch as services (see examples/ dir);
> so it makes sense to have those in a package.  -httpd/-nntpd
> generally run on the same machines, but they don't need -watch
> on the same server (they can be running off git-clone/fetch &&
> public-inbox-index)

I haven't looked into watch yet -- I've been implementing for my use
case first, which is using public-inbox as an archiver for a mailing
list hosted on the same server.  But that does make sense, and I'll make
sure to account for it.

>>     # Add spamassassin to the PATH of public-inbox-mda,
>>     # public-inbox-watch, etc.
>>     services.public-inbox.path = with pkgs; [ spamassassin ];
>
> They'll all need git, too (unless that's in the default path).
> Also, httpd/nntpd don't need spamassassin.

Yeah, this is only for -mda and -watch.  git is just added to the PATH
of every public-inbox-* executable in the package using a wrapper script.

> Note: one slight oddity is there's also a "publicinboxwatch.spamc"
> in addition to "publicinboxmda.spamc"...  I figure some folks will
> want differently-configured spamcheckers depending on whether the
> mail hits -mda or -watch (so -watch defaults to not having a
> spamchecker at all).

Noticed that, and will account for it.  As above, I'm just not using
-watch in my setup.

> Does Nix allow users to set things in the config file directly?
> (instead of going through the functional language).

It does; see below.

> I'm also not sure if you need to have
> "services.public-inbox.mda.spamCheck" at all, since "spamc"
> is the default value.

Correct -- that's a remnant of when I had it disabled because I hadn't
set up spamassassin yet.  I should have deleted the line rather than
changing it.

>>     services.public-inbox.http.mount = "/lists/archives/";
>
> I think all the services would want access to the same
> directories, not just httpd (if I'm understanding that config
> correctly).  Also, httpd/nntpd only need read-only access to their
> mount points, in case that affects things...

"mount" here is in the PSGI sense, not the file system sense.  My
public-inboxes are at https://example.org/lists/archives/.  Maybe
there's a better name.

> The purity and rollback parts definitely sound good :)
>
> However, I'm wondering what level of customization can be
> supported by editing the public-inbox config directly? (instead
> of using the Nix language)
>
> Having both an upstream and distro-specific ways to configure the
> same thing could be confusing to both users and people trying to
> help them.
>
> I can agree with things like PATH, environment variables,
> services-to-enable and mount-points being environment and/or
> distro-specific.  The rest (everything in public-inbox-config(5)),
> I'm not sure about; it would increase support/doc overhead.

We accept that we can't package every option and have two conventions
for making sure that our users can always use every option upstream
gives them.

One is to provide an option called extraConfig, which just adds a string
verbatim to the end of the configuration file.  The other is to provide
an option called config, which is structured Nix that gets turned into
the appropriate configuration.  The latter is preferred, because then
from our point of view the configuration is still structured, and can be
read back in Nix without having to parse a string, etc.  Since
public-inbox's configuration format is well-defined as git config,
there's no reason to support the former.  So, supposing you introduce a
new option, publicinbox.foo, and the NixOS module hasn't yet been
updated to support it natively yet.  In that case, a user could use this
option as an escape hatch to use it anyway:

    services.public-inbox.config = {
      publicinbox.foo = "bar";
    };

This will then be compiled into git config using a function I've written
that essentially runs git config --add for each config option to build
up a configuration file.

By now I'm sure you're wondering "why bother adding individual NixOS
options for each setting at all, if you can do this?", and there are a
couple of reasons why we try to do it anyway.  One is that we can do
type checking -- setting publicinboxmda.spamcheck to "invalid" can be a
build-time failure rather than a runtime one.  The other is that we can
provide documentation for each option, and our users can see
documentation for every option available on their NixOS system in one
place at <https://nixos.org/nixos/options.html>.  We regularly hear from
users that this is one of their favourite things about NixOS.  A single
place to search configuration options for every package they use.

I hear your concerns about this being difficult for people trying to
help NixOS users with public-inbox.  It's absolutely not my goal to
fragment the ecosystem.  I'm not aware of this having been a significant
issue for any of the hundreds of modules we have so far.  Users seem to
be generally pretty good at figuring out what's a NixOS issue and what's
an upstream issue, and, if a NixOS user does need upstream support, it's
still easy enough for them to find the generated configuration file to
share with non-NixOS users.  Overall, we've found that the benefits of
"heavyweight" NixOS modules, for want of a better term, outweigh the
disadvantages.  Should this end up having a negative impact on the
public-inbox system, I'd be happy to review the approach.  But I think
that instead, we'll end up with public-inbox being accessible to more
people by being available with the advantages of NixOS --
declarative configuration, reproducibility, near-atomic updates and
rollbacks, etc.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] public-inbox-v2-format(5): fix formatting
  2019-10-07 20:52  4% ` Eric Wong
@ 2019-10-07 21:13  6%   ` Alyssa Ross
  2019-10-15 20:11  4%     ` [PATCH 2/1] doc: v2-format: get man output under 80 cols Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-07 21:13 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 480 bytes --]

> I noticed both the original and post-patch manpage exceeds the
> limits of an 80-column terminal.  I think it's worth it to
> reflow and perhaps reword to ensure folks who are limited to 80
> columns can viwe (but I can take care of it, too, if you're busy)
>
> Even fitting 80 columns on my screen is a challenge with the
> giant fonts I need to workaround poor eyesight :<

I appreciate your commitment!  I think you'd be a better judge of how
to format it nicely than me. :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Do I need multiple publicinbox.<name>.address values?
@ 2019-10-07 22:13  6% Alyssa Ross
  2019-10-08  0:10  4% ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-07 22:13 UTC (permalink / raw)
  To: meta

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

Suppose I have a mailing list, foo-discuss@example.org, and a
public-inbox set up, subscribed to that mailing list, that is subscribed
to that list as public-inbox+foo-discuss@example.org, where my MTA
delivers to public-inbox using public-inbox-mda.

In this case, mail will be To: foo-discuss@example.org, but RCPT TO (and
therefore ORIGINAL_RECIPIENT) public-inbox+foo-discuss@example.org.  In
my testing, public-inbox requires *both* the To: header and
ORIGINAL_RECIPIENT to match publicinbox.<name>.address.

But, public-inbox-config(7) has this to say for
publicinbox.<name>.address:

> The email address of the public-inbox.  May be specified
> more than once for merging multiple mailing lists (or migrating
> to new addresses).

Nothing in this suggests to me that I would need to set it more than
once for my simple use case.

So my question is, am I overcomplicating this -- should I be able to set
a single value for publicinbox.<name>.address that will get my
public-inbox to accept mail from the list?  Or, alternatively, should a
sentence be added to that description explaining that multiple addresses
will also be required for the case where public-inbox is subscribed
itself to a mailing list?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: Do I need multiple publicinbox.<name>.address values?
  2019-10-07 22:13  6% Do I need multiple publicinbox.<name>.address values? Alyssa Ross
@ 2019-10-08  0:10  4% ` Eric Wong
  2019-10-08 12:18  0%   ` Eric W. Biederman
  2019-10-09 11:59  6%   ` Alyssa Ross
  0 siblings, 2 replies; 49+ results
From: Eric Wong @ 2019-10-08  0:10 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta, Eric W. Biederman

Alyssa Ross <hi@alyssa.is> wrote:

> Subject: Do I need multiple publicinbox.<name>.address values?

Absolutely not

> Suppose I have a mailing list, foo-discuss@example.org, and a
> public-inbox set up, subscribed to that mailing list, that is subscribed
> to that list as public-inbox+foo-discuss@example.org, where my MTA
> delivers to public-inbox using public-inbox-mda.

Currently, -mda does if you're mirroring, unfortunately.  I
think Eric Biederman is/was working on List-Id support to drop
that requirement, but I'm not sure where that is...

Eric B: would you mind if I take List-Id support over?  I've got
some hours free in the coming days(s)... (I think :x)

> In this case, mail will be To: foo-discuss@example.org, but RCPT TO (and
> therefore ORIGINAL_RECIPIENT) public-inbox+foo-discuss@example.org.  In
> my testing, public-inbox requires *both* the To: header and
> ORIGINAL_RECIPIENT to match publicinbox.<name>.address.

Yes, but To:/Cc: and ORIGINAL_RECIPIENT can be different.
lore.kernel.org uses two addresses: one for the well-known
address (e.g. linux-kernel@vger.kernel.org) and the other
something@something.kernel.org, which nobody (should) be
sending to.

> But, public-inbox-config(7) has this to say for
> publicinbox.<name>.address:

> > The email address of the public-inbox.  May be specified
> > more than once for merging multiple mailing lists (or migrating
> > to new addresses).

> Nothing in this suggests to me that I would need to set it more than
> once for my simple use case.

Sorry for that confusion.  Historically, I wrote public-inbox
and -mda because I needed to migrate mailing lists with a
several month window where the old list would be active,
but I was preparing for the switch:

So it started as:

  posters -> to_be_shutdown_host -> MTA (postfix) -> public-inbox-mda

And ends up being:

  posters -> MTA (postfix) -> public-inbox-mda --[1]--> (mlmmj|mailman)


It sounds like what you're doing is:

  MTA -> (mlmmj|mailman) -> public-inbox-mda

Which wasn't my original intended usecase for -mda, but is for -watch.

> So my question is, am I overcomplicating this -- should I be able to set
> a single value for publicinbox.<name>.address that will get my
> public-inbox to accept mail from the list?  Or, alternatively, should a
> sentence be added to that description explaining that multiple addresses
> will also be required for the case where public-inbox is subscribed
> itself to a mailing list?

Generally, I use -watch exclusively when mirroring lists.
With -watch, you won't need to set extra addresses.

But apparently some folks prefer -mda for everything.  So I
guess we'll have to better support that, too (via List-Id).



(*) via scripts/ssoma-replay, which I intend to replace with an
    nntpd-based replacement at some point for v2 compatibility.

^ permalink raw reply	[relevance 4%]

* Re: public-inbox-init with minimal info
  2019-10-07 20:52  5%               ` Alyssa Ross
@ 2019-10-08  7:11  4%                 ` Eric Wong
  2019-10-09 12:09  6%                   ` Alyssa Ross
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-08  7:11 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:

<snip>

> Eric Wong wrote:
> > Does Nix allow users to set things in the config file directly?
> > (instead of going through the functional language).
> 
> It does; see below.

<snip>

> >>     services.public-inbox.http.mount = "/lists/archives/";
> >
> > I think all the services would want access to the same
> > directories, not just httpd (if I'm understanding that config
> > correctly).  Also, httpd/nntpd only need read-only access to their
> > mount points, in case that affects things...
> 
> "mount" here is in the PSGI sense, not the file system sense.  My
> public-inboxes are at https://example.org/lists/archives/.  Maybe
> there's a better name.

Ah, so that overrides the Plack::Builder DSL/language.  We also
have an analogous support problem for PSGI vs public-inbox-config,
so I've been avoiding any overlap between them.

Perhaps "psgi_mount" would be clearer? *shrug*

> > The purity and rollback parts definitely sound good :)
> >
> > However, I'm wondering what level of customization can be
> > supported by editing the public-inbox config directly? (instead
> > of using the Nix language)
> >
> > Having both an upstream and distro-specific ways to configure the
> > same thing could be confusing to both users and people trying to
> > help them.
> >
> > I can agree with things like PATH, environment variables,
> > services-to-enable and mount-points being environment and/or
> > distro-specific.  The rest (everything in public-inbox-config(5)),
> > I'm not sure about; it would increase support/doc overhead.
> 
> We accept that we can't package every option and have two conventions
> for making sure that our users can always use every option upstream
> gives them.
> 
> One is to provide an option called extraConfig, which just adds a string
> verbatim to the end of the configuration file.  The other is to provide
> an option called config, which is structured Nix that gets turned into
> the appropriate configuration.  The latter is preferred, because then
> from our point of view the configuration is still structured, and can be
> read back in Nix without having to parse a string, etc.  Since
> public-inbox's configuration format is well-defined as git config,
> there's no reason to support the former.  So, supposing you introduce a
> new option, publicinbox.foo, and the NixOS module hasn't yet been
> updated to support it natively yet.  In that case, a user could use this
> option as an escape hatch to use it anyway:
> 
>     services.public-inbox.config = {
>       publicinbox.foo = "bar";
>     };
> 
> This will then be compiled into git config using a function I've written
> that essentially runs git config --add for each config option to build
> up a configuration file.
> 
> By now I'm sure you're wondering "why bother adding individual NixOS
> options for each setting at all, if you can do this?", and there are a
> couple of reasons why we try to do it anyway.  One is that we can do
> type checking -- setting publicinboxmda.spamcheck to "invalid" can be a
> build-time failure rather than a runtime one.  The other is that we can
> provide documentation for each option, and our users can see
> documentation for every option available on their NixOS system in one
> place at <https://nixos.org/nixos/options.html>.  We regularly hear from
> users that this is one of their favourite things about NixOS.  A single
> place to search configuration options for every package they use.

Cool.  An upside for non-NixOS users is we get more experienced
and clueful maintainers reporting bugs to upstream as a result :)

Btw, would it be helpful if public-inbox provided a linter for
its config own file?

I'm actually thinking of doing some sort of graphing tool using
Graph::Easy to visualize all the relationships between various
components, and maybe it can parse the config file to show
people how things work; and it could do the linting as a side-effect.

> I hear your concerns about this being difficult for people trying to
> help NixOS users with public-inbox.  It's absolutely not my goal to
> fragment the ecosystem.  I'm not aware of this having been a significant
> issue for any of the hundreds of modules we have so far.  Users seem to
> be generally pretty good at figuring out what's a NixOS issue and what's
> an upstream issue, and, if a NixOS user does need upstream support, it's
> still easy enough for them to find the generated configuration file to
> share with non-NixOS users.  Overall, we've found that the benefits of
> "heavyweight" NixOS modules, for want of a better term, outweigh the
> disadvantages.  Should this end up having a negative impact on the
> public-inbox system, I'd be happy to review the approach.  But I think
> that instead, we'll end up with public-inbox being accessible to more
> people by being available with the advantages of NixOS --
> declarative configuration, reproducibility, near-atomic updates and
> rollbacks, etc.

Thanks, that is all very reassuring to read.

^ permalink raw reply	[relevance 4%]

* Re: Do I need multiple publicinbox.<name>.address values?
  2019-10-08  0:10  4% ` Eric Wong
@ 2019-10-08 12:18  0%   ` Eric W. Biederman
  2019-10-08 22:11  0%     ` Eric Wong
  2019-10-09 11:59  6%   ` Alyssa Ross
  1 sibling, 1 reply; 49+ results
From: Eric W. Biederman @ 2019-10-08 12:18 UTC (permalink / raw)
  To: Eric Wong; +Cc: Alyssa Ross, meta

Eric Wong <e@80x24.org> writes:

> Alyssa Ross <hi@alyssa.is> wrote:
>
>> Subject: Do I need multiple publicinbox.<name>.address values?
>
> Absolutely not
>
>> Suppose I have a mailing list, foo-discuss@example.org, and a
>> public-inbox set up, subscribed to that mailing list, that is subscribed
>> to that list as public-inbox+foo-discuss@example.org, where my MTA
>> delivers to public-inbox using public-inbox-mda.
>
> Currently, -mda does if you're mirroring, unfortunately.  I
> think Eric Biederman is/was working on List-Id support to drop
> that requirement, but I'm not sure where that is...
>
> Eric B: would you mind if I take List-Id support over?  I've got
> some hours free in the coming days(s)... (I think :x)

I believe I have the config side of the work done.  I haven't
figured out how to add this to public-inbox-mda/public-inbox-watch.

Let me send out what I have and then you can work on the bits
for public-inbox-watch public-inbox-mda.

Last round I was messing with this I almost had my imap fetcher in
shape. I may try again but let's get the listid thing settled first
if we can.

Eric

^ permalink raw reply	[relevance 0%]

* Re: Do I need multiple publicinbox.<name>.address values?
  2019-10-08 12:18  0%   ` Eric W. Biederman
@ 2019-10-08 22:11  0%     ` Eric Wong
  2019-10-08 22:24  0%       ` Eric W. Biederman
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-08 22:11 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: Alyssa Ross, meta

"Eric W. Biederman" <ebiederm@xmission.com> wrote:
> Eric Wong <e@80x24.org> writes:
> 
> > Alyssa Ross <hi@alyssa.is> wrote:
> >
> >> Subject: Do I need multiple publicinbox.<name>.address values?
> >
> > Absolutely not
> >
> >> Suppose I have a mailing list, foo-discuss@example.org, and a
> >> public-inbox set up, subscribed to that mailing list, that is subscribed
> >> to that list as public-inbox+foo-discuss@example.org, where my MTA
> >> delivers to public-inbox using public-inbox-mda.
> >
> > Currently, -mda does if you're mirroring, unfortunately.  I
> > think Eric Biederman is/was working on List-Id support to drop
> > that requirement, but I'm not sure where that is...
> >
> > Eric B: would you mind if I take List-Id support over?  I've got
> > some hours free in the coming days(s)... (I think :x)
> 
> I believe I have the config side of the work done.  I haven't
> figured out how to add this to public-inbox-mda/public-inbox-watch.
> 
> Let me send out what I have and then you can work on the bits
> for public-inbox-watch public-inbox-mda.

Thanks, will do.

> Last round I was messing with this I almost had my imap fetcher in
> shape. I may try again but let's get the listid thing settled first
> if we can.

Alright.

Btw, would using libcurl for IMAP support be easier for you?

I'm considering introducing libcurl support via Inline::C (it
might be easier to hook into the event loop for other things).

I'm also thinking about distributing some C via *.h header files
so it's easier to sparse/lint the code w/o it being embedded
inside a Perl file (I'd send those *.h files to Inline::C for
production use).



Using *.h since MakeMaker will automatically assume any *.c
files will be made into an XS extension, and I'm not sure how to
workaround that... (using GNU make w/o MakeMaker is an option)

Whether it's Perl or C; I want to keep everything on end-user
systems in source form, so they can just hack/update the source
if needed instead of having to find/download/build it to
experiment.

^ permalink raw reply	[relevance 0%]

* Re: Do I need multiple publicinbox.<name>.address values?
  2019-10-08 22:11  0%     ` Eric Wong
@ 2019-10-08 22:24  0%       ` Eric W. Biederman
  0 siblings, 0 replies; 49+ results
From: Eric W. Biederman @ 2019-10-08 22:24 UTC (permalink / raw)
  To: Eric Wong; +Cc: Alyssa Ross, meta

Eric Wong <e@80x24.org> writes:

> "Eric W. Biederman" <ebiederm@xmission.com> wrote:
>> Eric Wong <e@80x24.org> writes:
>> 
>> > Alyssa Ross <hi@alyssa.is> wrote:
>> >
>> >> Subject: Do I need multiple publicinbox.<name>.address values?
>> >
>> > Absolutely not
>> >
>> >> Suppose I have a mailing list, foo-discuss@example.org, and a
>> >> public-inbox set up, subscribed to that mailing list, that is subscribed
>> >> to that list as public-inbox+foo-discuss@example.org, where my MTA
>> >> delivers to public-inbox using public-inbox-mda.
>> >
>> > Currently, -mda does if you're mirroring, unfortunately.  I
>> > think Eric Biederman is/was working on List-Id support to drop
>> > that requirement, but I'm not sure where that is...
>> >
>> > Eric B: would you mind if I take List-Id support over?  I've got
>> > some hours free in the coming days(s)... (I think :x)
>> 
>> I believe I have the config side of the work done.  I haven't
>> figured out how to add this to public-inbox-mda/public-inbox-watch.
>> 
>> Let me send out what I have and then you can work on the bits
>> for public-inbox-watch public-inbox-mda.
>
> Thanks, will do.
>
>> Last round I was messing with this I almost had my imap fetcher in
>> shape. I may try again but let's get the listid thing settled first
>> if we can.
>
> Alright.
>
> Btw, would using libcurl for IMAP support be easier for you?

Right now I think I just need to make certain all of my prereqs are
merged and push the code out.  I just don't have as much time to work
on these things as I am used to so it is taking me forever to get
anything done.

For what it is worth below is my imap import script below.

The hardest part has been making certain I get the error handling
correct when unexpected errors happen.  Because errors do happen.

> I'm considering introducing libcurl support via Inline::C (it
> might be easier to hook into the event loop for other things).
>
> I'm also thinking about distributing some C via *.h header files
> so it's easier to sparse/lint the code w/o it being embedded
> inside a Perl file (I'd send those *.h files to Inline::C for
> production use).
>
>
>
> Using *.h since MakeMaker will automatically assume any *.c
> files will be made into an XS extension, and I'm not sure how to
> workaround that... (using GNU make w/o MakeMaker is an option)
>
> Whether it's Perl or C; I want to keep everything on end-user
> systems in source form, so they can just hack/update the source
> if needed instead of having to find/download/build it to
> experiment.


#<--- scripts/import_imap_mailbox ---->
#!/usr/bin/perl -w
# Script to import a IMAP mailbox into a public-inbox
=begin usage
	./import_imap_mailbox imap://username@hostname
=cut
use strict;
use warnings;
use Getopt::Long qw(:config gnu_getopt no_ignore_case auto_abbrev);
use Mail::IMAPClient;
use IO::Socket;
use IO::Socket::SSL;
use Data::Dumper;
use Email::Simple;
use Email::MIME;
use File::Basename;
use File::Sync qw(sync);
use Term::ReadKey;
use PublicInbox::Config;
use PublicInbox::Address;
use PublicInbox::IMAPTracker;
use PublicInbox::InboxWritable;
use POSIX qw(strftime);
sub usage { "Usage:\n".join('', grep(/\t/, `head -n 24 $0`)) }
my $push_origin = 1;
my $sanitize = 1;
my %opts = (
	'--push-origin!' => \$push_origin,
	'--sanitize!' => \$sanitize,
);
GetOptions(%opts) or die usage();

my $mail_url = shift @ARGV or die usage();
my $mail_hostname;
my $mail_domainname;
my $mail_username;
my $mail_password;
if ($mail_url =~ m$\Aimap://([^@]+)[@]([^@]+)(/|)\z$) {
	$mail_username = $1;
	$mail_hostname = $2;
	if ($mail_hostname eq "mail.xmission.com") {
		$mail_domainname = "xmission.com";
	}
} else {
	die usage();
}

chomp(my $committer_name = `git config user.name`);
chomp(my $committer_email = `git config user.email`);
my $committer = "$committer_name <$committer_email>";

my $mail_addr     = $mail_username . '@' . $mail_domainname;
my $url_base = 'imap://' . $mail_username . '@' . $mail_hostname . '/' ;

sub list_hdr_ibx($$)
{
	my ($config, $list_hdr) = @_;
	my $list_id;
	if ($list_hdr =~ m/\0/) {
		warn("Bad List-ID: $list_hdr contains a null\n");
		return undef;
	} elsif ($list_hdr =~ m/\A[^<>]*<(\S*)>\s*\z/) {
		$list_id = $1;
	} else {
		warn("Bad List-ID: $list_hdr\n");
		return undef;
	}
	my $ibx = $config->lookup_list_id($list_id);
	if (!defined($ibx)) {
		warn("Cound not find inbox for List-ID: $list_id\n");
	}

	print(" List-ID: $list_id\n");
	$ibx;
}

sub deliveredto_ibx($$)
{
	my ($config, $deliveredto) = @_;
	my ($email) = PublicInbox::Address::emails($deliveredto);
	if (!defined($email)) {
		warn("No email in Delivered-To: $deliveredto\n");
		return undef;
	}
	my $ibx = $config->lookup($email);
	if (!defined($ibx)) {
		warn("Cound not find inbox for deliveredto: $email\n");
		return undef;
	}

	print(" deliveredto: $deliveredto\n");
	$ibx;
}

sub mbox_default_ibx($$) {
	my ($config, $mailbox) = @_;
	my $addr = $mail_username . '+' . $mailbox . '@' . $mail_domainname;
	if ($mailbox eq 'INBOX') {
		$addr = $mail_addr;
	}
	my $ibx = $config->lookup($addr);
	if (defined($ibx) and !defined($ibx->{mainrepo})) {
		$ibx = undef;
	}
	$ibx;
}

sub ibx_gitdir($)
{
	my ($ibx) = @_;
	my $repo = $ibx->{mainrepo};
	die("Inbox without mainrepo") unless defined($repo);
	my $git_dir = $repo;
	if (-d "$repo/git") {
		my $last = 0;
		opendir(my $dh, "$repo/git") || die("Can not open git dir $repo/git/\n");
		while(my $name = readdir($dh)) {
			if ($name =~ m/^([0-9]+).git$/) {
				if ($last < $1) {
					$last = $1;
				}
			}
		}
		closedir($dh);
		$git_dir = "$repo/git/$last.git";
	}
	return $git_dir;
}

sub email_dest($$)
{
	my ($config, $mime) = @_;
	my %ibxs;
	my $hdr = $mime->header_obj;
	my @list_hdrs = $hdr->header_raw('List-ID');
	for my $list_hdr (@list_hdrs) {
		my $ibx = list_hdr_ibx($config, $list_hdr);
		if (defined($ibx)) {
			$ibxs{$ibx->{mainrepo}} = $ibx;
		}
	}
	my @DeliveredTo = $hdr->header_raw('Delivered-To');
	for my $deliveredto (@DeliveredTo) {
		my $ibx = deliveredto_ibx($config, $deliveredto);
		if (defined($ibx)) {
			$ibxs{$ibx->{mainrepo}} = $ibx;
		}
	}
	my @ibxs = values %ibxs;
	return @ibxs;
}

sub index_inbox ($)
{
	my ($ibx) = @_;
	my $repo = $ibx->{mainrepo};
	my $pid = fork();
	if (!defined($pid)) {
		warn("public-inbox-index $repo failed to start\n");
		return;
	}
	if ($pid != 0) {
		# Wait for the process so zombies don't accumulate
		waitpid($pid, 0);
		return;
	}
	# child
	exec( 'public-inbox-index', $repo);
	warn("exec public-inbox-index $repo failed\n");
	exit(1);
}

sub imap_sleep ()
{
	my $sleep = 5;
	print "\nSleeping for $sleep minutes\n\n";
	sleep($sleep*60);
}

sub fsck_inbox ($)
{
	my ($git_dir) = @_;
	if (system(('git', '--git-dir', $git_dir, 'fsck')) != 0) {
		die "git fsck failed: $?";
	}
}

sub push_inbox ($)
{
	my ($git_dir) = @_;
	if (!$push_origin) {
		return;
	}
	print("pushing $git_dir...\n");
	if (system(('git', '--git-dir', $git_dir, 'push', 'origin')) != 0) {
		die "git push failed: $?";
	}
}

print("Enter your imap password: ");
ReadMode('noecho');
$mail_password = ReadLine(0);
ReadMode('normal');
chomp($mail_password);
print("\n");

sub fetch_mailbox ($$$$)
{
	my ($config, $tracker, $client, $mailbox) = @_;
	my $now = time();
	print("mailbox: $mailbox @ " .
	      strftime("%Y-%m-%d %H:%M:%S %z", localtime(time()))
	      . "\n");

	my $default_ibx = mbox_default_ibx($config, $mailbox);

	if (!defined($default_ibx)) {
		print("skipping $mailbox no default inbox\n");
		return 0;
	}

	my %importers;
	my $url = $url_base  . $mailbox;
	my ($last_validity, $last_uid) = $tracker->get_last($url);

	$client->select($mailbox);
	$client->Peek(1);
	$client->Uid(1);
	my $validity = $client->uidvalidity($mailbox) or die("No uid validity");
	if (defined($last_validity) and ($validity ne $last_validity)) {
		die ("Unexpected uid validity $validity expected $last_validity");
	}

	my $search_str="ALL";
	if (defined($last_uid)) {
		# Find the last seen and all higher articles
		$search_str = "UID $last_uid:*";
	}
	my $uids = $client->search($search_str);
	if (!defined($uids) || (scalar(@$uids) == 0)) {
		print("No uids found! $@\n");
		return 0;
	}

	my $last = undef;
	my @sorted_uids = sort { $a <=> $b } @$uids;
	# Cap the number of uids to process at once
	my $more = 0;
	my $uid_count = scalar(@sorted_uids);
	if ($uid_count > 100) {
		@sorted_uids = @sorted_uids[0..99];
		$more = $uid_count - 100;
	}
	for my $uid (@sorted_uids) {
		print("UID: $validity $uid\n");
		if (defined($last_uid)) {
			if ($uid == $last_uid) {
				next;
			}
			if ($uid < $last_uid) {
				print("uid $uid not below last $last_uid, skipping.\n");
				next;
			}
		}
		my $email_str = $client->message_string($uid) or die "Could not message_string $@\n";
		my $email_len = length($email_str);
		my $mime = Email::MIME->new($email_str);
		$mime->{-public_inbox_raw} = $email_str;

		my @dests = email_dest($config, $mime);
		if (scalar(@dests) == 0) {
			push(@dests, $default_ibx);
		}
		die ("no destination for the email") unless scalar(@dests) > 0;
		printf(" dests: %d\n", scalar(@dests));
		for my $ibx (@dests) {
			my $git_dir = ibx_gitdir($ibx);
			print " git_dir: $git_dir\n";
			my $im = $importers{$git_dir}->[0];
			if (!defined($im)) {
				$ibx = PublicInbox::InboxWritable->new($ibx);
				$im = $ibx->importer(1);
				die "no im" unless defined($im);
				my @arr = ( $im, $ibx );
				$importers{$git_dir} = \@arr;
			}
			if (defined($im->{mm}->{num_highwater})) {
				print "Last: $git_dir: " . $im->{mm}->{num_highwater} . "\n";
			} else {
				print "Last: $git_dir: 0\n";
			}
			$im->add($mime);
			print "This: $git_dir: " . $im->{mm}->{num_highwater} . "\n";
		}
		#$client->delete_message($uid);
		$last = $uid;
	}

	if ($last) {
		die ("no git_dirs for $url") unless scalar(keys %importers) > 0;
		for my $git_dir (keys %importers) {
			my $ref = delete $importers{$git_dir};
			my ($im, $ibx) = @$ref;
			$im->done();
			push_inbox($git_dir);
		}
		print("updating tracker for $url...\n");
		$tracker->update_last($url, $validity, $last);

		#$client->delete_message($uids) or die ("Could not delete messages");
	}

	$client->close($mailbox);
	return $more;
}

sub fetch_mail ()
{
	my $config = eval { PublicInbox::Config->new };
	die("No public inbox config found!") unless $config;

	my $tracker = PublicInbox::IMAPTracker->new();

	my $socket = IO::Socket::SSL->new(
		PeerAddr => $mail_hostname,
		PeerPort => 993,
		Timeout  => 5,

		SSL_verify_mode => SSL_VERIFY_PEER,
		IO::Socket::SSL::default_ca(),
	);
	if (!defined($socket)) {
		die("Could not open socket to mailserver: $@\n");
	}

	my $client = Mail::IMAPClient->new(
		Socket   => $socket,
		User     => $mail_username,
		Password => $mail_password,
		Timeout  => 5,
	    );
	if (!defined($client)) {
		die("Could not initialize imap client $@\n");
	}
	if (!$client->IsAuthenticated()) {
		die("Could not authenticate against IMAP: $@\n");
	}
	if (!$client->IsConnected()) {
		die("Could not connect to the IMAP server: $@\n");
	}

	my $mailboxes = $client->folders();
	my @sorted_mailboxes = sort { $a  cmp $b } @$mailboxes;

	my $more;
	do {
		$more = 0;
		for my $mailbox (@sorted_mailboxes) {
			$more += fetch_mailbox($config, $tracker, $client, $mailbox);
		}
	} while ($more > 0);

	$client->logout();
}

sub relevant_inbox
{
	my ($ibx) = @_;
	# Verify the mailbox is one that would come from the server
	my $lc_user = lc($mail_username);
	my $lc_domain = lc($mail_domainname);
	foreach (@{$ibx->{address}}) {
		my $lc_addr = lc($_);
		if ($lc_addr =~ m/${lc_user}[+][^@]+[@]${lc_domain}/) {
			return 1;
		}
	}
	return 0;
}

sub sanitize_inboxes ()
{
	my $config = eval { PublicInbox::Config->new };
	die("No public inbox config found!") unless $config;

	$config->each_inbox(
		sub {
			my ($ibx) = @_;

			return unless relevant_inbox($ibx);

			print $ibx->{name} . "\n";
			my $git_dir = ibx_gitdir($ibx);
			fsck_inbox($git_dir);
			eval { push_inbox($git_dir); };
			index_inbox($ibx);
		}
	);
}

if ($sanitize) {
	sanitize_inboxes();
	sync();
}

for (;;imap_sleep()) {
	# Run fetch_mail in it's own separate process so
	# that if something goes wrong the process exits
	# and everything cleans up properly.
	#
	# Running fetch_mail in an eval block is not enough
	# to prevent leaks of locks and other resources.
	my $pid = fork();
	if (!defined($pid)) {
		warn("fork failed:\n");
		continue;
	}
	elsif ($pid == 0) {
		# in the child
		fetch_mail();
		exit(0);
	}
	else {
		warn("-------------------- CHILD: $pid --------------------\n");
		waitpid($pid, 0);
		sync();
	}
}

1;

^ permalink raw reply	[relevance 0%]

* Re: Do I need multiple publicinbox.<name>.address values?
  2019-10-08  0:10  4% ` Eric Wong
  2019-10-08 12:18  0%   ` Eric W. Biederman
@ 2019-10-09 11:59  6%   ` Alyssa Ross
  2019-10-10 10:06  4%     ` Eric Wong
  1 sibling, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-09 11:59 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]

> Sorry for that confusion.  Historically, I wrote public-inbox
> and -mda because I needed to migrate mailing lists with a
> several month window where the old list would be active,
> but I was preparing for the switch:
>
> So it started as:
>
>   posters -> to_be_shutdown_host -> MTA (postfix) -> public-inbox-mda
>
> And ends up being:
>
>   posters -> MTA (postfix) -> public-inbox-mda --[1]--> (mlmmj|mailman)
>
>
> It sounds like what you're doing is:
>
>   MTA -> (mlmmj|mailman) -> public-inbox-mda
>
> Which wasn't my original intended usecase for -mda, but is for -watch.

Interesting!  The reason I set it up this way is that mailman (at least
v3) doesn't archive raw incoming mail -- the default archiver,
Hyperkitty, pulls out the body and some headers it cares about, saves
them into a database, and discards the rest.  Which IMO is not great for
an archiver (although I do like Hyperkitty as a web interface for people
who haven't yet seen the light of mail).  So my plan is to use
public-inbox as my primary archiver, and just use Hyperkitty as a
"modern" web interface.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: public-inbox-init with minimal info
  2019-10-08  7:11  4%                 ` Eric Wong
@ 2019-10-09 12:09  6%                   ` Alyssa Ross
  2019-10-10  8:19  4%                     ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-09 12:09 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 1683 bytes --]

>> >>     services.public-inbox.http.mount = "/lists/archives/";
>> >
>> > I think all the services would want access to the same
>> > directories, not just httpd (if I'm understanding that config
>> > correctly).  Also, httpd/nntpd only need read-only access to their
>> > mount points, in case that affects things...
>> 
>> "mount" here is in the PSGI sense, not the file system sense.  My
>> public-inboxes are at https://example.org/lists/archives/.  Maybe
>> there's a better name.
>
> Ah, so that overrides the Plack::Builder DSL/language.  We also
> have an analogous support problem for PSGI vs public-inbox-config,
> so I've been avoiding any overlap between them.
>
> Perhaps "psgi_mount" would be clearer? *shrug*

I shied away from that because it would only be clearer if you know what
PSGI is, and the module takes care of all of that.  I also considered
httpMount, but since it's already in the http namespace that felt
redundant.

> Btw, would it be helpful if public-inbox provided a linter for
> its config own file?

Very much so!  That would let us lint config files at build time, and
fail the system build if they were invalid, meaning a system could
"never" have an invalid config file.  We already do this with nginx --
the linter doesn't catch everything, but it's wonderful when it catches
something that would otherwise have left you without a working web
server.

Here's an idea for a lint: I lost most of a day wondering what I had
done wrong, before realising that I was setting mainrepo to all.git,
rather than its parent directory.  The name "mainrepo" isn't great, IMO,
but a lint could have accomodated for that.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[relevance 6%]

* [PATCH] doc: explain publicinbox.<name>.watchheader
@ 2019-10-09 13:21 13% Alyssa Ross
  2019-10-10  8:51  4% ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Alyssa Ross @ 2019-10-09 13:21 UTC (permalink / raw)
  To: meta; +Cc: Alyssa Ross

It wasn't clear to me exactly what this does -- in particular, what
happens if it isn't specified?  Does it support multiple values?  A
very brief explanation can answer both of these questions without
making somebody look at the code.
---
The part about it not supporting multiple values is my best guess
based on a quick glance at the code -- I may well have missed
something.

 Documentation/public-inbox-config.pod | 3 +++
 Documentation/public-inbox-watch.pod  | 4 ++++
 2 files changed, 7 insertions(+)

diff --git a/Documentation/public-inbox-config.pod b/Documentation/public-inbox-config.pod
index 4e8ba4d..8d545f7 100644
--- a/Documentation/public-inbox-config.pod
+++ b/Documentation/public-inbox-config.pod
@@ -80,6 +80,9 @@ Default: none; only for L<public-inbox-watch(1)> users
 	[publicinbox "test"]
 		watchheader = List-Id:<test.example.com>
 
+If specified, L<public-inbox-watch(1)> will only process mail matching
+the given header.  Multiple values are not currently supported.
+
 Default: none; only for L<public-inbox-watch(1)> users
 
 =item publicinbox.<name>.nntpmirror
diff --git a/Documentation/public-inbox-watch.pod b/Documentation/public-inbox-watch.pod
index 3f282f0..198c170 100644
--- a/Documentation/public-inbox-watch.pod
+++ b/Documentation/public-inbox-watch.pod
@@ -15,7 +15,11 @@ In ~/.public-inbox/config:
 		mainrepo = /path/to/test.example.com.git
 
 		; config keys specific to public-inbox-watch:
+
 		watch = maildir:/path/to/maildirs/.INBOX.test/
+
+		; optional, emails that don't have a header matching
+		; value will be skipped
 		watchheader = List-Id:<test.example.com>
 
 	[publicinboxwatch]
-- 
2.23.0


^ permalink raw reply related	[relevance 13%]

* Re: public-inbox-init with minimal info
  2019-10-09 12:09  6%                   ` Alyssa Ross
@ 2019-10-10  8:19  4%                     ` Eric Wong
  2019-10-16 10:04  0%                       ` Eric Wong
  0 siblings, 1 reply; 49+ results
From: Eric Wong @ 2019-10-10  8:19 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> >> >>     services.public-inbox.http.mount = "/lists/archives/";
> >> >
> >> > I think all the services would want access to the same
> >> > directories, not just httpd (if I'm understanding that config
> >> > correctly).  Also, httpd/nntpd only need read-only access to their
> >> > mount points, in case that affects things...
> >> 
> >> "mount" here is in the PSGI sense, not the file system sense.  My
> >> public-inboxes are at https://example.org/lists/archives/.  Maybe
> >> there's a better name.
> >
> > Ah, so that overrides the Plack::Builder DSL/language.  We also
> > have an analogous support problem for PSGI vs public-inbox-config,
> > so I've been avoiding any overlap between them.
> >
> > Perhaps "psgi_mount" would be clearer? *shrug*
> 
> I shied away from that because it would only be clearer if you know what
> PSGI is, and the module takes care of all of that.  I also considered
> httpMount, but since it's already in the http namespace that felt
> redundant.

Maybe "url_mount"?  Naming is one of the toughest problems :<

> > Btw, would it be helpful if public-inbox provided a linter for
> > its config own file?
> 
> Very much so!  That would let us lint config files at build time, and
> fail the system build if they were invalid, meaning a system could
> "never" have an invalid config file.  We already do this with nginx --
> the linter doesn't catch everything, but it's wonderful when it catches
> something that would otherwise have left you without a working web
> server.

OK, adding TODO item.

> Here's an idea for a lint: I lost most of a day wondering what I had
> done wrong, before realising that I was setting mainrepo to all.git,
> rather than its parent directory.  The name "mainrepo" isn't great, IMO,
> but a lint could have accomodated for that.

Sorry about wasting your time!  Yeah, "mainrepo" is a bad name,
especially with v2 :<

I guess we can shift to "inboxdir" and keep the old alias
indefinitely for compatibility, since there's already INBOX_DIR
all over the documentation.

I originally intended for each inbox to have another repo
for rejected/spam messages; but just put it in PI_EMERGENCY,
instead.

^ permalink raw reply	[relevance 4%]

* Re: [PATCH] doc: explain publicinbox.<name>.watchheader
  2019-10-09 13:21 13% [PATCH] doc: explain publicinbox.<name>.watchheader Alyssa Ross
@ 2019-10-10  8:51  4% ` Eric Wong
  0 siblings, 0 replies; 49+ results
From: Eric Wong @ 2019-10-10  8:51 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> It wasn't clear to me exactly what this does -- in particular, what
> happens if it isn't specified?  Does it support multiple values?  A
> very brief explanation can answer both of these questions without
> making somebody look at the code.

Thanks, applied and pushed.

> ---
> The part about it not supporting multiple values is my best guess
> based on a quick glance at the code -- I may well have missed
> something.

That's correct.  I think a single value here is fine.
If it supported multiple values, it would be confusing as
to whether it needs to match all the headers, or if matching
any one header would put it into an inbox.

And I also question the need for either case...

^ permalink raw reply	[relevance 4%]

* Re: Do I need multiple publicinbox.<name>.address values?
  2019-10-09 11:59  6%   ` Alyssa Ross
@ 2019-10-10 10:06  4%     ` Eric Wong
  0 siblings, 0 replies; 49+ results
From: Eric Wong @ 2019-10-10 10:06 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Alyssa Ross <hi@alyssa.is> wrote:
> > Sorry for that confusion.  Historically, I wrote public-inbox
> > and -mda because I needed to migrate mailing lists with a
> > several month window where the old list would be active,
> > but I was preparing for the switch:
> >
> > So it started as:
> >
> >   posters -> to_be_shutdown_host -> MTA (postfix) -> public-inbox-mda
> >
> > And ends up being:
> >
> >   posters -> MTA (postfix) -> public-inbox-mda --[1]--> (mlmmj|mailman)
> >
> >
> > It sounds like what you're doing is:
> >
> >   MTA -> (mlmmj|mailman) -> public-inbox-mda
> >
> > Which wasn't my original intended usecase for -mda, but is for -watch.
> 
> Interesting!  The reason I set it up this way is that mailman (at least
> v3) doesn't archive raw incoming mail -- the default archiver,
> Hyperkitty, pulls out the body and some headers it cares about, saves
> them into a database, and discards the rest.  Which IMO is not great for
> an archiver (although I do like Hyperkitty as a web interface for people
> who haven't yet seen the light of mail).  So my plan is to use
> public-inbox as my primary archiver, and just use Hyperkitty as a
> "modern" web interface.

Yeah, right now you could have Mailman deliver to an address ->
Maildir which public-inbox-watch runs on.  It's a bit roundabout
and you'd need to age out old messages from the Maildir.

Or you could write an NNTP -> pipe-to-arbitrary command script
which can pipe to Mailman or mlmmj, keeping track of which
messages have already been sent (or wait for me to write it).

I've been meaning to replace scripts/ssoma-replay (for mlmmj)
with that script at some point.  I've never actually run
Mailman, but I assume it allows raw messages to be piped to it
like most mail tools.

^ permalink raw reply	[relevance 4%]

* [PATCH 2/1] doc: v2-format: get man output under 80 cols
  2019-10-07 21:13  6%   ` Alyssa Ross
@ 2019-10-15 20:11  4%     ` Eric Wong
  0 siblings, 0 replies; 49+ results
From: Eric Wong @ 2019-10-15 20:11 UTC (permalink / raw)
  To: meta; +Cc: Alyssa Ross

Alyssa Ross <hi@alyssa.is> wrote:
> > I noticed both the original and post-patch manpage exceeds the
> > limits of an 80-column terminal.  I think it's worth it to
> > reflow and perhaps reword to ensure folks who are limited to 80
> > columns can viwe (but I can take care of it, too, if you're busy)
> >
> > Even fitting 80 columns on my screen is a challenge with the
> > giant fonts I need to workaround poor eyesight :<
> 
> I appreciate your commitment!  I think you'd be a better judge of how
> to format it nicely than me. :)

I'll apply this on top of your patch since there's wording
changes, too:

------8<----
Subject: [PATCH] doc: v2-format: get man output under 80 cols

We need to better ensure our manpage output is readable with a
standard terminal width.  And fix some wording while we're at
it:

* use "inbox" instead of "list" for our storage
* replace the last "$PART" reference with "$SHARD"
---
 Documentation/public-inbox-v2-format.pod | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/Documentation/public-inbox-v2-format.pod b/Documentation/public-inbox-v2-format.pod
index 58e6b6d3..10c63090 100644
--- a/Documentation/public-inbox-v2-format.pod
+++ b/Documentation/public-inbox-v2-format.pod
@@ -20,17 +20,17 @@ databases for parallelism by "shards".
 
 =head2 INBOX OVERVIEW AND DEFINITIONS
 
-$EPOCH - Integer starting with 0 based on time
-$SCHEMA_VERSION - PublicInbox::Search::SCHEMA_VERSION used by Xapian
-$PART - Integer (0..NPROCESSORS)
-
-	foo/                              # assuming "foo" is the name of the list
-	- inbox.lock                      # lock file (flock) to protect global state
-	- git/$EPOCH.git                  # normal git repositories
-	- all.git                         # empty git repo, alternates to git/$EPOCH.git
-	- xap$SCHEMA_VERSION/$SHARD       # per-shard Xapian DB
-	- xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP and threading
-	- msgmap.sqlite3                  # same the v1 msgmap
+  $EPOCH - Integer starting with 0 based on time
+  $SCHEMA_VERSION - DB schema version (for Xapian)
+  $SHARD - Integer starting with 0 based on parallelism
+
+  foo/                              # "foo" is the name of the inbox
+  - inbox.lock                      # lock file to protect global state
+  - git/$EPOCH.git                  # normal git repositories
+  - all.git                         # empty, alternates to $EPOCH.git
+  - xap$SCHEMA_VERSION/$SHARD       # per-shard Xapian DB
+  - xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP, threading
+  - msgmap.sqlite3                  # same the v1 msgmap
 
 For blob lookups, the reader only needs to open the "all.git"
 repository with $GIT_DIR/objects/info/alternates which references

^ permalink raw reply related	[relevance 4%]

* Re: public-inbox-init with minimal info
  2019-10-10  8:19  4%                     ` Eric Wong
@ 2019-10-16 10:04  0%                       ` Eric Wong
  0 siblings, 0 replies; 49+ results
From: Eric Wong @ 2019-10-16 10:04 UTC (permalink / raw)
  To: Alyssa Ross; +Cc: meta

Eric Wong <e@80x24.org> wrote:
> Alyssa Ross <hi@alyssa.is> wrote:
> > Here's an idea for a lint: I lost most of a day wondering what I had
> > done wrong, before realising that I was setting mainrepo to all.git,
> > rather than its parent directory.  The name "mainrepo" isn't great, IMO,
> > but a lint could have accomodated for that.
> 
> Sorry about wasting your time!  Yeah, "mainrepo" is a bad name,
> especially with v2 :<
> 
> I guess we can shift to "inboxdir" and keep the old alias
> indefinitely for compatibility, since there's already INBOX_DIR
> all over the documentation.

Patches posted at:
  https://public-inbox.org/meta/20191016085955.23674-1-e@80x24.org/

(and I needed a 3/2 fixup after deploying :x)
Will merge to master soonish.  I think it's the last major thing
to do before 1.2.0 (and some more doc updates coming...)

^ permalink raw reply	[relevance 0%]

* [PATCH] t/hl_mod: accept "make" or "makefile" for Makefile
@ 2021-06-22 12:09 14% Alyssa Ross
  0 siblings, 0 replies; 49+ results
From: Alyssa Ross @ 2021-06-22 12:09 UTC (permalink / raw)
  To: meta; +Cc: Alyssa Ross

Version 4.0 of highlight has renamed the "make" language to
"makefile".  So just check the string starts with "make", to handle
both 3.x and 4.x.

I tested that public-inbox does actually work with highlight 4 -- it
can highlight my Makefile fine. :)
---
 t/hl_mod.t | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/hl_mod.t b/t/hl_mod.t
index 95057354..e28f26b5 100644
--- a/t/hl_mod.t
+++ b/t/hl_mod.t
@@ -16,7 +16,7 @@ ok($hls, 'initialized OK');
 is($hls->_shebang2lang(\"#!/usr/bin/perl -w\n"), 'perl', 'perl shebang OK');
 is($hls->{-ext2lang}->{'pm'}, 'perl', '.pm suffix OK');
 is($hls->{-ext2lang}->{'pl'}, 'perl', '.pl suffix OK');
-is($hls->_path2lang('Makefile'), 'make', 'Makefile OK');
+like($hls->_path2lang('Makefile'), qr/\Amake/, 'Makefile OK');
 my $str = do { local $/; open(my $fh, __FILE__); <$fh> };
 my $orig = $str;
 
-- 
2.31.1


^ permalink raw reply related	[relevance 14%]

Results 1-49 of 49 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-09-15 13:48 14% [PATCH] t/spawn: fix with unusual env locations Alyssa Ross
2019-09-15 18:55  4% ` Eric Wong
2019-09-24  1:11  6%   ` Alyssa Ross
2019-09-24  4:01  4%     ` Eric Wong
2019-09-24 22:12  5%       ` Test failures in build sandbox Alyssa Ross
2019-09-26  8:44  4%         ` Eric Wong
2019-10-02 23:23  6%           ` Alyssa Ross
2019-10-03  0:46 11%             ` Alyssa Ross
2019-10-03  1:28  4%               ` Eric Wong
2019-10-03  1:43  6%                 ` Alyssa Ross
2019-10-03  7:57  5%                   ` [PATCH] t/search: bail out on `git init --shared' failures Eric Wong
2019-10-03 10:15 12%                     ` Alyssa Ross
2019-09-24 20:05 14% [PATCH] hlmod: update for highlight 3.51 API change Alyssa Ross
2019-09-26  1:59  4% ` Eric Wong
2019-09-26 13:18       ` Konstantin Ryabitsev
2019-09-26 15:07  6%     ` Alyssa Ross
2019-09-26 15:08  6%   ` Alyssa Ross
2019-09-24 20:43  6% [PATCH] #!/usr/bin/perl -> #!/usr/bin/env perl Alyssa Ross
2019-09-26  2:49  3% ` Eric Wong
2019-10-01 17:47  6%   ` Alyssa Ross
2019-10-01 20:58  4%     ` Eric Wong
2019-10-02 21:36 13%       ` [PATCH] tests: recommend running create-certs.pl with $^X Alyssa Ross
2019-10-02 23:19  4%         ` Eric Wong
2019-10-03 11:16  6% public-inbox-init with minimal info Alyssa Ross
2019-10-04  2:45  4% ` Eric Wong
2019-10-04 11:18  6%   ` Alyssa Ross
2019-10-05  5:14  3%     ` Eric Wong
2019-10-05 13:05  6%       ` Alyssa Ross
2019-10-05 19:58  4%         ` Eric Wong
2019-10-06  9:52  6%           ` Alyssa Ross
2019-10-06 12:01  4%             ` Eric Wong
2019-10-07 20:52  5%               ` Alyssa Ross
2019-10-08  7:11  4%                 ` Eric Wong
2019-10-09 12:09  6%                   ` Alyssa Ross
2019-10-10  8:19  4%                     ` Eric Wong
2019-10-16 10:04  0%                       ` Eric Wong
2019-10-07 20:27 13% [PATCH] public-inbox-v2-format(5): fix formatting Alyssa Ross
2019-10-07 20:52  4% ` Eric Wong
2019-10-07 21:13  6%   ` Alyssa Ross
2019-10-15 20:11  4%     ` [PATCH 2/1] doc: v2-format: get man output under 80 cols Eric Wong
2019-10-07 22:13  6% Do I need multiple publicinbox.<name>.address values? Alyssa Ross
2019-10-08  0:10  4% ` Eric Wong
2019-10-08 12:18  0%   ` Eric W. Biederman
2019-10-08 22:11  0%     ` Eric Wong
2019-10-08 22:24  0%       ` Eric W. Biederman
2019-10-09 11:59  6%   ` Alyssa Ross
2019-10-10 10:06  4%     ` Eric Wong
2019-10-09 13:21 13% [PATCH] doc: explain publicinbox.<name>.watchheader Alyssa Ross
2019-10-10  8:51  4% ` Eric Wong
2021-06-22 12:09 14% [PATCH] t/hl_mod: accept "make" or "makefile" for Makefile Alyssa Ross

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).