user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
* Bug related to (maybe?) / in Message-Id
@ 2023-02-16 21:05 Uwe Kleine-König
  2023-02-16 21:36 ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Uwe Kleine-König @ 2023-02-16 21:05 UTC (permalink / raw)
  To: meta

[-- Attachment #1: Type: text/plain, Size: 1053 bytes --]

Hello,

The mail by Alexander Dahl that is (currently) the first hit on
https://lore.ptxdist.org/ptxdist/?q=ptxd_make_world_compile_commands_filter
results in a 404 when I follow the link.

The original mail has

	Message-ID: <Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de>

and the corresponding link is:

	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de/

I noticed this on public-inbox 1.8.0-1~bpo11+1 from Debian, upgrading to
1.9.0-1~bpo11+1 didn't help.

Other mails with / in Message-Id are not accessible either, I tested
with:

	YyHu/412LT8uQTy1@lenoch
	Y0/5xdFZO3u0952+@lenoch

I also wonder why these mails yield the webserver's 404 page and not the
one provided by the public-inbox cgi?!

Is this a problem in public-inbox, or is the apache configuration
somehow borked? Any hints welcome.

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Bug related to (maybe?) / in Message-Id
  2023-02-16 21:05 Bug related to (maybe?) / in Message-Id Uwe Kleine-König
@ 2023-02-16 21:36 ` Eric Wong
  2023-02-17  8:52   ` Uwe Kleine-König
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2023-02-16 21:36 UTC (permalink / raw)
  To: Uwe Kleine-König; +Cc: meta

Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:
> Hello,
> 
> The mail by Alexander Dahl that is (currently) the first hit on
> https://lore.ptxdist.org/ptxdist/?q=ptxd_make_world_compile_commands_filter
> results in a 404 when I follow the link.
> 
> The original mail has
> 
> 	Message-ID: <Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de>
> 
> and the corresponding link is:
> 
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de/
> 
> I noticed this on public-inbox 1.8.0-1~bpo11+1 from Debian, upgrading to
> 1.9.0-1~bpo11+1 didn't help.
> 
> Other mails with / in Message-Id are not accessible either, I tested
> with:
> 
> 	YyHu/412LT8uQTy1@lenoch
> 	Y0/5xdFZO3u0952+@lenoch

The TODO file has this:

	* use REQUEST_URI properly for CGI / mod_perl2 compatibility
	  with Message-IDs which include '%' (done?)

So I guess it's not done...  To deal with '/' in the Message-ID,
$env->{REQUEST_URI} really needs to be the raw, undecoded URI
specified in the PSGI specs[1].

I'm not sure how to go about it Apache+CGI or mod_perl2..

Fwiw, the recommended configuration is:
(nginx|haproxy) -> varnish -> public-inbox-{httpd,netd}

Maybe Apache2 mpm_event reverse proxy can work in lieu of
(nginx|haproxy), but /T/, /t/, /t.mbox.gz requests are a bit
faster on -httpd/-netd since 1.6+ on SMP machines.

> I also wonder why these mails yield the webserver's 404 page and not the
> one provided by the public-inbox cgi?!

This may be the small size public-inbox's 404 page.  I don't
know Apache configs well, but I know nginx did something
similar.

> Is this a problem in public-inbox, or is the apache configuration
> somehow borked? Any hints welcome.

Do you have access to that server and can show us the configs?
REQUEST_URI really needs to be raw in accordance to PSGI specs.

This can dump the request $env to stderr and show us
REQUEST_URI, PATH_INFO, SCRIPT_NAME, and anything else
which may enlighten us:

diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm
index 9ffcb879..f67fe8e6 100644
--- a/lib/PublicInbox/WWW.pm
+++ b/lib/PublicInbox/WWW.pm
@@ -52,7 +52,8 @@ sub call {
 		# none of the keys we care about will need escaping
 		($k // '', uri_unescape($v // ''))
 	} split(/[&;]+/, $env->{QUERY_STRING});
-
+	use Data::Dumper; $Data::Dumper::Useqq = 1;
+	warn Dumper($env);
 	my $path_info = path_info_raw($env);
 	my $method = $env->{REQUEST_METHOD};
 

[1] PSGI specs: git clone https://github.com/plack/psgi-specs

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: Bug related to (maybe?) / in Message-Id
  2023-02-16 21:36 ` Eric Wong
@ 2023-02-17  8:52   ` Uwe Kleine-König
  2023-02-17 10:28     ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Uwe Kleine-König @ 2023-02-17  8:52 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

[-- Attachment #1: Type: text/plain, Size: 6047 bytes --]

Hello Eric,

first of all: Thanks for your quick answer.

On Thu, Feb 16, 2023 at 09:36:28PM +0000, Eric Wong wrote:
> Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:
> > Hello,
> > 
> > The mail by Alexander Dahl that is (currently) the first hit on
> > https://lore.ptxdist.org/ptxdist/?q=ptxd_make_world_compile_commands_filter
> > results in a 404 when I follow the link.
> > 
> > The original mail has
> > 
> > 	Message-ID: <Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de>
> > 
> > and the corresponding link is:
> > 
> > 	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de/
> > 
> > I noticed this on public-inbox 1.8.0-1~bpo11+1 from Debian, upgrading to
> > 1.9.0-1~bpo11+1 didn't help.
> > 
> > Other mails with / in Message-Id are not accessible either, I tested
> > with:
> > 
> > 	YyHu/412LT8uQTy1@lenoch
> > 	Y0/5xdFZO3u0952+@lenoch
> 
> The TODO file has this:
> 
> 	* use REQUEST_URI properly for CGI / mod_perl2 compatibility
> 	  with Message-IDs which include '%' (done?)
> 
> So I guess it's not done...  To deal with '/' in the Message-ID,
> $env->{REQUEST_URI} really needs to be the raw, undecoded URI
> specified in the PSGI specs[1].
> 
> I'm not sure how to go about it Apache+CGI or mod_perl2..
> 
> Fwiw, the recommended configuration is:
> (nginx|haproxy) -> varnish -> public-inbox-{httpd,netd}
> 
> Maybe Apache2 mpm_event reverse proxy can work in lieu of
> (nginx|haproxy), but /T/, /t/, /t.mbox.gz requests are a bit
> faster on -httpd/-netd since 1.6+ on SMP machines.
> 
> > I also wonder why these mails yield the webserver's 404 page and not the
> > one provided by the public-inbox cgi?!
> 
> This may be the small size public-inbox's 404 page.  I don't
> know Apache configs well, but I know nginx did something
> similar.

> > Is this a problem in public-inbox, or is the apache configuration
> > somehow borked? Any hints welcome.
> 
> Do you have access to that server and can show us the configs?
> REQUEST_URI really needs to be raw in accordance to PSGI specs.
> 
> This can dump the request $env to stderr and show us
> REQUEST_URI, PATH_INFO, SCRIPT_NAME, and anything else
> which may enlighten us:
> 
> diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm
> index 9ffcb879..f67fe8e6 100644
> --- a/lib/PublicInbox/WWW.pm
> +++ b/lib/PublicInbox/WWW.pm
> @@ -52,7 +52,8 @@ sub call {
>  		# none of the keys we care about will need escaping
>  		($k // '', uri_unescape($v // ''))
>  	} split(/[&;]+/, $env->{QUERY_STRING});
> -
> +	use Data::Dumper; $Data::Dumper::Useqq = 1;
> +	warn Dumper($env);
>  	my $path_info = path_info_raw($env);
>  	my $method = $env->{REQUEST_METHOD};

I added that patch and for the reported request this didn't trigger,
which I assume means that public-inbox isn't called at all.
 
Playing around with slashes got my admin and me on the right trail:
https://httpd.apache.org/docs/current/mod/core.html#allowencodedslashes

We set that to "On" and now it (mostly) works. Maybe it's worth adding
this hint to the documentation even though apache isn't the most
recommended setup? Maybe other servers have a similar security setting?

I wrote "mostly" because

	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de/
	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de
	https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/

work as expected;

	https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de

however does not, that yields a short "Not Found".

With the patch applied the logged stuff for these URLs is mostly
identical. REMOTE_PORT differs which is expected. Otherwise only
PATH_INFO, PATH_TRANSLATED and REQUEST_URI differ. They are
respectively:

	"PATH_INFO" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de/",
	"PATH_TRANSLATED" => "/usr/lib/cgi-bin/public-inbox.cgi/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de/",
	"REQUEST_URI" => "/ptxdist/Y+07h0l%2FzJJAgs9s\@falbala.internal.home.lespocky.de/",

	"PATH_INFO" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de",
	"PATH_TRANSLATED" => "/usr/lib/cgi-bin/public-inbox.cgi/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de",
	"REQUEST_URI" => "/ptxdist/Y+07h0l%2FzJJAgs9s\@falbala.internal.home.lespocky.de",

	"PATH_INFO" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de/",
	"PATH_TRANSLATED" => "/usr/lib/cgi-bin/public-inbox.cgi/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de/",
	"REQUEST_URI" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de/",

	"PATH_INFO" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de",
	"PATH_TRANSLATED" => "/usr/lib/cgi-bin/public-inbox.cgi/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de",
	"REQUEST_URI" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de",

which I think is all as expected. In all cases we have

	"SCRIPT_NAME" => "",

. Not sure making the last URL work is easily possible (and worth the
effort)? If a Message-Id ends in "/T" or similar the result will always
be ambigous?

One thing I just noticed is:

$ curl https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/T
Redirecting to https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/T

which makes Firefox say: "The page isn’t redirecting properly". It works
fine with the / replaced by %2F:

$ curl https://lore.ptxdist.org/ptxdist/Y+07h0l%2fzJJAgs9s@falbala.internal.home.lespocky.de/T
Redirecting to https://lore.ptxdist.org/ptxdist/Y+07h0l%2fzJJAgs9s@falbala.internal.home.lespocky.de/T/#u

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Bug related to (maybe?) / in Message-Id
  2023-02-17  8:52   ` Uwe Kleine-König
@ 2023-02-17 10:28     ` Eric Wong
  2023-02-17 10:32       ` [PATCH] TODO: handle more cases of unencoded slashes Eric Wong
                         ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Eric Wong @ 2023-02-17 10:28 UTC (permalink / raw)
  To: Uwe Kleine-König; +Cc: meta

Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:
> Hello Eric,
> 
> first of all: Thanks for your quick answer.

No problem.  I've been stuck on coderepo indexing stuff and needed the
distraction :<

> Playing around with slashes got my admin and me on the right trail:
> https://httpd.apache.org/docs/current/mod/core.html#allowencodedslashes
> 
> We set that to "On" and now it (mostly) works. Maybe it's worth adding
> this hint to the documentation even though apache isn't the most
> recommended setup? Maybe other servers have a similar security setting?

Thanks for figuring that out.  Perhaps a note in
Documentation/public-inbox.cgi.pod suffices (patch welcome :).

> I wrote "mostly" because
> 
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de/
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/
> 
> work as expected;
> 
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de
> 
> however does not, that yields a short "Not Found".

Most importantly, public-inbox itself only generates the first URL,
right? (`%2F' and trailing `/')

> With the patch applied the logged stuff for these URLs is mostly
> identical. REMOTE_PORT differs which is expected. Otherwise only
> PATH_INFO, PATH_TRANSLATED and REQUEST_URI differ. They are
> respectively:

Actually, only REQUEST_URI matters for mail retrieval URLs,
<snip irrelevant lines>

> 	"REQUEST_URI" => "/ptxdist/Y+07h0l%2FzJJAgs9s\@falbala.internal.home.lespocky.de/",
> 	"REQUEST_URI" => "/ptxdist/Y+07h0l%2FzJJAgs9s\@falbala.internal.home.lespocky.de",
> 	"REQUEST_URI" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de/",

I'm pleasantly surprised the 3rd case works, actually :>
I do hope we're not generating those URLs on our own.

> 	"REQUEST_URI" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de",

> . Not sure making the last URL work is easily possible (and worth the
> effort)? If a Message-Id ends in "/T" or similar the result will always
> be ambigous?

I think the last case is fixable and should be fixed.  It's
reasonable a user unfamiliar w/o URL encoding knowledge would
paste `/' w/o encoding it to `%2F', first

> One thing I just noticed is:
> 
> $ curl https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/T
> Redirecting to https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/T
> 
> which makes Firefox say: "The page isn’t redirecting properly". It works
> fine with the / replaced by %2F:
> 
> $ curl https://lore.ptxdist.org/ptxdist/Y+07h0l%2fzJJAgs9s@falbala.internal.home.lespocky.de/T
> Redirecting to https://lore.ptxdist.org/ptxdist/Y+07h0l%2fzJJAgs9s@falbala.internal.home.lespocky.de/T/#u

Yeah, that's another case that should work.

I don't really like the current giant if/elsif block
of route possibilities in lib/PublicInbox/WWW.pm but have
some ideas on how to improve it...  But a lot on my plate
already and way overdue for another release :<


It's a tragedy that mutt changed their default Message-ID format :<
Newer versions allow overriding $message_id_format in muttrc,
but mutt 2.0.5 in Debian 11 won't have it.

Fwiw, I run mutt on several machines/arches/OSes, getting them
all to the same new version of mutt will take many years and I
don't want compile + keep numerous binaries up-to-date in the
mean time.

I also use msmtp and distribute my muttrc + ~/bin to all of
them, so I wrote this msmtp wrapper which is mutt-version-agnostic:

$ cat $HOME/bin/msmtp-msgid
eval 'exec perl -w -S $0 ${1+"$@"}'
if 0; # running under some shell
# in muttrc:
# set sendmail = msmtp-msgid
use v5.12;
my ($hdr, $bdy) = split(/\n\n/, do { local $/; <STDIN> }, 2);
if ($hdr !~ /^Message-ID:\s*<\d{14}\b/aims) {
	my $h;
	if ($hdr =~ /^Message-ID:[^@]+\@([^>]+)>/ims) {
		$h = $1;
	} else {
		require Sys::Hostname;
		$h = Sys::Hostname::hostname();
	}
	require POSIX;
	require Time::HiRes;
	my ($time, $msec) = Time::HiRes::gettimeofday();
	my $msgid = POSIX::strftime("%Y%m%d%H%M%S.M$msec", gmtime($time));
	$hdr =~ s/^Message-ID:[^\n]+/Message-ID: <$msgid\@$h>/msi;
}
my $pid = open(my $fh, '|-', 'msmtp', @ARGV) // die "popen: $!";
print $fh $hdr, "\n\n", $bdy or die "print $!";
close $fh or die "msmtp \$?=$?";
__END__

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH] TODO: handle more cases of unencoded slashes
  2023-02-17 10:28     ` Eric Wong
@ 2023-02-17 10:32       ` Eric Wong
  2023-02-17 11:08       ` [PATCH] public-inbox.cgi(1): Mention AllowEncodedSlashes for Apache setups Uwe Kleine-König
  2023-02-18 17:58       ` Bug related to (maybe?) / in Message-Id Thomas Weißschuh
  2 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-02-17 10:32 UTC (permalink / raw)
  To: meta; +Cc: Uwe Kleine-König

Nowadays, mutt defaults to Message-IDs with `/' in them :<
---
 TODO | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/TODO b/TODO
index 1537179e..77453eba 100644
--- a/TODO
+++ b/TODO
@@ -75,6 +75,9 @@ all need to be considered for everything we introduce)
 * use REQUEST_URI properly for CGI / mod_perl2 compatibility
   with Message-IDs which include '%' (done?)
 
+* handle unencoded slashes from user-generated URLs properly
+  https://public-inbox.org/meta/20230217085255.xcsaoozloz2yuxil@pengutronix.de/
+
 * better test cases, make faster by reusing more setup
   code across tests
 

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH] public-inbox.cgi(1): Mention AllowEncodedSlashes for Apache setups
  2023-02-17 10:28     ` Eric Wong
  2023-02-17 10:32       ` [PATCH] TODO: handle more cases of unencoded slashes Eric Wong
@ 2023-02-17 11:08       ` Uwe Kleine-König
  2023-02-17 13:15         ` Eric Wong
  2023-02-18 17:58       ` Bug related to (maybe?) / in Message-Id Thomas Weißschuh
  2 siblings, 1 reply; 11+ messages in thread
From: Uwe Kleine-König @ 2023-02-17 11:08 UTC (permalink / raw)
  To: meta

When AllowEncodedSlashes is Off (the default setting), URLs containing
%2f are replied with a 404 error without calling the CGI. To (maybe)
prevent others debugging this issue add a hint with the solution.
---
 Documentation/public-inbox.cgi.pod | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/public-inbox.cgi.pod b/Documentation/public-inbox.cgi.pod
index 71f8a6f5f696..369813d09b91 100644
--- a/Documentation/public-inbox.cgi.pod
+++ b/Documentation/public-inbox.cgi.pod
@@ -16,6 +16,11 @@ CGI with Perl is slow due to code loading overhead and web servers lack
 the scheduling fairness of L<public-inbox-httpd(1)> for handling git
 clones and streaming large mbox downloads.
 
+=head1 COMPATIBILITY NOTE
+
+When using the CGI with Apache, make sure to set AllowEncodedSlashes to On, as
+public-inbox makes heavy use of encoded slashes.
+
 =head1 CONTACT
 
 Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>
-- 
2.39.1


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH] public-inbox.cgi(1): Mention AllowEncodedSlashes for Apache setups
  2023-02-17 11:08       ` [PATCH] public-inbox.cgi(1): Mention AllowEncodedSlashes for Apache setups Uwe Kleine-König
@ 2023-02-17 13:15         ` Eric Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-02-17 13:15 UTC (permalink / raw)
  To: Uwe Kleine-König; +Cc: meta

Thanks, pushed as 849538969a6c0ef0319291f31dc69acdec4481db

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Bug related to (maybe?) / in Message-Id
  2023-02-17 10:28     ` Eric Wong
  2023-02-17 10:32       ` [PATCH] TODO: handle more cases of unencoded slashes Eric Wong
  2023-02-17 11:08       ` [PATCH] public-inbox.cgi(1): Mention AllowEncodedSlashes for Apache setups Uwe Kleine-König
@ 2023-02-18 17:58       ` Thomas Weißschuh
  2023-02-18 18:06         ` Thomas Weißschuh
  2 siblings, 1 reply; 11+ messages in thread
From: Thomas Weißschuh @ 2023-02-18 17:58 UTC (permalink / raw)
  To: Eric Wong; +Cc: Uwe Kleine-König, meta

On Fri, Feb 17, 2023 at 10:28:28AM +0000, Eric Wong wrote:
> It's a tragedy that mutt changed their default Message-ID format :<
> Newer versions allow overriding $message_id_format in muttrc,
> but mutt 2.0.5 in Debian 11 won't have it.
> 
> Fwiw, I run mutt on several machines/arches/OSes, getting them
> all to the same new version of mutt will take many years and I
> don't want compile + keep numerous binaries up-to-date in the
> mean time.

Note:

You can also use mutt's builtin facilities to get custom
Message-IDs.

# in muttrc
send-hook . "source ./msgid"

# in file "msgid" next to muttrc
# put any command within the backticks
my_hdr Message-ID: <`uuidgen -r`@t-8ch.de>

The extra config file and "source" command are needed to force
reevaluation of the backticks for each message.

This should also work in older versions of mutt and has the advantage
that mutt knows about the actual Message-ID, for example showing it with
edit_headers=yes.

> I also use msmtp and distribute my muttrc + ~/bin to all of
> them, so I wrote this msmtp wrapper which is mutt-version-agnostic:
> 
> $ cat $HOME/bin/msmtp-msgid
> eval 'exec perl -w -S $0 ${1+"$@"}'
> if 0; # running under some shell
> # in muttrc:
> # set sendmail = msmtp-msgid
> use v5.12;
> my ($hdr, $bdy) = split(/\n\n/, do { local $/; <STDIN> }, 2);
> if ($hdr !~ /^Message-ID:\s*<\d{14}\b/aims) {
> 	my $h;
> 	if ($hdr =~ /^Message-ID:[^@]+\@([^>]+)>/ims) {
> 		$h = $1;
> 	} else {
> 		require Sys::Hostname;
> 		$h = Sys::Hostname::hostname();
> 	}
> 	require POSIX;
> 	require Time::HiRes;
> 	my ($time, $msec) = Time::HiRes::gettimeofday();
> 	my $msgid = POSIX::strftime("%Y%m%d%H%M%S.M$msec", gmtime($time));
> 	$hdr =~ s/^Message-ID:[^\n]+/Message-ID: <$msgid\@$h>/msi;
> }
> my $pid = open(my $fh, '|-', 'msmtp', @ARGV) // die "popen: $!";
> print $fh $hdr, "\n\n", $bdy or die "print $!";
> close $fh or die "msmtp \$?=$?";
> __END__

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Bug related to (maybe?) / in Message-Id
  2023-02-18 17:58       ` Bug related to (maybe?) / in Message-Id Thomas Weißschuh
@ 2023-02-18 18:06         ` Thomas Weißschuh
  2023-03-07  8:31           ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Thomas Weißschuh @ 2023-02-18 18:06 UTC (permalink / raw)
  To: Eric Wong; +Cc: Uwe Kleine-König, meta

On Sat, Feb 18, 2023 at 05:58:57PM +0000, Thomas Weißschuh wrote:
> On Fri, Feb 17, 2023 at 10:28:28AM +0000, Eric Wong wrote:
> > It's a tragedy that mutt changed their default Message-ID format :<
> > Newer versions allow overriding $message_id_format in muttrc,
> > but mutt 2.0.5 in Debian 11 won't have it.
> > 
> > Fwiw, I run mutt on several machines/arches/OSes, getting them
> > all to the same new version of mutt will take many years and I
> > don't want compile + keep numerous binaries up-to-date in the
> > mean time.
> 
> Note:
> 
> You can also use mutt's builtin facilities to get custom
> Message-IDs.
> 
> # in muttrc
> send-hook . "source ./msgid"

One mistake: This needs to be the full path to the "msgid" config:

> send-hook . "source ~/.config/mutt/msgid"

> # in file "msgid" next to muttrc
> # put any command within the backticks
> my_hdr Message-ID: <`uuidgen -r`@t-8ch.de>
> 
> The extra config file and "source" command are needed to force
> reevaluation of the backticks for each message.
> 
> This should also work in older versions of mutt and has the advantage
> that mutt knows about the actual Message-ID, for example showing it with
> edit_headers=yes.
> 
> > I also use msmtp and distribute my muttrc + ~/bin to all of
> > them, so I wrote this msmtp wrapper which is mutt-version-agnostic:
> > 
> > $ cat $HOME/bin/msmtp-msgid
> > eval 'exec perl -w -S $0 ${1+"$@"}'
> > if 0; # running under some shell
> > # in muttrc:
> > # set sendmail = msmtp-msgid
> > use v5.12;
> > my ($hdr, $bdy) = split(/\n\n/, do { local $/; <STDIN> }, 2);
> > if ($hdr !~ /^Message-ID:\s*<\d{14}\b/aims) {
> > 	my $h;
> > 	if ($hdr =~ /^Message-ID:[^@]+\@([^>]+)>/ims) {
> > 		$h = $1;
> > 	} else {
> > 		require Sys::Hostname;
> > 		$h = Sys::Hostname::hostname();
> > 	}
> > 	require POSIX;
> > 	require Time::HiRes;
> > 	my ($time, $msec) = Time::HiRes::gettimeofday();
> > 	my $msgid = POSIX::strftime("%Y%m%d%H%M%S.M$msec", gmtime($time));
> > 	$hdr =~ s/^Message-ID:[^\n]+/Message-ID: <$msgid\@$h>/msi;
> > }
> > my $pid = open(my $fh, '|-', 'msmtp', @ARGV) // die "popen: $!";
> > print $fh $hdr, "\n\n", $bdy or die "print $!";
> > close $fh or die "msmtp \$?=$?";
> > __END__

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Bug related to (maybe?) / in Message-Id
  2023-02-18 18:06         ` Thomas Weißschuh
@ 2023-03-07  8:31           ` Eric Wong
  2023-03-07 22:30             ` Eric Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2023-03-07  8:31 UTC (permalink / raw)
  To: Thomas Weißschuh; +Cc: Uwe Kleine-König, meta

Thomas Weißschuh <thomas@t-8ch.de> wrote:
> > # in file "msgid" next to muttrc
> > # put any command within the backticks
> > my_hdr Message-ID: <`uuidgen -r`@t-8ch.de>

I didn't notice this earlier, but IMHO having Message-IDs
prefixed with `%Y%m%d%H%M%S' can convey human-meaningful
information in the URL without having to open the URL.

(I made git-send-email do that in 2016, too)

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Bug related to (maybe?) / in Message-Id
  2023-03-07  8:31           ` Eric Wong
@ 2023-03-07 22:30             ` Eric Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-03-07 22:30 UTC (permalink / raw)
  To: Thomas Weißschuh; +Cc: Uwe Kleine-König, meta

Eric Wong <e@80x24.org> wrote:
> Thomas Weißschuh <thomas@t-8ch.de> wrote:
> > > # in file "msgid" next to muttrc
> > > # put any command within the backticks
> > > my_hdr Message-ID: <`uuidgen -r`@t-8ch.de>
> 
> I didn't notice this earlier, but IMHO having Message-IDs
> prefixed with `%Y%m%d%H%M%S' can convey human-meaningful
> information in the URL without having to open the URL.

Btw, mutt may gain a more reasonable default:
https://marc.info/?i=20230307_213502_4a7Zc1MB@breakpoint.cc
(not sure how I feel about underscores, though)

(I brought this up a few years back, but wasn't successful:
https://marc.info/?i=20210110213922.iqTO594q5DM@dcvr )

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2023-03-07 22:30 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-16 21:05 Bug related to (maybe?) / in Message-Id Uwe Kleine-König
2023-02-16 21:36 ` Eric Wong
2023-02-17  8:52   ` Uwe Kleine-König
2023-02-17 10:28     ` Eric Wong
2023-02-17 10:32       ` [PATCH] TODO: handle more cases of unencoded slashes Eric Wong
2023-02-17 11:08       ` [PATCH] public-inbox.cgi(1): Mention AllowEncodedSlashes for Apache setups Uwe Kleine-König
2023-02-17 13:15         ` Eric Wong
2023-02-18 17:58       ` Bug related to (maybe?) / in Message-Id Thomas Weißschuh
2023-02-18 18:06         ` Thomas Weißschuh
2023-03-07  8:31           ` Eric Wong
2023-03-07 22:30             ` Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).