* stable 1.6.1 release? [was: [PATCH] eml: fix undefined vars on <Perl 5.28]
@ 2020-12-26 20:35 6% ` Eric Wong
0 siblings, 0 replies; 11+ results
From: Eric Wong @ 2020-12-26 20:35 UTC (permalink / raw)
To: Ali Alnubani, Konstantin Ryabitsev; +Cc: meta
Ali Alnubani <alialnu@nvidia.com> wrote:
> I no longer see the uninitialized value warnings or the test
> failure on Debian 9 with both patches applied on master. Do
> you plan on creating a new release tag soon with these fixes?
Thanks, pushed to master and I've started a stable-1.6 branch
to https://80x24.org/public-inbox.git with 29 commits
cherry-picked from master which I hope are suitable.
There's two more I'm tempted to cherry-pick, though they
introduce behavior changes:
fe01d7b117c8b1e1 import: drop X-Status in addition to Status
1bf653ad139bf7bb nntp+www: drop List-* and Archived-At headers
Konstantin: thoughts on 1bf653ad139bf7bb being suitable for 1.6.1?
Anyways, here's what I have so far:
fdbd73069af6eed9 eml: fix undefined vars on <Perl 5.28
e16e09b239b4d8bf t/config: test --get-urlmatch for git <2.26
933fce93167eba86 inboxidle: avoid needless syscalls on refresh
a0b470cbaf01c699 inboxidle: clue users into resolving ENOSPC from inotify
b782533a0413578d inbox: name variable for values loop iterator
4f1a683dc895a7bd public-inbox-v[12]-format.pod: make lexgrog happy
7a92c24157953dc6 manifest.js.gz: fix per-inbox /$INBOX/manifest.js.gz
78e81ae914ad24df Fix manpage section of perl module documentation
a4a1a74a2f60ec58 t/psgi_v2: ignore warnings on missing P::M::ReverseProxy
1cbb6243533fc2d4 daemon: support --daemonize without Net::Server::Daemonize
734daa9b165e248c doc: v2-format: drop repeated word
b63c27f36a44d8de over: ensure old, merged {tid} is really gone
c39ed01a3a4c6c46 wwwattach: prevent deep-linking via Referer match
0366c73f20b436d4 t/eml.t: workaround newer Email::MIME* behavior
bf14a3670da72358 nntp: attempt RFC 5536 3.1.5-conformant Path: headers
2fcf2b14a9ce3336 nntp: delimit Newsgroup: header with commas
31f9b61a318f4daf tls: epollbit: account for miscellaneous OpenSSL errors
5efbbd5e3e45ff3a scripts/dupe-finder: restore $dbh variable
59cc88bb5bc5ce3e searchidx: index lower-case List-Id value
4ccff6f9122da89c ds: add missing label for systems w/o EPOLLEXCLUSIVE
f9c3b3746445219b imap: avoid raising exception if client disconnects
e578a012532cd91f idxstack: fix comment about file_char
d94b6dd634381748 mda: match List-Id insensitively
c6ca576baf1700a8 mid: drop repeated ';' in mid_escape() regular expression
8e9d4f877730dbdf doc: post-1.6 updates, start 1.7
d6d442866106248e config: warn on multiple values for some fields
64f7ab3a571b9db0 doc: txt2pre: more manpage URLs
915e01b9cd771a84 doc: flow: include -imapd
dec02da946b6bb29 t/indexlevels-mirror: fix improperly skipped test
Thoughts?
Fwiw, I consider these two the most important and was
considering a 1.6.1 release even before the recent fixes:
7a92c24157953dc6 manifest.js.gz: fix per-inbox /$INBOX/manifest.js.gz
b63c27f36a44d8de over: ensure old, merged {tid} is really gone
^ permalink raw reply [relevance 6%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
2020-11-05 7:50 8% ` Eric Wong
@ 2020-11-06 11:57 8% ` Andrey Melnikov
0 siblings, 0 replies; 11+ results
From: Andrey Melnikov @ 2020-11-06 11:57 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
чт, 5 нояб. 2020 г. в 10:50, Eric Wong <e@80x24.org>:
>
[...]
> > Haha. It simply does not expect folded multiline Xref: header. And
> > generate by itself a new (unfolded, separated by space) header when
> > storing article on disk.
> > Is really Newsgroups: & Xref: headers need folding?
>
> Nope, but currently PublicInbox::Eml folds by default
> (and Email::MIME/Email::Simple did in ->as_string, too).
>
> I didn't expect clients to have problems with folded lines since
> Received: and Cc: are often folded, anyways.
>
> Does leafnode struggle with those headers? Or just Newsgroups/Xref?
There are two versions of leafnode - leafnode1 (1.11.11) and leafnode2
(2.0.0.alpha20140727b).
The first version correctly parses headers (it knows about folding),
the second version - not. At least Xref: is affected. Newsgroups:
header is expected to be folded.
> We may need a separate API method for setting NNTP-only headers
> to appease leafnode...
Or write in docs a big fat warning about using leafnode2.
> > > > and new:
> > > > Path: nntp.lore.kernel.org!not.for.mail
> > > > save more extra bytes...
> > > >
> > > > Hehe... yet another cosmetic thing:
> > > > $ telnet nntp.lore.kernel.org 119
> > > > Trying 54.189.247.149...
> > > > Connected to korg-lkml-1-news-lb-839eef9f3a4cef4e.elb.us-west-2.amazonaws.com.
> > > > Escape character is '^]'.
> > > > 201 nntp://nntp.lore.kernel.org ready - post via email
> > > > quit
> > > > Connection closed by foreign host.
> > > >
> > > > 'nntp://nntp.lore.kernel.org' is not a node name per RFC... But this
> > > > is a configuration problem on nntp.lore.kernel.org side.
> > >
> > > You can notify helpdesk@kernel.org to get it fixed.
> >
> > This is cosmetics. nntp readers do not use this hostname.
>
> It may be nice to extract hostname from URI, anyways, since it
> should be possible to advertise NNTPS endpoint in the WWW UI.
=~ s|[^:]+://||; ?
^ permalink raw reply [relevance 8%]
* [PATCH v2] doc/standards: add RFCs for URL schemes
@ 2020-11-05 7:54 7% ` Eric Wong
0 siblings, 0 replies; 11+ results
From: Eric Wong @ 2020-11-05 7:54 UTC (permalink / raw)
To: meta
v2 to resolve conflict from 44227c2624e4f954943d632cd5335396351373be
("nntp: delimit Newsgroup: header with commas").
Still extremely unhappy with performance of this series...
------8<------
Subject: [PATCH] doc/standards: add RFCs for URL schemes
We linkify these in the WWW UI, and will support them in other
places. These URL schemes may end up being stored in
external/detached indices for indexing non-git-based mail
stores.
---
Documentation/standards.perl | 3 +++
1 file changed, 3 insertions(+)
diff --git a/Documentation/standards.perl b/Documentation/standards.perl
index 1c56830e..3ae64ddf 100755
--- a/Documentation/standards.perl
+++ b/Documentation/standards.perl
@@ -28,6 +28,9 @@ my $rfcs = [
1036 => 'Standard for Interchange of USENET Messages',
5536 => 'Netnews Article Format',
5537 => 'Netnews Architecture and Protocols',
+ 1738 => 'Uniform resource locators',
+ 5092 => 'IMAP URL scheme',
+ 5538 => 'NNTP URI schemes',
6048 => 'NNTP additions to LIST command (TODO)',
8054 => 'NNTP compression',
4642 => 'NNTP TLS',
^ permalink raw reply related [relevance 7%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
2020-11-04 21:01 7% ` Andrey Melnikov
@ 2020-11-05 7:50 8% ` Eric Wong
2020-11-06 11:57 8% ` Andrey Melnikov
0 siblings, 1 reply; 11+ results
From: Eric Wong @ 2020-11-05 7:50 UTC (permalink / raw)
To: Andrey Melnikov; +Cc: meta
Andrey Melnikov <temnota.am@gmail.com> wrote:
> ????, 4 ????????. 2020 ??. ?? 22:47, Eric Wong <e@80x24.org>:
> >
> > Andrey Melnikov <temnota.am@gmail.com> wrote:
> > > ????, 4 ????????. 2020 ??. ?? 13:44, Eric Wong <e@80x24.org>:
> > > > I suppose a few extra bytes won't hurt, too much :x
> > >
> > > Current:
> > > Path: y
> > > org.linuxdriverproject.driverdev-devel:35214
> > > org.kernel.vger.linux-devicetree:433117
> > > org.kernel.vger.linux-kernel:3765410
> > > org.kernel.vger.linux-media:192196
> >
> > Huh? From what I can see, the last 4 lines are part of the
> > Xref: header, which is unchanged atm (I plan on speeding it up
> > with extindex, though).
>
> Yeah, leafnode rewrite Xref: header..
> -- raw --
> Sender: "devel" <driverdev-devel-bounces@linuxdriverproject.org>
> Path: y
> Xref: nntp://nntp.lore.kernel.org
> org.linuxdriverproject.driverdev-devel:35265
> org.kernel.vger.linux-devicetree:433316
> org.kernel.vger.linux-kernel:3766059
> org.kernel.vger.linux-mmc:59564
> org.kernel.vger.linux-wireless:204883
> org.kernel.vger.netdev:733193
> Newsgroups: org.linuxdriverproject.driverdev-devel
> org.kernel.vger.linux-devicetree
> org.kernel.vger.linux-kernel org.kernel.vger.linux-mmc
> org.kernel.vger.linux-wireless org.kernel.vger.netdev
> Archived-At: <https://lore.kernel.org/driverdev-devel/20201104191554.GA3972736@bogus/>
> List-Archive: <https://lore.kernel.org/driverdev-devel/>
> -- on disk --
> Sender: "devel" <driverdev-devel-bounces@linuxdriverproject.org>
> Path: y
> org.linuxdriverproject.driverdev-devel:35265
> org.kernel.vger.linux-devicetree:433316
> org.kernel.vger.linux-kernel:3766059
> org.kernel.vger.linux-mmc:59564
> org.kernel.vger.linux-wireless:204883
> org.kernel.vger.netdev:733193
> Newsgroups: org.linuxdriverproject.driverdev-devel
> org.kernel.vger.linux-devicetree
> org.kernel.vger.linux-kernel org.kernel.vger.linux-mmc
> org.kernel.vger.linux-wireless org.kernel.vger.netdev
> Archived-At: <https://lore.kernel.org/driverdev-devel/20201104191554.GA3972736@bogus/>
> List-Archive: <https://lore.kernel.org/driverdev-devel/>
> Xref: news.localnet org.linuxdriverproject.driverdev-devel:4323
> ---
> Haha. It simply does not expect folded multiline Xref: header. And
> generate by itself a new (unfolded, separated by space) header when
> storing article on disk.
> Is really Newsgroups: & Xref: headers need folding?
Nope, but currently PublicInbox::Eml folds by default
(and Email::MIME/Email::Simple did in ->as_string, too).
I didn't expect clients to have problems with folded lines since
Received: and Cc: are often folded, anyways.
Does leafnode struggle with those headers? Or just Newsgroups/Xref?
We may need a separate API method for setting NNTP-only headers
to appease leafnode...
>
> > > and new:
> > > Path: nntp.lore.kernel.org!not.for.mail
> > > save more extra bytes...
> > >
> > > Hehe... yet another cosmetic thing:
> > > $ telnet nntp.lore.kernel.org 119
> > > Trying 54.189.247.149...
> > > Connected to korg-lkml-1-news-lb-839eef9f3a4cef4e.elb.us-west-2.amazonaws.com.
> > > Escape character is '^]'.
> > > 201 nntp://nntp.lore.kernel.org ready - post via email
> > > quit
> > > Connection closed by foreign host.
> > >
> > > 'nntp://nntp.lore.kernel.org' is not a node name per RFC... But this
> > > is a configuration problem on nntp.lore.kernel.org side.
> >
> > You can notify helpdesk@kernel.org to get it fixed.
>
> This is cosmetics. nntp readers do not use this hostname.
It may be nice to extract hostname from URI, anyways, since it
should be possible to advertise NNTPS endpoint in the WWW UI.
^ permalink raw reply [relevance 8%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
2020-11-04 19:47 8% ` Eric Wong
@ 2020-11-04 21:01 7% ` Andrey Melnikov
2020-11-05 7:50 8% ` Eric Wong
0 siblings, 1 reply; 11+ results
From: Andrey Melnikov @ 2020-11-04 21:01 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
ср, 4 нояб. 2020 г. в 22:47, Eric Wong <e@80x24.org>:
>
> Andrey Melnikov <temnota.am@gmail.com> wrote:
> > ср, 4 нояб. 2020 г. в 13:44, Eric Wong <e@80x24.org>:
> > > I suppose a few extra bytes won't hurt, too much :x
> >
> > Current:
> > Path: y
> > org.linuxdriverproject.driverdev-devel:35214
> > org.kernel.vger.linux-devicetree:433117
> > org.kernel.vger.linux-kernel:3765410
> > org.kernel.vger.linux-media:192196
>
> Huh? From what I can see, the last 4 lines are part of the
> Xref: header, which is unchanged atm (I plan on speeding it up
> with extindex, though).
Yeah, leafnode rewrite Xref: header..
-- raw --
Sender: "devel" <driverdev-devel-bounces@linuxdriverproject.org>
Path: y
Xref: nntp://nntp.lore.kernel.org
org.linuxdriverproject.driverdev-devel:35265
org.kernel.vger.linux-devicetree:433316
org.kernel.vger.linux-kernel:3766059
org.kernel.vger.linux-mmc:59564
org.kernel.vger.linux-wireless:204883
org.kernel.vger.netdev:733193
Newsgroups: org.linuxdriverproject.driverdev-devel
org.kernel.vger.linux-devicetree
org.kernel.vger.linux-kernel org.kernel.vger.linux-mmc
org.kernel.vger.linux-wireless org.kernel.vger.netdev
Archived-At: <https://lore.kernel.org/driverdev-devel/20201104191554.GA3972736@bogus/>
List-Archive: <https://lore.kernel.org/driverdev-devel/>
-- on disk --
Sender: "devel" <driverdev-devel-bounces@linuxdriverproject.org>
Path: y
org.linuxdriverproject.driverdev-devel:35265
org.kernel.vger.linux-devicetree:433316
org.kernel.vger.linux-kernel:3766059
org.kernel.vger.linux-mmc:59564
org.kernel.vger.linux-wireless:204883
org.kernel.vger.netdev:733193
Newsgroups: org.linuxdriverproject.driverdev-devel
org.kernel.vger.linux-devicetree
org.kernel.vger.linux-kernel org.kernel.vger.linux-mmc
org.kernel.vger.linux-wireless org.kernel.vger.netdev
Archived-At: <https://lore.kernel.org/driverdev-devel/20201104191554.GA3972736@bogus/>
List-Archive: <https://lore.kernel.org/driverdev-devel/>
Xref: news.localnet org.linuxdriverproject.driverdev-devel:4323
---
Haha. It simply does not expect folded multiline Xref: header. And
generate by itself a new (unfolded, separated by space) header when
storing article on disk.
Is really Newsgroups: & Xref: headers need folding?
> > and new:
> > Path: nntp.lore.kernel.org!not.for.mail
> > save more extra bytes...
> >
> > Hehe... yet another cosmetic thing:
> > $ telnet nntp.lore.kernel.org 119
> > Trying 54.189.247.149...
> > Connected to korg-lkml-1-news-lb-839eef9f3a4cef4e.elb.us-west-2.amazonaws.com.
> > Escape character is '^]'.
> > 201 nntp://nntp.lore.kernel.org ready - post via email
> > quit
> > Connection closed by foreign host.
> >
> > 'nntp://nntp.lore.kernel.org' is not a node name per RFC... But this
> > is a configuration problem on nntp.lore.kernel.org side.
>
> You can notify helpdesk@kernel.org to get it fixed.
This is cosmetics. nntp readers do not use this hostname.
> Though I suppose the public-inbox code could warn or extract the
> hostname out of it.
>
> (NNTPS support would be nice, but maybe OpenSSL or O::Socket::SSL
> on RHEL/CentOS 8 is too old)
^ permalink raw reply [relevance 7%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
2020-11-04 11:46 8% ` Andrey Melnikov
@ 2020-11-04 19:47 8% ` Eric Wong
2020-11-04 21:01 7% ` Andrey Melnikov
0 siblings, 1 reply; 11+ results
From: Eric Wong @ 2020-11-04 19:47 UTC (permalink / raw)
To: Andrey Melnikov; +Cc: meta
Andrey Melnikov <temnota.am@gmail.com> wrote:
> ср, 4 нояб. 2020 г. в 13:44, Eric Wong <e@80x24.org>:
> > I suppose a few extra bytes won't hurt, too much :x
>
> Current:
> Path: y
> org.linuxdriverproject.driverdev-devel:35214
> org.kernel.vger.linux-devicetree:433117
> org.kernel.vger.linux-kernel:3765410
> org.kernel.vger.linux-media:192196
Huh? From what I can see, the last 4 lines are part of the
Xref: header, which is unchanged atm (I plan on speeding it up
with extindex, though).
> and new:
> Path: nntp.lore.kernel.org!not.for.mail
> save more extra bytes...
>
> Hehe... yet another cosmetic thing:
> $ telnet nntp.lore.kernel.org 119
> Trying 54.189.247.149...
> Connected to korg-lkml-1-news-lb-839eef9f3a4cef4e.elb.us-west-2.amazonaws.com.
> Escape character is '^]'.
> 201 nntp://nntp.lore.kernel.org ready - post via email
> quit
> Connection closed by foreign host.
>
> 'nntp://nntp.lore.kernel.org' is not a node name per RFC... But this
> is a configuration problem on nntp.lore.kernel.org side.
You can notify helpdesk@kernel.org to get it fixed.
Though I suppose the public-inbox code could warn or extract the
hostname out of it.
(NNTPS support would be nice, but maybe OpenSSL or O::Socket::SSL
on RHEL/CentOS 8 is too old)
^ permalink raw reply [relevance 8%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
2020-11-04 10:44 7% ` Eric Wong
@ 2020-11-04 11:46 8% ` Andrey Melnikov
2020-11-04 19:47 8% ` Eric Wong
0 siblings, 1 reply; 11+ results
From: Andrey Melnikov @ 2020-11-04 11:46 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
ср, 4 нояб. 2020 г. в 13:44, Eric Wong <e@80x24.org>:
>
> Andrey Melnikov <temnota.am@gmail.com> wrote:
>
> (re-adding meta@public-inbox.org to Cc)
>
> > ср, 4 нояб. 2020 г. в 02:59, Eric Wong <e@80x24.org>:
> > >
> > > Andrey Melnikov <temnota.am@gmail.com> wrote:
> > > > This break filtering on my leafnode.
> > >
> > > OK, thanks for that info.
> > >
> > > > What about "Path:" header?
> > >
> > > Ah, oops; missed that. I put "y" there since I figured it
> > > wouldn't matter for leafnode (but only lightly tested it).
> > >
> > > Since we don't have any NNTP propagation path, I guess putting
> > > just server-name there is appropriate? (not yet deployed)
> > Yes. Leafnode only checks presence on Path: header.
>
> OK, that matches my limited, one-off experience with it.
>
> > > + # required for leafnode (RFC 5536 3.1.5)
> > > + $hdr->header_set('Path', $server_name);
> > I think
> > $hdr->header_set('Path', $server_name . '!not-for-mail');
> > more RFC conform.
>
> I guess... Though it seems like blindly following stuff for
> historical reasons, since I'm not seeing a great explanation of
> "not-for-mail", other than it's "common" in RFC 5536/5537...
> "not-for-mail" could be a valid hostname, after all.
>
> I suppose a few extra bytes won't hurt, too much :x
Current:
Path: y
org.linuxdriverproject.driverdev-devel:35214
org.kernel.vger.linux-devicetree:433117
org.kernel.vger.linux-kernel:3765410
org.kernel.vger.linux-media:192196
and new:
Path: nntp.lore.kernel.org!not.for.mail
save more extra bytes...
Hehe... yet another cosmetic thing:
$ telnet nntp.lore.kernel.org 119
Trying 54.189.247.149...
Connected to korg-lkml-1-news-lb-839eef9f3a4cef4e.elb.us-west-2.amazonaws.com.
Escape character is '^]'.
201 nntp://nntp.lore.kernel.org ready - post via email
quit
Connection closed by foreign host.
'nntp://nntp.lore.kernel.org' is not a node name per RFC... But this
is a configuration problem on nntp.lore.kernel.org side.
> ----------8<--------
> Subject: [PATCH] nntp: attempt RFC 5536 3.1.5-conformant Path: headers
>
> Perhaps some NNTP clients would be unhappy with the old value
> "y". So use a bit more bandwidth+space to use the server-name
> and historical "!not-for-mail" tail-entry to better conform to
> a published RFC.
>
> Reported-by: Andrey Melnikov <temnota.am@gmail.com>
> ---
> lib/PublicInbox/NNTP.pm | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
> index 783c0076..2f821fa6 100644
> --- a/lib/PublicInbox/NNTP.pm
> +++ b/lib/PublicInbox/NNTP.pm
> @@ -415,10 +415,6 @@ sub set_nntp_headers ($$) {
> my ($hdr, $smsg) = @_;
> my ($mid) = $smsg->{mid};
>
> - # why? leafnode requires a Path: header for some inexplicable
> - # reason. We'll fake the shortest one possible.
> - $hdr->header_set('Path', 'y');
> -
> # leafnode (and maybe other NNTP clients) have trouble dealing
> # with v2 messages which have multiple Message-IDs (either due
> # to our own content-based dedupe or buggy git-send-email versions).
> @@ -438,11 +434,15 @@ sub set_nntp_headers ($$) {
> $hdr->header_set('Xref', $xref);
>
> # RFC 5536 3.1.4
> - my $newsgroups = (split(/ /, $xref, 2))[1]; # drop server name
> + my ($server_name, $newsgroups) = split(/ /, $xref, 2);
> $newsgroups =~ s/:[0-9]+\b//g; # drop NNTP article numbers
> $newsgroups =~ tr/ /,/;
> $hdr->header_set('Newsgroups', $newsgroups);
>
> + # *something* here is required for leafnode, try to follow
> + # RFC 5536 3.1.5...
> + $hdr->header_set('Path', $server_name . '!not-for-mail');
> +
> header_append($hdr, 'List-Post', "<mailto:$ibx->{-primary_address}>");
> if (my $url = $ibx->base_url) {
> $mid = mid_escape($mid);
^ permalink raw reply [relevance 8%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
[not found] ` <CA+PODjr0NcFBu42mOdFnmf4vvxb3YZ+nUt=GqWEq+agXa8Au0Q@mail.gmail.com>
@ 2020-11-04 10:44 7% ` Eric Wong
2020-11-04 11:46 8% ` Andrey Melnikov
0 siblings, 1 reply; 11+ results
From: Eric Wong @ 2020-11-04 10:44 UTC (permalink / raw)
To: Andrey Melnikov; +Cc: meta
Andrey Melnikov <temnota.am@gmail.com> wrote:
(re-adding meta@public-inbox.org to Cc)
> ср, 4 нояб. 2020 г. в 02:59, Eric Wong <e@80x24.org>:
> >
> > Andrey Melnikov <temnota.am@gmail.com> wrote:
> > > This break filtering on my leafnode.
> >
> > OK, thanks for that info.
> >
> > > What about "Path:" header?
> >
> > Ah, oops; missed that. I put "y" there since I figured it
> > wouldn't matter for leafnode (but only lightly tested it).
> >
> > Since we don't have any NNTP propagation path, I guess putting
> > just server-name there is appropriate? (not yet deployed)
> Yes. Leafnode only checks presence on Path: header.
OK, that matches my limited, one-off experience with it.
> > + # required for leafnode (RFC 5536 3.1.5)
> > + $hdr->header_set('Path', $server_name);
> I think
> $hdr->header_set('Path', $server_name . '!not-for-mail');
> more RFC conform.
I guess... Though it seems like blindly following stuff for
historical reasons, since I'm not seeing a great explanation of
"not-for-mail", other than it's "common" in RFC 5536/5537...
"not-for-mail" could be a valid hostname, after all.
I suppose a few extra bytes won't hurt, too much :x
----------8<--------
Subject: [PATCH] nntp: attempt RFC 5536 3.1.5-conformant Path: headers
Perhaps some NNTP clients would be unhappy with the old value
"y". So use a bit more bandwidth+space to use the server-name
and historical "!not-for-mail" tail-entry to better conform to
a published RFC.
Reported-by: Andrey Melnikov <temnota.am@gmail.com>
---
lib/PublicInbox/NNTP.pm | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 783c0076..2f821fa6 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -415,10 +415,6 @@ sub set_nntp_headers ($$) {
my ($hdr, $smsg) = @_;
my ($mid) = $smsg->{mid};
- # why? leafnode requires a Path: header for some inexplicable
- # reason. We'll fake the shortest one possible.
- $hdr->header_set('Path', 'y');
-
# leafnode (and maybe other NNTP clients) have trouble dealing
# with v2 messages which have multiple Message-IDs (either due
# to our own content-based dedupe or buggy git-send-email versions).
@@ -438,11 +434,15 @@ sub set_nntp_headers ($$) {
$hdr->header_set('Xref', $xref);
# RFC 5536 3.1.4
- my $newsgroups = (split(/ /, $xref, 2))[1]; # drop server name
+ my ($server_name, $newsgroups) = split(/ /, $xref, 2);
$newsgroups =~ s/:[0-9]+\b//g; # drop NNTP article numbers
$newsgroups =~ tr/ /,/;
$hdr->header_set('Newsgroups', $newsgroups);
+ # *something* here is required for leafnode, try to follow
+ # RFC 5536 3.1.5...
+ $hdr->header_set('Path', $server_name . '!not-for-mail');
+
header_append($hdr, 'List-Post', "<mailto:$ibx->{-primary_address}>");
if (my $url = $ibx->base_url) {
$mid = mid_escape($mid);
^ permalink raw reply related [relevance 7%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
2020-11-03 23:11 8% ` Andrey Melnikov
@ 2020-11-03 23:59 7% ` Eric Wong
[not found] ` <CA+PODjr0NcFBu42mOdFnmf4vvxb3YZ+nUt=GqWEq+agXa8Au0Q@mail.gmail.com>
0 siblings, 1 reply; 11+ results
From: Eric Wong @ 2020-11-03 23:59 UTC (permalink / raw)
To: Andrey Melnikov; +Cc: meta
Andrey Melnikov <temnota.am@gmail.com> wrote:
> This break filtering on my leafnode.
OK, thanks for that info.
> What about "Path:" header?
Ah, oops; missed that. I put "y" there since I figured it
wouldn't matter for leafnode (but only lightly tested it).
Since we don't have any NNTP propagation path, I guess putting
just server-name there is appropriate? (not yet deployed)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 783c0076..cb095bca 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -415,10 +415,6 @@ sub set_nntp_headers ($$) {
my ($hdr, $smsg) = @_;
my ($mid) = $smsg->{mid};
- # why? leafnode requires a Path: header for some inexplicable
- # reason. We'll fake the shortest one possible.
- $hdr->header_set('Path', 'y');
-
# leafnode (and maybe other NNTP clients) have trouble dealing
# with v2 messages which have multiple Message-IDs (either due
# to our own content-based dedupe or buggy git-send-email versions).
@@ -438,11 +434,14 @@ sub set_nntp_headers ($$) {
$hdr->header_set('Xref', $xref);
# RFC 5536 3.1.4
- my $newsgroups = (split(/ /, $xref, 2))[1]; # drop server name
+ my ($server_name, $newsgroups) = split(/ /, $xref, 2);
$newsgroups =~ s/:[0-9]+\b//g; # drop NNTP article numbers
$newsgroups =~ tr/ /,/;
$hdr->header_set('Newsgroups', $newsgroups);
+ # required for leafnode (RFC 5536 3.1.5)
+ $hdr->header_set('Path', $server_name);
+
header_append($hdr, 'List-Post', "<mailto:$ibx->{-primary_address}>");
if (my $url = $ibx->base_url) {
$mid = mid_escape($mid);
^ permalink raw reply related [relevance 7%]
* Re: [PATCH] nntp: delimit Newsgroup: header with commas
2020-11-03 22:55 14% ` [PATCH] nntp: delimit Newsgroup: header with commas Eric Wong
@ 2020-11-03 23:11 8% ` Andrey Melnikov
2020-11-03 23:59 7% ` Eric Wong
0 siblings, 1 reply; 11+ results
From: Andrey Melnikov @ 2020-11-03 23:11 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
ср, 4 нояб. 2020 г. в 01:56, Eric Wong <e@80x24.org>:
>
> Andrey Melnikov <temnota.am@gmail.com> wrote:
> > Hi.
> >
> > nntpd generate wrong "Newsgoups:" header when message cross-posted to
> > multiple groups. Currently, it write multiple groups separated with
> > space, but rfc1036 chapter 2.1.3 says "... Multiple newsgroups may be
> > specified, separated by a comma. ...".
> > and Path: header too - rfc1036 chapter 2.1.6 says " ... The names may
> > be separated by any punctuation character or characters (except "."
> > which is considered part of the hostname). ..."
> > Currently both headers separated as email-style with spaces,
>
> Thanks for the bug report. Does this break known clients or
> is it just something you happened to notice?
This break filtering on my leafnode.
> I'm using RFC 5536 since it obsoletes 1036.
> Testing the following on news.public-inbox.org:
>
> ----------8<-------
> Subject: [PATCH] nntp: delimit Newsgroup: header with commas
>
> ...instead of spaces. This is specified in RFC 5536 3.1.4.
>
> Include references to RFC 1036, 5536 and 5537 in our docs while
> we're at it.
>
> Reported-by: Andrey Melnikov <temnota.am@gmail.com>
> Link: https://public-inbox.org/meta/CA+PODjpUN5Q4gBFQhAzUNuMasVEdmp9f=8Uo0Ej0mFumdSwi4w@mail.gmail.com/
> ---
> Documentation/standards.perl | 3 +++
> lib/PublicInbox/NNTP.pm | 11 ++++++++---
> 2 files changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/standards.perl b/Documentation/standards.perl
> index 0ac6cc52..1c56830e 100755
> --- a/Documentation/standards.perl
> +++ b/Documentation/standards.perl
> @@ -25,6 +25,9 @@ EOF
> my $rfcs = [
> 3977 => 'NNTP',
> 977 => 'NNTP (old)',
> + 1036 => 'Standard for Interchange of USENET Messages',
> + 5536 => 'Netnews Article Format',
> + 5537 => 'Netnews Architecture and Protocols',
> 6048 => 'NNTP additions to LIST command (TODO)',
> 8054 => 'NNTP compression',
> 4642 => 'NNTP TLS',
> diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
> index 102ef42c..783c0076 100644
> --- a/lib/PublicInbox/NNTP.pm
> +++ b/lib/PublicInbox/NNTP.pm
> @@ -432,12 +432,17 @@ sub set_nntp_headers ($$) {
> $hdr->header_set('X-Alt-Message-ID', @alt);
> }
>
> - # clobber some
> + # clobber some existing headers
> my $ibx = $smsg->{-ibx};
> my $xref = xref($smsg->{nntp}, $ibx, $smsg->{num}, $mid);
> $hdr->header_set('Xref', $xref);
> - $xref =~ s/:[0-9]+//g;
> - $hdr->header_set('Newsgroups', (split(/ /, $xref, 2))[1]);
> +
> + # RFC 5536 3.1.4
> + my $newsgroups = (split(/ /, $xref, 2))[1]; # drop server name
> + $newsgroups =~ s/:[0-9]+\b//g; # drop NNTP article numbers
> + $newsgroups =~ tr/ /,/;
> + $hdr->header_set('Newsgroups', $newsgroups);
> +
> header_append($hdr, 'List-Post', "<mailto:$ibx->{-primary_address}>");
> if (my $url = $ibx->base_url) {
> $mid = mid_escape($mid);
What about "Path:" header?
^ permalink raw reply [relevance 8%]
* [PATCH] nntp: delimit Newsgroup: header with commas
@ 2020-11-03 22:55 14% ` Eric Wong
2020-11-03 23:11 8% ` Andrey Melnikov
0 siblings, 1 reply; 11+ results
From: Eric Wong @ 2020-11-03 22:55 UTC (permalink / raw)
To: Andrey Melnikov; +Cc: meta
Andrey Melnikov <temnota.am@gmail.com> wrote:
> Hi.
>
> nntpd generate wrong "Newsgoups:" header when message cross-posted to
> multiple groups. Currently, it write multiple groups separated with
> space, but rfc1036 chapter 2.1.3 says "... Multiple newsgroups may be
> specified, separated by a comma. ...".
> and Path: header too - rfc1036 chapter 2.1.6 says " ... The names may
> be separated by any punctuation character or characters (except "."
> which is considered part of the hostname). ..."
> Currently both headers separated as email-style with spaces,
Thanks for the bug report. Does this break known clients or
is it just something you happened to notice?
I'm using RFC 5536 since it obsoletes 1036.
Testing the following on news.public-inbox.org:
----------8<-------
Subject: [PATCH] nntp: delimit Newsgroup: header with commas
...instead of spaces. This is specified in RFC 5536 3.1.4.
Include references to RFC 1036, 5536 and 5537 in our docs while
we're at it.
Reported-by: Andrey Melnikov <temnota.am@gmail.com>
Link: https://public-inbox.org/meta/CA+PODjpUN5Q4gBFQhAzUNuMasVEdmp9f=8Uo0Ej0mFumdSwi4w@mail.gmail.com/
---
Documentation/standards.perl | 3 +++
lib/PublicInbox/NNTP.pm | 11 ++++++++---
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/Documentation/standards.perl b/Documentation/standards.perl
index 0ac6cc52..1c56830e 100755
--- a/Documentation/standards.perl
+++ b/Documentation/standards.perl
@@ -25,6 +25,9 @@ EOF
my $rfcs = [
3977 => 'NNTP',
977 => 'NNTP (old)',
+ 1036 => 'Standard for Interchange of USENET Messages',
+ 5536 => 'Netnews Article Format',
+ 5537 => 'Netnews Architecture and Protocols',
6048 => 'NNTP additions to LIST command (TODO)',
8054 => 'NNTP compression',
4642 => 'NNTP TLS',
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 102ef42c..783c0076 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -432,12 +432,17 @@ sub set_nntp_headers ($$) {
$hdr->header_set('X-Alt-Message-ID', @alt);
}
- # clobber some
+ # clobber some existing headers
my $ibx = $smsg->{-ibx};
my $xref = xref($smsg->{nntp}, $ibx, $smsg->{num}, $mid);
$hdr->header_set('Xref', $xref);
- $xref =~ s/:[0-9]+//g;
- $hdr->header_set('Newsgroups', (split(/ /, $xref, 2))[1]);
+
+ # RFC 5536 3.1.4
+ my $newsgroups = (split(/ /, $xref, 2))[1]; # drop server name
+ $newsgroups =~ s/:[0-9]+\b//g; # drop NNTP article numbers
+ $newsgroups =~ tr/ /,/;
+ $hdr->header_set('Newsgroups', $newsgroups);
+
header_append($hdr, 'List-Post', "<mailto:$ibx->{-primary_address}>");
if (my $url = $ibx->base_url) {
$mid = mid_escape($mid);
^ permalink raw reply related [relevance 14%]
Results 1-11 of 11 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-10-27 7:54 [PATCH 00/52] detached external index: mostly Eric Wong
2020-10-27 7:54 ` [PATCH 01/52] doc/standards: add RFCs for URL schemes Eric Wong
2020-11-05 7:54 7% ` [PATCH v2] " Eric Wong
2020-11-03 12:29 nntpd generate wrong headers Andrey Melnikov
2020-11-03 22:55 14% ` [PATCH] nntp: delimit Newsgroup: header with commas Eric Wong
2020-11-03 23:11 8% ` Andrey Melnikov
2020-11-03 23:59 7% ` Eric Wong
[not found] ` <CA+PODjr0NcFBu42mOdFnmf4vvxb3YZ+nUt=GqWEq+agXa8Au0Q@mail.gmail.com>
2020-11-04 10:44 7% ` Eric Wong
2020-11-04 11:46 8% ` Andrey Melnikov
2020-11-04 19:47 8% ` Eric Wong
2020-11-04 21:01 7% ` Andrey Melnikov
2020-11-05 7:50 8% ` Eric Wong
2020-11-06 11:57 8% ` Andrey Melnikov
2020-12-26 11:27 [Debian 9][Perl 5.24] uninitialized value errors in Encode/MIME/Header.pm Ali Alnubani
2020-12-26 12:25 ` [PATCH] eml: fix undefined vars on <Perl 5.28 Eric Wong
2020-12-26 14:10 ` Ali Alnubani
2020-12-26 20:35 6% ` stable 1.6.1 release? [was: [PATCH] eml: fix undefined vars on <Perl 5.28] Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).