* [PATCH 5/5] www: mbox* drop unneeded {base_url} memoizations
@ 2022-08-20 8:01 7% ` Eric Wong
0 siblings, 0 replies; 6+ results
From: Eric Wong @ 2022-08-20 8:01 UTC (permalink / raw)
To: meta
That field is not needed since List-* and Archived-At headers
are no longer appended as of commit:
1bf653ad139bf7bb (nntp+www: drop List-* and Archived-At headers, 2020-12-10)
---
lib/PublicInbox/Mbox.pm | 3 +--
lib/PublicInbox/MboxGz.pm | 3 +--
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/Mbox.pm b/lib/PublicInbox/Mbox.pm
index e65f38f0..2ef8ff2b 100644
--- a/lib/PublicInbox/Mbox.pm
+++ b/lib/PublicInbox/Mbox.pm
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
# Streaming interface for mboxrd HTTP responses
@@ -82,7 +82,6 @@ sub no_over_raw ($) {
# /$INBOX/$MESSAGE_ID/raw
sub emit_raw {
my ($ctx) = @_;
- $ctx->{base_url} = $ctx->{ibx}->base_url($ctx->{env});
my $over = $ctx->{ibx}->over or return no_over_raw($ctx);
my ($id, $prev);
my $mip = $ctx->{next_arg} = [ $ctx->{mid}, \$id, \$prev ];
diff --git a/lib/PublicInbox/MboxGz.pm b/lib/PublicInbox/MboxGz.pm
index 3ed33867..56f5adcf 100644
--- a/lib/PublicInbox/MboxGz.pm
+++ b/lib/PublicInbox/MboxGz.pm
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
package PublicInbox::MboxGz;
use strict;
@@ -22,7 +22,6 @@ sub async_next ($) {
sub mbox_gz {
my ($self, $cb, $fn) = @_;
$self->{cb} = $cb;
- $self->{base_url} = $self->{ibx}->base_url($self->{env});
$self->{gz} = PublicInbox::GzipFilter::gzip_or_die();
$fn = to_filename($fn // '') // 'no-subject';
# http://www.iana.org/assignments/media-types/application/gzip
^ permalink raw reply related [relevance 7%]
* [PATCH 5/5] imapd|nntpd: drop ->base_url preload
@ 2022-08-08 23:53 6% ` Eric Wong
0 siblings, 0 replies; 6+ results
From: Eric Wong @ 2022-08-08 23:53 UTC (permalink / raw)
To: meta
It was never useful for IMAP, and NNTP hasn't needed it since:
1bf653ad139bf7bb (nntp+www: drop List-* and Archived-At headers, 2020-12-10)
---
lib/PublicInbox/IMAPD.pm | 1 -
lib/PublicInbox/NNTPD.pm | 1 -
2 files changed, 2 deletions(-)
diff --git a/lib/PublicInbox/IMAPD.pm b/lib/PublicInbox/IMAPD.pm
index ba6ad05d..78323e57 100644
--- a/lib/PublicInbox/IMAPD.pm
+++ b/lib/PublicInbox/IMAPD.pm
@@ -44,7 +44,6 @@ sub _refresh_ibx { # pi_cfg->each_inbox cb
PublicInbox::IMAP::ensure_slices_exist($imapd, $ibx);
# preload to avoid fragmentation:
$ibx->description;
- $ibx->base_url;
# ensure dummies are selectable:
do {
$dummies->{$ngname} = $dummy;
diff --git a/lib/PublicInbox/NNTPD.pm b/lib/PublicInbox/NNTPD.pm
index 4f550bb0..4401a29b 100644
--- a/lib/PublicInbox/NNTPD.pm
+++ b/lib/PublicInbox/NNTPD.pm
@@ -45,7 +45,6 @@ sub refresh_groups {
# only valid if msgmap and over works
# preload to avoid fragmentation:
$ibx->description;
- $ibx->base_url;
} else {
delete $groups->{$ngname};
# Note: don't be tempted to delete more for memory
^ permalink raw reply related [relevance 6%]
* [PATCH 1/2] nntp: remove unused header_append method
@ 2021-03-16 8:48 6% ` Eric Wong
0 siblings, 0 replies; 6+ results
From: Eric Wong @ 2021-03-16 8:48 UTC (permalink / raw)
To: meta
It was unused since 1bf653ad139bf7bb3d853ab0b5eae3eaa1b13a95
("nntp+www: drop List-* and Archived-At headers")
---
lib/PublicInbox/NNTP.pm | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 18822d3b..f7d99913 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -409,15 +409,6 @@ sub cmd_quit ($) {
undef;
}
-sub header_append ($$$) {
- my ($hdr, $k, $v) = @_;
- my @v = $hdr->header_raw($k);
- foreach (@v) {
- return if $v eq $_;
- }
- $hdr->header_set($k, @v, $v);
-}
-
sub xref_by_tc ($$$) {
my ($xref, $pi_cfg, $smsg) = @_;
my $by_addr = $pi_cfg->{-by_addr};
^ permalink raw reply related [relevance 6%]
* Re: stable 1.6.1 release? [was: [PATCH] eml: fix undefined vars on <Perl 5.28]
2020-12-26 20:35 6% ` stable 1.6.1 release? [was: [PATCH] eml: fix undefined vars on <Perl 5.28] Eric Wong
@ 2020-12-31 0:06 0% ` Eric Wong
0 siblings, 0 replies; 6+ results
From: Eric Wong @ 2020-12-31 0:06 UTC (permalink / raw)
To: Ali Alnubani, Konstantin Ryabitsev; +Cc: meta
Eric Wong <e@80x24.org> wrote:
> Ali Alnubani <alialnu@nvidia.com> wrote:
> > I no longer see the uninitialized value warnings or the test
> > failure on Debian 9 with both patches applied on master. Do
> > you plan on creating a new release tag soon with these fixes?
>
> Thanks, pushed to master and I've started a stable-1.6 branch
> to https://80x24.org/public-inbox.git with 29 commits
> cherry-picked from master which I hope are suitable.
>
> There's two more I'm tempted to cherry-pick, though they
> introduce behavior changes:
>
> fe01d7b117c8b1e1 import: drop X-Status in addition to Status
I've picked fe01d7b117c8b1e1 since it's a potential privacy leak.
> 1bf653ad139bf7bb nntp+www: drop List-* and Archived-At headers
Doesn't apply cleanly, but trivial to fix. But not sure if it's
a big-enough issue. Anyways, I'll release 1.6.1 before 2021...
(still lots of stuff brewing for lei + extindex...)
^ permalink raw reply [relevance 0%]
* stable 1.6.1 release? [was: [PATCH] eml: fix undefined vars on <Perl 5.28]
@ 2020-12-26 20:35 6% ` Eric Wong
2020-12-31 0:06 0% ` Eric Wong
0 siblings, 1 reply; 6+ results
From: Eric Wong @ 2020-12-26 20:35 UTC (permalink / raw)
To: Ali Alnubani, Konstantin Ryabitsev; +Cc: meta
Ali Alnubani <alialnu@nvidia.com> wrote:
> I no longer see the uninitialized value warnings or the test
> failure on Debian 9 with both patches applied on master. Do
> you plan on creating a new release tag soon with these fixes?
Thanks, pushed to master and I've started a stable-1.6 branch
to https://80x24.org/public-inbox.git with 29 commits
cherry-picked from master which I hope are suitable.
There's two more I'm tempted to cherry-pick, though they
introduce behavior changes:
fe01d7b117c8b1e1 import: drop X-Status in addition to Status
1bf653ad139bf7bb nntp+www: drop List-* and Archived-At headers
Konstantin: thoughts on 1bf653ad139bf7bb being suitable for 1.6.1?
Anyways, here's what I have so far:
fdbd73069af6eed9 eml: fix undefined vars on <Perl 5.28
e16e09b239b4d8bf t/config: test --get-urlmatch for git <2.26
933fce93167eba86 inboxidle: avoid needless syscalls on refresh
a0b470cbaf01c699 inboxidle: clue users into resolving ENOSPC from inotify
b782533a0413578d inbox: name variable for values loop iterator
4f1a683dc895a7bd public-inbox-v[12]-format.pod: make lexgrog happy
7a92c24157953dc6 manifest.js.gz: fix per-inbox /$INBOX/manifest.js.gz
78e81ae914ad24df Fix manpage section of perl module documentation
a4a1a74a2f60ec58 t/psgi_v2: ignore warnings on missing P::M::ReverseProxy
1cbb6243533fc2d4 daemon: support --daemonize without Net::Server::Daemonize
734daa9b165e248c doc: v2-format: drop repeated word
b63c27f36a44d8de over: ensure old, merged {tid} is really gone
c39ed01a3a4c6c46 wwwattach: prevent deep-linking via Referer match
0366c73f20b436d4 t/eml.t: workaround newer Email::MIME* behavior
bf14a3670da72358 nntp: attempt RFC 5536 3.1.5-conformant Path: headers
2fcf2b14a9ce3336 nntp: delimit Newsgroup: header with commas
31f9b61a318f4daf tls: epollbit: account for miscellaneous OpenSSL errors
5efbbd5e3e45ff3a scripts/dupe-finder: restore $dbh variable
59cc88bb5bc5ce3e searchidx: index lower-case List-Id value
4ccff6f9122da89c ds: add missing label for systems w/o EPOLLEXCLUSIVE
f9c3b3746445219b imap: avoid raising exception if client disconnects
e578a012532cd91f idxstack: fix comment about file_char
d94b6dd634381748 mda: match List-Id insensitively
c6ca576baf1700a8 mid: drop repeated ';' in mid_escape() regular expression
8e9d4f877730dbdf doc: post-1.6 updates, start 1.7
d6d442866106248e config: warn on multiple values for some fields
64f7ab3a571b9db0 doc: txt2pre: more manpage URLs
915e01b9cd771a84 doc: flow: include -imapd
dec02da946b6bb29 t/indexlevels-mirror: fix improperly skipped test
Thoughts?
Fwiw, I consider these two the most important and was
considering a 1.6.1 release even before the recent fixes:
7a92c24157953dc6 manifest.js.gz: fix per-inbox /$INBOX/manifest.js.gz
b63c27f36a44d8de over: ensure old, merged {tid} is really gone
^ permalink raw reply [relevance 6%]
* Re: Extra newline when retrieving messages
@ 2020-12-10 22:38 5% ` Eric Wong
0 siblings, 0 replies; 6+ results
From: Eric Wong @ 2020-12-10 22:38 UTC (permalink / raw)
To: meta
Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
<snip>
> That said, should public-inbox consider this case when generating the
> /raw and /t.mbox.gz messages? If the Archived-At and List-Archive
> headers are listed in the DKIM-Signature header, skip inserting them
> into the generated message?
Fwiw, I've been wondering if they are even necessary. I don't
think anybody misses them with -imapd. So I prefer to just drop
them and have less code:
---------8<---------
Subject: [PATCH] nntp+www: drop List-* and Archived-At headers
These headers can conflict with headers in the DKIM signature;
and parsing the DKIM-Signature header to determine whether or
not we can safely add a header would be more code and CPU
cycles.
Since IMAP seems fine without these headers (and JMAP will
likely be, too), there's likely no need to continue appending
these to every message. Nowadays, developers seem sufficiently
trained to use URLs with Message-IDs in them. So drop the
headers and save some cycles and bandwidth all around.
---
lib/PublicInbox/Mbox.pm | 32 ++++++--------------------------
lib/PublicInbox/MboxGz.pm | 2 +-
lib/PublicInbox/NNTP.pm | 8 --------
t/nntp.t | 9 ---------
t/psgi_mount.t | 14 ++++----------
5 files changed, 11 insertions(+), 54 deletions(-)
diff --git a/lib/PublicInbox/Mbox.pm b/lib/PublicInbox/Mbox.pm
index c88350c9..c8e4b406 100644
--- a/lib/PublicInbox/Mbox.pm
+++ b/lib/PublicInbox/Mbox.pm
@@ -20,7 +20,7 @@ sub getline {
my $ibx = $ctx->{ibx};
my $eml = $ibx->smsg_eml($smsg) or return;
my $n = $ctx->{smsg} = $ibx->over->next_by_mid(@{$ctx->{next_arg}});
- $ctx->zmore(msg_hdr($ctx, $eml, $smsg->{mid}));
+ $ctx->zmore(msg_hdr($ctx, $eml));
if ($n) {
$ctx->translate(msg_body($eml));
} else { # last message
@@ -46,7 +46,7 @@ sub async_eml { # for async_blob_cb
# next message
$ctx->{smsg} = $ctx->{ibx}->over->next_by_mid(@{$ctx->{next_arg}});
- $ctx->zmore(msg_hdr($ctx, $eml, $smsg->{mid}));
+ $ctx->zmore(msg_hdr($ctx, $eml));
$ctx->{http_out}->write($ctx->translate(msg_body($eml)));
}
@@ -74,7 +74,7 @@ sub no_over_raw ($) {
my $mref = $ctx->{ibx}->msg_by_mid($ctx->{mid}) or return;
my $eml = PublicInbox::Eml->new($mref);
[ 200, res_hdr($ctx, $eml->header_str('Subject')),
- [ msg_hdr($ctx, $eml, $ctx->{mid}) . msg_body($eml) ] ]
+ [ msg_hdr($ctx, $eml) . msg_body($eml) ] ]
}
# /$INBOX/$MESSAGE_ID/raw
@@ -90,8 +90,8 @@ sub emit_raw {
$ctx->psgi_response(200, $res_hdr);
}
-sub msg_hdr ($$;$) {
- my ($ctx, $eml, $mid) = @_;
+sub msg_hdr ($$) {
+ my ($ctx, $eml) = @_;
my $header_obj = $eml->header_obj;
# drop potentially confusing headers, ssoma already should've dropped
@@ -99,31 +99,11 @@ sub msg_hdr ($$;$) {
foreach my $d (qw(Lines Bytes Content-Length Status)) {
$header_obj->header_set($d);
}
- my $ibx = $ctx->{ibx};
- my $base = $ctx->{base_url};
- $mid = $ctx->{mid} unless defined $mid;
- $mid = mid_escape($mid);
- my @append = (
- 'Archived-At', "<$base$mid/>",
- 'List-Archive', "<$base>",
- );
my $crlf = $header_obj->crlf;
my $buf = $header_obj->as_string;
# fixup old bug from import (pre-a0c07cba0e5d8b6a)
$buf =~ s/\A[\r\n]*From [^\r\n]*\r?\n//s;
- $buf = "From mboxrd\@z Thu Jan 1 00:00:00 1970" . $crlf . $buf;
-
- for (my $i = 0; $i < @append; $i += 2) {
- my $k = $append[$i];
- my $v = $append[$i + 1];
- my @v = $header_obj->header_raw($k);
- $buf .= "$k: $v$crlf" if !grep(/\A\Q$v\E\z/, @v);
- }
- my $post_addr = $ibx->{-primary_address};
- if ($post_addr && $header_obj->header_raw('List-Post')) {
- $buf .= "List-Post: <mailto:$post_addr>$crlf";
- }
- $buf .= $crlf;
+ "From mboxrd\@z Thu Jan 1 00:00:00 1970" . $crlf . $buf . $crlf;
}
sub msg_body ($) {
diff --git a/lib/PublicInbox/MboxGz.pm b/lib/PublicInbox/MboxGz.pm
index ab3c9770..7b054845 100644
--- a/lib/PublicInbox/MboxGz.pm
+++ b/lib/PublicInbox/MboxGz.pm
@@ -38,7 +38,7 @@ sub getline {
my $cb = $self->{cb} or return;
while (my $smsg = $cb->($self)) {
my $eml = $self->{ibx}->smsg_eml($smsg) or next;
- $self->zmore(msg_hdr($self, $eml, $smsg->{mid}));
+ $self->zmore(msg_hdr($self, $eml));
return $self->translate(msg_body($eml));
}
# signal that we're done and can return undef next call:
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 097fdb84..11a7ffb8 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -487,14 +487,6 @@ sub set_nntp_headers ($$) {
# *something* here is required for leafnode, try to follow
# RFC 5536 3.1.5...
$hdr->header_set('Path', $server_name . '!not-for-mail');
- if (my $post_addr = $ibx->{-primary_address}) {
- header_append($hdr, 'List-Post', "<mailto:$post_addr>");
- }
- if (my $url = $ibx->base_url) {
- $mid = mid_escape($mid);
- header_append($hdr, 'Archived-At', "<$url$mid/>");
- header_append($hdr, 'List-Archive', "<$url>");
- }
}
sub art_lookup ($$$) {
diff --git a/t/nntp.t b/t/nntp.t
index b745886d..36eb6945 100644
--- a/t/nntp.t
+++ b/t/nntp.t
@@ -121,12 +121,6 @@ use PublicInbox::Config;
PublicInbox::NNTP::set_nntp_headers($hdr, $smsg);
is_deeply([ $mime->header('Message-ID') ], [ "<$mid>" ],
'Message-ID unchanged');
- is_deeply([ $mime->header('Archived-At') ], [ "<${u}a\@b/>" ],
- 'Archived-At: set');
- is_deeply([ $mime->header('List-Archive') ], [ "<$u>" ],
- 'List-Archive: set');
- is_deeply([ $mime->header('List-Post') ], [ '<mailto:a@example.com>' ],
- 'List-Post: set');
is_deeply([ $mime->header('Newsgroups') ], [ 'test' ],
'Newsgroups: set');
is_deeply([ $mime->header('Xref') ], [ 'example.com test:1' ],
@@ -137,9 +131,6 @@ use PublicInbox::Config;
PublicInbox::NNTP::set_nntp_headers($hdr, $smsg);
is_deeply([ $mime->header('Message-ID') ], [ "<$mid>" ],
'Message-ID unchanged');
- is_deeply([ $mime->header('Archived-At') ],
- [ "<${u}a\@b/>", '<http://mirror.example.com/m/a@b/>' ],
- 'Archived-At: appended');
is_deeply([ $mime->header('Xref') ], [ 'example.com test:2' ],
'Old Xref: clobbered');
}
diff --git a/t/psgi_mount.t b/t/psgi_mount.t
index dac62c1a..48d8e5c0 100644
--- a/t/psgi_mount.t
+++ b/t/psgi_mount.t
@@ -67,11 +67,9 @@ test_psgi($app, sub {
$res = $cb->(GET('/a/test/blah%40example.com/raw'));
is($res->code, 200, 'OK with URLMap mount');
- like($res->content, qr!^List-Archive: <http://[^/]+/a/test/>!m,
- 'List-Archive set in /raw mboxrd');
like($res->content,
- qr!^Archived-At: <http://[^/]+/a/test/blah\@example\.com/>!m,
- 'Archived-At set in /raw mboxrd');
+ qr/^Message-Id: <blah\@example\.com>\n/sm,
+ 'headers appear in /raw');
# redirects
$res = $cb->(GET('/a/test/m/blah%40example.com.html'));
@@ -94,12 +92,8 @@ SKIP: {
my $gz = $res->content;
my $raw;
IO::Uncompress::Gunzip::gunzip(\$gz => \$raw);
- like($raw, qr!^List-Archive: <http://[^/]+/a/test/>!m,
- 'List-Archive set in /t.mbox.gz mboxrd');
- like($raw,
- qr!^Archived-At:\x20
- <http://[^/]+/a/test/blah\@example\.com/>!mx,
- 'Archived-At set in /t.mbox.gz mboxrd');
+ like($raw, qr!^Message-Id:\x20<blah\@example\.com>\n!sm,
+ 'headers appear in /t.mbox.gz mboxrd');
});
}
^ permalink raw reply related [relevance 5%]
Results 1-6 of 6 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-12-10 20:21 Extra newline when retrieving messages Konstantin Ryabitsev
2020-12-10 20:55 ` Eric Wong
2020-12-10 21:43 ` Konstantin Ryabitsev
2020-12-10 22:38 5% ` Eric Wong
2020-12-26 11:27 [Debian 9][Perl 5.24] uninitialized value errors in Encode/MIME/Header.pm Ali Alnubani
2020-12-26 12:25 ` [PATCH] eml: fix undefined vars on <Perl 5.28 Eric Wong
2020-12-26 14:10 ` Ali Alnubani
2020-12-26 20:35 6% ` stable 1.6.1 release? [was: [PATCH] eml: fix undefined vars on <Perl 5.28] Eric Wong
2020-12-31 0:06 0% ` Eric Wong
2021-03-16 8:48 [PATCH 0/2] some dead code elimination Eric Wong
2021-03-16 8:48 6% ` [PATCH 1/2] nntp: remove unused header_append method Eric Wong
2022-08-08 23:53 [PATCH 0/5] daemon/netd-related improvements Eric Wong
2022-08-08 23:53 6% ` [PATCH 5/5] imapd|nntpd: drop ->base_url preload Eric Wong
2022-08-20 8:01 [PATCH 0/5] various minor annoyance fixups Eric Wong
2022-08-20 8:01 7% ` [PATCH 5/5] www: mbox* drop unneeded {base_url} memoizations Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).