* Re: [PATCH 13/15] www_listing: support publicInbox.nameIsUrl
2023-11-30 11:41 5% ` [PATCH 13/15] www_listing: support publicInbox.nameIsUrl Eric Wong
@ 2023-12-01 1:29 7% ` Kyle Meyer
0 siblings, 0 replies; 3+ results
From: Kyle Meyer @ 2023-12-01 1:29 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
Eric Wong writes:
> This is a convenient (and slightly memory-saving) alternative to
> specifying a `publicinbox.*.url' entry for every single inbox
> when using publicinbox.wwwListing.
Nice.
> +=item publicinbox.nameIsUrl
> +
> +Treat the name of the public inbox as it's unqualified URL when
s/it's/its/
^ permalink raw reply [relevance 7%]
* [PATCH 13/15] www_listing: support publicInbox.nameIsUrl
2023-11-30 11:40 5% [PATCH 00/15] various cindex fixes + speedups Eric Wong
@ 2023-11-30 11:41 5% ` Eric Wong
2023-12-01 1:29 7% ` Kyle Meyer
0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2023-11-30 11:41 UTC (permalink / raw)
To: meta
This is a convenient (and slightly memory-saving) alternative to
specifying a `publicinbox.*.url' entry for every single inbox
when using publicinbox.wwwListing.
---
Documentation/public-inbox-config.pod | 19 ++++++++++++++++++-
lib/PublicInbox/WwwListing.pm | 21 +++++++++++++--------
2 files changed, 31 insertions(+), 9 deletions(-)
diff --git a/Documentation/public-inbox-config.pod b/Documentation/public-inbox-config.pod
index d394d31f..d2017704 100644
--- a/Documentation/public-inbox-config.pod
+++ b/Documentation/public-inbox-config.pod
@@ -273,7 +273,9 @@ Default: 25
A comma-delimited list of listings to hide the inbox from.
-Valid values are currently C<www> and C<manifest>.
+Valid values are currently C<www> and C<manifest> for non-C<404>
+values of L</publicinbox.wwwListing> and L</publicinbox.grokManifest>,
+respectively
Default: none
@@ -379,6 +381,21 @@ TODO support showing cgit listing
Default: C<404>
+=item publicinbox.nameIsUrl
+
+Treat the name of the public inbox as it's unqualified URL when
+using C<publicInbox.wwwListing=all>. This is, every
+C<[publicinbox "foo"]> section implicitly sets C<publicinbox.foo.url=foo>.
+
+This is a convenient alternative to specifying
+C<publicinbox.E<lt>nameE<gt>.url> for every single inbox if
+your inbox URLs are domain-agnostic when using
+C<publicInbox.wwwListing=all>
+
+Default: false
+
+New in public-inbox 2.0.0 (PENDING).
+
=item publicinbox.grokmanifest
Controls the generation of a grokmirror-compatible gzipped JSON file
diff --git a/lib/PublicInbox/WwwListing.pm b/lib/PublicInbox/WwwListing.pm
index 21e5b8bc..e3d2e84c 100644
--- a/lib/PublicInbox/WwwListing.pm
+++ b/lib/PublicInbox/WwwListing.pm
@@ -41,10 +41,7 @@ sub list_match_i { # ConfigIter callback
if (defined($section)) {
return if $section !~ m!\Apublicinbox\.([^/]+)\z!;
my $ibx = $cfg->lookup_name($1) or return;
- if (!$ibx->{-hide}->{$ctx->hide_key} &&
- grep(/$re/, @{$ibx->{url} // []})) {
- $ctx->ibx_entry($ibx);
- }
+ $ctx->ibx_entry($ibx) unless $ctx->hide_inbox($ibx, $re);
} else { # undef == "EOF"
$ctx->{-wcb}->($ctx->psgi_triple);
}
@@ -54,13 +51,17 @@ sub url_filter {
my ($ctx, $key, $default) = @_;
$key //= 'publicInbox.wwwListing';
$default //= '404';
- my $v = $ctx->{www}->{pi_cfg}->{lc $key} // $default;
+ my $cfg = $ctx->{www}->{pi_cfg};
+ my $v = $cfg->{lc $key} // $default;
again:
if ($v eq 'match=domain') {
my $h = $ctx->{env}->{HTTP_HOST} // $ctx->{env}->{SERVER_NAME};
$h =~ s/:[0-9]+\z//;
(qr!\A(?:https?:)?//\Q$h\E(?::[0-9]+)?/!i, "url:$h");
} elsif ($v eq 'all') {
+ my $niu = $cfg->{lc 'publicinbox.nameIsUrl'};
+ defined($niu) && $cfg->git_bool($niu) and
+ $ctx->{-name_is_url} = [ '.' ];
(qr/./, undef);
} elsif ($v eq '404') {
(undef, undef);
@@ -76,6 +77,12 @@ EOF
sub hide_key { 'www' }
+sub hide_inbox {
+ my ($ctx, $ibx, $re) = @_;
+ $ibx->{-hide}->{$ctx->hide_key} ||
+ !grep(/$re/, @{$ibx->{url} // $ctx->{-name_is_url} // []})
+}
+
sub add_misc_ibx { # MiscSearch->retry_reopen callback
my ($misc, $ctx, $re, $qs) = @_;
require PublicInbox::SearchQuery;
@@ -104,15 +111,13 @@ sub add_misc_ibx { # MiscSearch->retry_reopen callback
$ctx->ibx_entry($pi_cfg->ALL // die('BUG: ->ALL expected'), {});
}
my $mset = $misc->mset($qs, $opt); # sorts by $MODIFIED (mtime)
- my $hide_key = $ctx->hide_key;
for my $mi ($mset->items) {
my $doc = $mi->get_document;
my ($eidx_key) = PublicInbox::Search::xap_terms('Q', $doc);
$eidx_key // next;
my $ibx = $pi_cfg->lookup_eidx_key($eidx_key) // next;
- next if $ibx->{-hide}->{$hide_key};
- grep(/$re/, @{$ibx->{url} // []}) or next;
+ next if $ctx->hide_inbox($ibx, $re);
$ctx->ibx_entry($ibx, $misc->doc2ibx_cache_ent($doc));
if ($r) { # for descriptions in search_nav_bot
my $pct = PublicInbox::Search::get_pct($mi);
^ permalink raw reply related [relevance 5%]
* [PATCH 00/15] various cindex fixes + speedups
@ 2023-11-30 11:40 5% Eric Wong
2023-11-30 11:41 5% ` [PATCH 13/15] www_listing: support publicInbox.nameIsUrl Eric Wong
0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2023-11-30 11:40 UTC (permalink / raw)
To: meta
Notable changes:
10/15 provides a huge speedup which will hopefully make
future developments faster.
12/15 probably obsoletes libgit2 for extindex "all" users.
13/15 can save some memory with many inboxes while making
configuration easier.
Eric Wong (15):
cindex: fix store_repo+repo_stored on no-op
codesearch: allow inbox count to exceed matches
config: reject newlines consistently in dir names
cindex: only create {-cidx_err} field on failures
cindex: keep batch pipe for pruning SHA-256 repos
cindex: store extensions.objectFormat with repo data
git: share unlinked pack checking code with gcf2
cindex: skip getpid guard for most OnDestroy use
spawn: drop IO layer support from redirects
cindex: speed up initial scan setup phase
inbox: expire resources more aggressively
git_async_cat: use git from "all" extindex if possible
www_listing: support publicInbox.nameIsUrl
inbox: shrink data structures for publicinbox.*.hide
codesearch: use retry_reopen for WWW
Documentation/public-inbox-config.pod | 19 +-
lib/PublicInbox/CodeSearch.pm | 54 +++--
lib/PublicInbox/CodeSearchIdx.pm | 286 ++++++++++++++++----------
lib/PublicInbox/Config.pm | 32 ++-
lib/PublicInbox/Gcf2.pm | 16 +-
lib/PublicInbox/Git.pm | 27 +--
lib/PublicInbox/GitAsyncCat.pm | 8 +-
lib/PublicInbox/Inbox.pm | 32 +--
lib/PublicInbox/MailDiff.pm | 3 +-
lib/PublicInbox/SearchIdx.pm | 5 +-
lib/PublicInbox/Spawn.pm | 32 +--
lib/PublicInbox/WwwListing.pm | 21 +-
12 files changed, 303 insertions(+), 232 deletions(-)
^ permalink raw reply [relevance 5%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2023-11-30 11:40 5% [PATCH 00/15] various cindex fixes + speedups Eric Wong
2023-11-30 11:41 5% ` [PATCH 13/15] www_listing: support publicInbox.nameIsUrl Eric Wong
2023-12-01 1:29 7% ` Kyle Meyer
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).