* [PATCH 3/5] nntp: art_lookup: use mid_lookup and simplify
2020-11-28 5:09 7% [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
@ 2020-11-28 5:09 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-11-28 5:09 UTC (permalink / raw)
To: meta
This lets us take advantage of mid_lookup speedup from the
previous commit.
While we're at it, start moving towards using `$ibx' as the
abbreviation for PublicInbox::Inbox objects even in the NNTP
code, since they've been shared with the WWW code for several
years, now.
---
lib/PublicInbox/NNTP.pm | 36 ++++++++++++------------------------
1 file changed, 12 insertions(+), 24 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 7b3b1ffe..c014eff0 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -483,42 +483,30 @@ sub set_nntp_headers ($$) {
sub art_lookup ($$$) {
my ($self, $art, $code) = @_;
- my $ng = $self->{ng};
- my ($n, $mid);
+ my ($ibx, $n);
my $err;
if (defined $art) {
if ($art =~ /\A[0-9]+\z/) {
$err = '423 no such article number in this group';
$n = int($art);
- goto find_mid;
+ goto find_ibx;
} elsif ($art =~ $ONE_MSGID) {
- $mid = $1;
- $err = r430;
- $n = $ng->mm->num_for($mid) if $ng;
- goto found if defined $n;
- foreach my $g (values %{$self->{nntpd}->{groups}}) {
- $n = $g->mm->num_for($mid);
- if (defined $n) {
- $ng = $g;
- goto found;
- }
- }
- return $err;
+ ($ibx, $n) = mid_lookup($self, $1);
+ goto found if $ibx;
+ return r430;
} else {
return r501;
}
} else {
$err = '420 no current article has been selected';
- $n = $self->{article};
- defined $n or return $err;
-find_mid:
- $ng or return '412 no newsgroup has been selected';
- $mid = $ng->mm->mid_for($n);
- defined $mid or return $err;
+ $n = $self->{article} // return $err;
+find_ibx:
+ $ibx = $self->{ng} or
+ return '412 no newsgroup has been selected';
}
found:
- my $smsg = $ng->over->get_art($n) or return $err;
- $smsg->{-ibx} = $ng;
+ my $smsg = $ibx->over->get_art($n) or return $err;
+ $smsg->{-ibx} = $ibx;
if ($code == 223) { # STAT
set_art($self, $n);
"223 $n <$smsg->{mid}> article retrieved - " .
@@ -528,7 +516,7 @@ found:
$smsg->{nntp_code} = $code;
set_art($self, $art);
# this dereferences to `undef'
- ${git_async_cat($ng->git, $smsg->{blob}, \&blob_cb, $smsg)};
+ ${git_async_cat($ibx->git, $smsg->{blob}, \&blob_cb, $smsg)};
}
}
^ permalink raw reply related [relevance 6%]
* [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups
@ 2020-11-28 5:09 7% Eric Wong
2020-11-28 5:09 6% ` [PATCH 3/5] nntp: art_lookup: use mid_lookup and simplify Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-11-28 5:09 UTC (permalink / raw)
To: meta
All of the O(n) iterations through newsgroups can either go
through the ->ALL extindex or is broken out into long_response
to not monopolize event loops.
So 50-100K newsgroups ought to be usable...
Further speeding up NEWGROUPS will require some MiscIdx
indexing additions, but that no longer hogs up an event
loop iteration.
One remaining problems is startup time with 50-100K newsgroups;
and that affects everything (especially -mda...)
Eric Wong (5):
nntp: NEWGROUPS uses long_response
nntp: speed up mid_lookup() using ->ALL extindex
nntp: art_lookup: use mid_lookup and simplify
nntp: XPATH uses ->ALL extindex, too
nntpd: remove redundant {groups} shortcut
lib/PublicInbox/NNTP.pm | 125 +++++++++++++++++++++++++--------------
lib/PublicInbox/NNTPD.pm | 8 +--
t/extsearch.t | 4 +-
t/nntp.t | 1 -
4 files changed, 87 insertions(+), 51 deletions(-)
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-11-28 5:09 7% [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
2020-11-28 5:09 6% ` [PATCH 3/5] nntp: art_lookup: use mid_lookup and simplify Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).