* [PATCH 5/5] nntpd: remove redundant {groups} shortcut
2020-11-28 5:09 6% [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
@ 2020-11-28 5:09 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-11-28 5:09 UTC (permalink / raw)
To: meta
It's not worth confusing hackers reading the source to have
two ways to access the same (large) hash table. So just
go through PublicInbox::Config objects for now since the
extra hash lookup isn't going to be noticeable.
I've also started favoring "for" instead of "foreach"
since they're the equivalent perlop and less wear on
my fingers + keyboard.
---
lib/PublicInbox/NNTP.pm | 12 ++++++------
lib/PublicInbox/NNTPD.pm | 3 +--
2 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index d314a3d1..3b16a66a 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -321,7 +321,7 @@ sub ngpat2re (;$) {
sub newnews_i {
my ($self, $names, $ts, $prev) = @_;
my $ngname = $names->[0];
- if (my $ibx = $self->{nntpd}->{groups}->{$ngname}) {
+ if (my $ibx = $self->{nntpd}->{pi_config}->{-by_newsgroup}->{$ngname}) {
if (my $over = $ibx->over) {
my $msgs = $over->query_ts($ts, $$prev);
if (scalar @$msgs) {
@@ -360,13 +360,13 @@ sub cmd_newnews ($$$$;$$) {
sub cmd_group ($$) {
my ($self, $group) = @_;
- my $no_such = '411 no such news group';
my $nntpd = $self->{nntpd};
- my $ng = $nntpd->{groups}->{$group} or return $no_such;
+ my $ibx = $nntpd->{pi_config}->{-by_newsgroup}->{$group} or
+ return '411 no such news group';
$nntpd->idler_start;
- $self->{ng} = $ng;
- my ($min, $max) = $ng->mm->minmax;
+ $self->{ng} = $ibx;
+ my ($min, $max) = $ibx->mm->minmax;
$self->{article} = $min;
my $est_size = $max - $min;
"211 $est_size $min $max $group";
@@ -743,7 +743,7 @@ EOF
}
# no warning here, $mid is just invalid
} else { # slow path for non-ALL users
- foreach my $ibx (values %{$self->{nntpd}->{groups}}) {
+ for my $ibx (values %{$pi_cfg->{-by_newsgroup}}) {
next if defined $self_ng && $ibx eq $self_ng;
my $n = $ibx->mm->num_for($mid);
return ($ibx, $n) if defined $n;
diff --git a/lib/PublicInbox/NNTPD.pm b/lib/PublicInbox/NNTPD.pm
index 33bc5fda..5e287857 100644
--- a/lib/PublicInbox/NNTPD.pm
+++ b/lib/PublicInbox/NNTPD.pm
@@ -60,9 +60,8 @@ sub refresh_groups {
}
});
$self->{groupnames} = [ sort(keys %$groups) ];
- $self->{pi_config} = $pi_config;
# this will destroy old groups that got deleted
- $self->{groups} = $groups;
+ $self->{pi_config} = $pi_config;
}
sub idler_start {
^ permalink raw reply related [relevance 7%]
* [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups
@ 2020-11-28 5:09 6% Eric Wong
2020-11-28 5:09 7% ` [PATCH 5/5] nntpd: remove redundant {groups} shortcut Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-11-28 5:09 UTC (permalink / raw)
To: meta
All of the O(n) iterations through newsgroups can either go
through the ->ALL extindex or is broken out into long_response
to not monopolize event loops.
So 50-100K newsgroups ought to be usable...
Further speeding up NEWGROUPS will require some MiscIdx
indexing additions, but that no longer hogs up an event
loop iteration.
One remaining problems is startup time with 50-100K newsgroups;
and that affects everything (especially -mda...)
Eric Wong (5):
nntp: NEWGROUPS uses long_response
nntp: speed up mid_lookup() using ->ALL extindex
nntp: art_lookup: use mid_lookup and simplify
nntp: XPATH uses ->ALL extindex, too
nntpd: remove redundant {groups} shortcut
lib/PublicInbox/NNTP.pm | 125 +++++++++++++++++++++++++--------------
lib/PublicInbox/NNTPD.pm | 8 +--
t/extsearch.t | 4 +-
t/nntp.t | 1 -
4 files changed, 87 insertions(+), 51 deletions(-)
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-11-28 5:09 6% [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
2020-11-28 5:09 7% ` [PATCH 5/5] nntpd: remove redundant {groups} shortcut Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).