user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [PATCH 5/5] nntpd: remove redundant {groups} shortcut
  2020-11-28  5:09  6% [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
@ 2020-11-28  5:09  7% ` Eric Wong
  0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-11-28  5:09 UTC (permalink / raw)
  To: meta

It's not worth confusing hackers reading the source to have
two ways to access the same (large) hash table.  So just
go through PublicInbox::Config objects for now since the
extra hash lookup isn't going to be noticeable.

I've also started favoring "for" instead of "foreach"
since they're the equivalent perlop and less wear on
my fingers + keyboard.
---
 lib/PublicInbox/NNTP.pm  | 12 ++++++------
 lib/PublicInbox/NNTPD.pm |  3 +--
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index d314a3d1..3b16a66a 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -321,7 +321,7 @@ sub ngpat2re (;$) {
 sub newnews_i {
 	my ($self, $names, $ts, $prev) = @_;
 	my $ngname = $names->[0];
-	if (my $ibx = $self->{nntpd}->{groups}->{$ngname}) {
+	if (my $ibx = $self->{nntpd}->{pi_config}->{-by_newsgroup}->{$ngname}) {
 		if (my $over = $ibx->over) {
 			my $msgs = $over->query_ts($ts, $$prev);
 			if (scalar @$msgs) {
@@ -360,13 +360,13 @@ sub cmd_newnews ($$$$;$$) {
 
 sub cmd_group ($$) {
 	my ($self, $group) = @_;
-	my $no_such = '411 no such news group';
 	my $nntpd = $self->{nntpd};
-	my $ng = $nntpd->{groups}->{$group} or return $no_such;
+	my $ibx = $nntpd->{pi_config}->{-by_newsgroup}->{$group} or
+		return '411 no such news group';
 	$nntpd->idler_start;
 
-	$self->{ng} = $ng;
-	my ($min, $max) = $ng->mm->minmax;
+	$self->{ng} = $ibx;
+	my ($min, $max) = $ibx->mm->minmax;
 	$self->{article} = $min;
 	my $est_size = $max - $min;
 	"211 $est_size $min $max $group";
@@ -743,7 +743,7 @@ EOF
 		}
 		# no warning here, $mid is just invalid
 	} else { # slow path for non-ALL users
-		foreach my $ibx (values %{$self->{nntpd}->{groups}}) {
+		for my $ibx (values %{$pi_cfg->{-by_newsgroup}}) {
 			next if defined $self_ng && $ibx eq $self_ng;
 			my $n = $ibx->mm->num_for($mid);
 			return ($ibx, $n) if defined $n;
diff --git a/lib/PublicInbox/NNTPD.pm b/lib/PublicInbox/NNTPD.pm
index 33bc5fda..5e287857 100644
--- a/lib/PublicInbox/NNTPD.pm
+++ b/lib/PublicInbox/NNTPD.pm
@@ -60,9 +60,8 @@ sub refresh_groups {
 		}
 	});
 	$self->{groupnames} = [ sort(keys %$groups) ];
-	$self->{pi_config} = $pi_config;
 	# this will destroy old groups that got deleted
-	$self->{groups} = $groups;
+	$self->{pi_config} = $pi_config;
 }
 
 sub idler_start {

^ permalink raw reply related	[relevance 7%]

* [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups
@ 2020-11-28  5:09  6% Eric Wong
  2020-11-28  5:09  7% ` [PATCH 5/5] nntpd: remove redundant {groups} shortcut Eric Wong
  0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-11-28  5:09 UTC (permalink / raw)
  To: meta

All of the O(n) iterations through newsgroups can either go
through the ->ALL extindex or is broken out into long_response
to not monopolize event loops.

So 50-100K newsgroups ought to be usable...

Further speeding up NEWGROUPS will require some MiscIdx
indexing additions, but that no longer hogs up an event
loop iteration.

One remaining problems is startup time with 50-100K newsgroups;
and that affects everything (especially -mda...)

Eric Wong (5):
  nntp: NEWGROUPS uses long_response
  nntp: speed up mid_lookup() using ->ALL extindex
  nntp: art_lookup: use mid_lookup and simplify
  nntp: XPATH uses ->ALL extindex, too
  nntpd: remove redundant {groups} shortcut

 lib/PublicInbox/NNTP.pm  | 125 +++++++++++++++++++++++++--------------
 lib/PublicInbox/NNTPD.pm |   8 +--
 t/extsearch.t            |   4 +-
 t/nntp.t                 |   1 -
 4 files changed, 87 insertions(+), 51 deletions(-)

^ permalink raw reply	[relevance 6%]

Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-11-28  5:09  6% [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
2020-11-28  5:09  7% ` [PATCH 5/5] nntpd: remove redundant {groups} shortcut Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).