* [PATCH 04/12] nntp: use grep operation for wildmat matching
2020-11-27 9:52 6% [PATCH 00/12] some NNTP-related fixes + speedups Eric Wong
@ 2020-11-27 9:52 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-11-27 9:52 UTC (permalink / raw)
To: meta
Based on experiences with the IMAP server, this ought to be
significantly faster (as to be demonstrated in the next
commit).
---
lib/PublicInbox/NNTP.pm | 22 +++++++++++-----------
lib/PublicInbox/NNTPD.pm | 4 +++-
2 files changed, 14 insertions(+), 12 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index b641bd23..68222196 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -136,29 +136,29 @@ sub list_headers ($;$) {
sub list_active ($;$) {
my ($self, $wildmat) = @_;
wildmat2re($wildmat);
- foreach my $ng (@{$self->{nntpd}->{grouplist}}) {
- $ng->{newsgroup} =~ $wildmat or next;
- group_line($self, $ng);
+ my $groups = $self->{nntpd}->{groups};
+ for my $ngname (grep(/$wildmat/, @{$self->{nntpd}->{groupnames}})) {
+ group_line($self, $groups->{$ngname});
}
}
sub list_active_times ($;$) {
my ($self, $wildmat) = @_;
wildmat2re($wildmat);
- foreach my $ng (@{$self->{nntpd}->{grouplist}}) {
- $ng->{newsgroup} =~ $wildmat or next;
- my $c = eval { $ng->uidvalidity } // time;
- more($self, "$ng->{newsgroup} $c $ng->{-primary_address}");
+ my $groups = $self->{nntpd}->{groups};
+ for my $ngname (grep(/$wildmat/, @{$self->{nntpd}->{groupnames}})) {
+ my $ibx = $groups->{$ngname};
+ my $c = eval { $ibx->uidvalidity } // time;
+ more($self, "$ngname $c $ibx->{-primary_address}");
}
}
sub list_newsgroups ($;$) {
my ($self, $wildmat) = @_;
wildmat2re($wildmat);
- foreach my $ng (@{$self->{nntpd}->{grouplist}}) {
- $ng->{newsgroup} =~ $wildmat or next;
- my $d = $ng->description;
- more($self, "$ng->{newsgroup} $d");
+ my $groups = $self->{nntpd}->{groups};
+ for my $ngname (grep(/$wildmat/, @{$self->{nntpd}->{groupnames}})) {
+ more($self, "$ngname ".$groups->{$ngname}->description);
}
}
diff --git a/lib/PublicInbox/NNTPD.pm b/lib/PublicInbox/NNTPD.pm
index 13b0f678..4de1944b 100644
--- a/lib/PublicInbox/NNTPD.pm
+++ b/lib/PublicInbox/NNTPD.pm
@@ -60,7 +60,9 @@ sub refresh_groups {
delete $groups->{$ngname};
}
});
- $self->{grouplist} = [ map { $groups->{$_} } sort(keys %$groups) ];
+ my @names = sort(keys %$groups);
+ $self->{grouplist} = [ map { $groups->{$_} } @names ];
+ $self->{groupnames} = \@names;
$self->{pi_config} = $pi_config;
# this will destroy old groups that got deleted
$self->{groups} = $groups;
^ permalink raw reply related [relevance 7%]
* [PATCH 00/12] some NNTP-related fixes + speedups
@ 2020-11-27 9:52 6% Eric Wong
2020-11-27 9:52 7% ` [PATCH 04/12] nntp: use grep operation for wildmat matching Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-11-27 9:52 UTC (permalink / raw)
To: meta
A few minor bugfixes and some major speedups for targeting
thousands/millions of inboxes based on stuff learned during
IMAP server development.
[PATCH 6/12] "miscsearch: implement ->newsgroup_matches"
may not be necessary, after all, but maybe it could also
be used for WWW.
Still a lot more to do...
Eric Wong (12):
nntp: use Inbox->uidvalidity instead of ->mm->created_at
nntpd: share {groups} hash with {-by_newsgroup} in Config
mm: min/max: return 0 instead of undef
nntp: use grep operation for wildmat matching
nntp: NEWNEWS: speed up filtering
miscsearch: implement ->newsgroup_matches
nntp: LIST ACTIVE.TIMES use angle brackets around address
nntp: move LIST iterators to long_response
t/extsearch: show a more realistic case
nntp: some minor golfing
nntp: xref: simplify sub signature
nntp: xref: use ->ALL extindex if available
lib/PublicInbox/Config.pm | 4 +-
lib/PublicInbox/ExtSearch.pm | 56 ++++++++++
lib/PublicInbox/IMAPD.pm | 2 +-
lib/PublicInbox/MiscSearch.pm | 40 +++++++
lib/PublicInbox/Msgmap.pm | 7 +-
lib/PublicInbox/NNTP.pm | 200 +++++++++++++++++++---------------
lib/PublicInbox/NNTPD.pm | 25 +++--
lib/PublicInbox/OverIdx.pm | 5 +
lib/PublicInbox/V2Writable.pm | 4 +-
t/extsearch.t | 54 ++++++++-
t/msgmap.t | 2 +-
t/nntp.t | 7 +-
12 files changed, 291 insertions(+), 115 deletions(-)
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-11-27 9:52 6% [PATCH 00/12] some NNTP-related fixes + speedups Eric Wong
2020-11-27 9:52 7% ` [PATCH 04/12] nntp: use grep operation for wildmat matching Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).