* [PATCH 12/12] *search: simplify retry_reopen users
2020-11-23 7:05 6% [PATCH 00/12] extindex: speed up manifest.js.gz generation Eric Wong
@ 2020-11-23 7:06 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-11-23 7:06 UTC (permalink / raw)
To: meta
Every callback uses `$self', and creating short-lived
array references is not necessary when it's just as
easy to copy the array in Perl (unlike C).
---
lib/PublicInbox/MiscSearch.pm | 8 ++++----
lib/PublicInbox/Search.pm | 10 +++++-----
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/lib/PublicInbox/MiscSearch.pm b/lib/PublicInbox/MiscSearch.pm
index 5a44d751..48ef6914 100644
--- a/lib/PublicInbox/MiscSearch.pm
+++ b/lib/PublicInbox/MiscSearch.pm
@@ -50,7 +50,7 @@ sub mi_qp_new ($) {
}
sub misc_enquire_once { # retry_reopen callback
- my ($self, $qr, $opt) = @{$_[0]};
+ my ($self, $qr, $opt) = @_;
my $eq = $PublicInbox::Search::X{Enquire}->new($self->{xdb});
$eq->set_query($qr);
my $desc = !$opt->{asc};
@@ -73,11 +73,11 @@ sub mset {
$qs = 'type:inbox' if $qs eq '';
my $qr = $qp->parse_query($qs, $PublicInbox::Search::QP_FLAGS);
$opt->{relevance} = 1 unless exists $opt->{relevance};
- retry_reopen($self, \&misc_enquire_once, [ $self, $qr, $opt ]);
+ retry_reopen($self, \&misc_enquire_once, $qr, $opt);
}
sub ibx_data_once {
- my ($self, $ibx) = @{$_[0]};
+ my ($self, $ibx) = @_;
my $xdb = $self->{xdb};
my $eidx_key = $ibx->eidx_key; # may be {inboxdir}, so private
my $head = $xdb->postlist_begin('Q'.$eidx_key);
@@ -92,7 +92,7 @@ sub ibx_data_once {
sub inbox_data {
my ($self, $ibx) = @_;
- retry_reopen($self, \&ibx_data_once, [ $self, $ibx ]);
+ retry_reopen($self, \&ibx_data_once, $ibx);
}
1;
diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm
index 05d5a133..574bc145 100644
--- a/lib/PublicInbox/Search.pm
+++ b/lib/PublicInbox/Search.pm
@@ -291,15 +291,15 @@ sub mset {
}
sub retry_reopen {
- my ($self, $cb, $arg) = @_;
+ my ($self, $cb, @arg) = @_;
for my $i (1..10) {
if (wantarray) {
my @ret;
- eval { @ret = $cb->($arg) };
+ eval { @ret = $cb->($self, @arg) };
return @ret unless $@;
} else {
my $ret;
- eval { $ret = $cb->($arg) };
+ eval { $ret = $cb->($self, @arg) };
return $ret unless $@;
}
# Exception: The revision being read has been discarded -
@@ -319,7 +319,7 @@ sub retry_reopen {
sub _do_enquire {
my ($self, $query, $opts) = @_;
- retry_reopen($self, \&_enquire_once, [ $self, $query, $opts ]);
+ retry_reopen($self, \&_enquire_once, $query, $opts);
}
# returns true if all docs have the THREADID value
@@ -329,7 +329,7 @@ sub has_threadid ($) {
}
sub _enquire_once { # retry_reopen callback
- my ($self, $query, $opts) = @{$_[0]};
+ my ($self, $query, $opts) = @_;
my $xdb = xdb($self);
my $enquire = $X{Enquire}->new($xdb);
$enquire->set_query($query);
^ permalink raw reply related [relevance 7%]
* [PATCH 00/12] extindex: speed up manifest.js.gz generation
@ 2020-11-23 7:05 6% Eric Wong
2020-11-23 7:06 7% ` [PATCH 12/12] *search: simplify retry_reopen users Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-11-23 7:05 UTC (permalink / raw)
To: meta
manifest.js.gz generation gets faster with this series
(~1000ms => ~40ms) on the current set of lore.kernel.org inboxes
We may need to rely on varnish to handle things up to 30-100K
inboxes, since manifest.js.gz generation won't monopolize the
-httpd event loop.
WwwListing (HTML) output still needs to be updated and searching
for inboxes needs to be implemented along with pagination for
30-100K inboxes.
Eric Wong (12):
miscsearch: a new Xapian sub-DB for extindex
move JSON module portability into PublicInbox::Config
git: add manifest_entry method
manifest: use ibx->git_epoch method for v2
inbox: git_epoch: remove ->version check
miscidx: put grokmirror manifest entries in Xapian docdata
extsearch: fix remaining "eindex" references
miscidx: cleanup git processes after manifest indexing
miscidx: store absolute git_dir of each epoch in docdata
extsearchidx: do not short-circuit MiscIdx on no-op v2 prepare
manifest: support faster generation via [extindex "all"]
*search: simplify retry_reopen users
MANIFEST | 3 +
lib/PublicInbox/Config.pm | 15 ++++
lib/PublicInbox/ExtSearch.pm | 8 +-
lib/PublicInbox/ExtSearchIdx.pm | 18 ++++-
lib/PublicInbox/Git.pm | 53 +++++++++++++
lib/PublicInbox/Inbox.pm | 6 +-
lib/PublicInbox/InboxWritable.pm | 2 -
lib/PublicInbox/ManifestJsGz.pm | 108 +++++++++-----------------
lib/PublicInbox/MiscIdx.pm | 125 +++++++++++++++++++++++++++++++
lib/PublicInbox/MiscSearch.pm | 98 ++++++++++++++++++++++++
lib/PublicInbox/Search.pm | 18 ++---
lib/PublicInbox/SearchIdx.pm | 7 +-
lib/PublicInbox/V2Writable.pm | 5 ++
script/public-inbox-extindex | 1 +
t/extsearch.t | 14 +++-
t/miscsearch.t | 57 ++++++++++++++
t/www_listing.t | 5 +-
17 files changed, 446 insertions(+), 97 deletions(-)
create mode 100644 lib/PublicInbox/MiscIdx.pm
create mode 100644 lib/PublicInbox/MiscSearch.pm
create mode 100644 t/miscsearch.t
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-11-23 7:05 6% [PATCH 00/12] extindex: speed up manifest.js.gz generation Eric Wong
2020-11-23 7:06 7% ` [PATCH 12/12] *search: simplify retry_reopen users Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).