* [PATCH 4/5] searchidx: rename get_val to int_val and return IV
2020-12-21 7:51 6% [PATCH 0/5] extindex->misc prep stuff Eric Wong
@ 2020-12-21 7:51 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-12-21 7:51 UTC (permalink / raw)
To: meta
Values can be strings in Xapian, although we currently use
integer values exclusively. Give the wrapper a more appropriate
name in case we start using string columns.
For future-proofing, we'll now return `undef' on missing columns
and coerce the return value to an IV (integer value) to save
memory, as sortable_unserialise returns a PV (pointer value)
scalar despite it existing to support numeric values.
---
lib/PublicInbox/SearchIdx.pm | 9 +++++----
t/search.t | 4 ++--
2 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
index b731f698..cf2c2c55 100644
--- a/lib/PublicInbox/SearchIdx.pm
+++ b/lib/PublicInbox/SearchIdx.pm
@@ -501,17 +501,18 @@ sub remove_eidx_info {
$self->{xdb}->replace_document($docid, $doc);
}
-sub get_val ($$) {
+sub int_val ($$) {
my ($doc, $col) = @_;
- sortable_unserialise($doc->get_value($col));
+ my $val = $doc->get_value($col) or return; # undefined is '' in Xapian
+ sortable_unserialise($val) + 0; # PV => IV conversion
}
sub smsg_from_doc ($) {
my ($doc) = @_;
my $data = $doc->get_data or return;
my $smsg = bless {}, 'PublicInbox::Smsg';
- $smsg->{ts} = get_val($doc, PublicInbox::Search::TS());
- my $dt = get_val($doc, PublicInbox::Search::DT());
+ $smsg->{ts} = int_val($doc, PublicInbox::Search::TS());
+ my $dt = int_val($doc, PublicInbox::Search::DT());
my ($yyyy, $mon, $dd, $hh, $mm, $ss) = unpack('A4A2A2A2A2A2', $dt);
$smsg->{ds} = timegm($ss, $mm, $hh, $dd, $mon - 1, $yyyy);
$smsg->load_from_data($data);
diff --git a/t/search.t b/t/search.t
index da9acb07..11143204 100644
--- a/t/search.t
+++ b/t/search.t
@@ -332,13 +332,13 @@ $ibx->with_umask(sub {
like($smsg->{to}, qr/\blist\@example\.com\b/, 'to appears');
my $doc = $m->get_document;
my $col = PublicInbox::Search::BYTES();
- my $bytes = PublicInbox::SearchIdx::get_val($doc, $col);
+ my $bytes = PublicInbox::SearchIdx::int_val($doc, $col);
like($bytes, qr/\A[0-9]+\z/, '$bytes stored as digit');
ok($bytes > 0, '$bytes is > 0');
is($bytes, $smsg->{bytes}, 'bytes Xapian value matches Over');
$col = PublicInbox::Search::UID();
- my $uid = PublicInbox::SearchIdx::get_val($doc, $col);
+ my $uid = PublicInbox::SearchIdx::int_val($doc, $col);
is($uid, $smsg->{num}, 'UID column matches {num}');
is($uid, $m->get_docid, 'UID column matches docid');
}
^ permalink raw reply related [relevance 7%]
* [PATCH 0/5] extindex->misc prep stuff
@ 2020-12-21 7:51 6% Eric Wong
2020-12-21 7:51 7% ` [PATCH 4/5] searchidx: rename get_val to int_val and return IV Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-12-21 7:51 UTC (permalink / raw)
To: meta
Still working on speedups to startup time and no-op index
runs on thousands of inboxes, but a few small cleanups
and improvements along the way...
Eric Wong (5):
inbox: delay ->version detection
isearch: use numeric sort for article numbers
use rel2abs_collapsed when loading Inbox objects
searchidx: rename get_val to int_val and return IV
extsearch*: drop unnecessary path canonicalization
lib/PublicInbox/Admin.pm | 11 +----------
lib/PublicInbox/Config.pm | 28 +++++++++++++++++++++++-----
lib/PublicInbox/ExtSearch.pm | 2 --
lib/PublicInbox/ExtSearchIdx.pm | 7 -------
lib/PublicInbox/Inbox.pm | 8 +++-----
lib/PublicInbox/Isearch.pm | 4 ++--
lib/PublicInbox/SearchIdx.pm | 9 +++++----
script/public-inbox-convert | 2 +-
script/public-inbox-init | 2 +-
t/search.t | 4 ++--
10 files changed, 38 insertions(+), 39 deletions(-)
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-12-21 7:51 6% [PATCH 0/5] extindex->misc prep stuff Eric Wong
2020-12-21 7:51 7% ` [PATCH 4/5] searchidx: rename get_val to int_val and return IV Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).