From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 16/18] cindex: do not guess integer maximum for Xapian
Date: Mon, 13 Nov 2023 13:15:49 +0000 [thread overview]
Message-ID: <20231113131551.843230-17-e@80x24.org> (raw)
In-Reply-To: <20231113131551.843230-1-e@80x24.org>
We can return an array to allow the caller to omit the internal
`-m' arg entirely. We'll also allow any non-positive values to
mean there's no limit; and we'll defer the "unlimited" case to
the XapHelper implementation. This frees us of having to deal
with mismatches between Perl and Xapian if Xapian was compiled
with 64-bit docid support and we're stuck on a 32-bit Perl
build.
---
lib/PublicInbox/CodeSearchIdx.pm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/CodeSearchIdx.pm b/lib/PublicInbox/CodeSearchIdx.pm
index 04c514fe..8e6b921d 100644
--- a/lib/PublicInbox/CodeSearchIdx.pm
+++ b/lib/PublicInbox/CodeSearchIdx.pm
@@ -501,11 +501,10 @@ sub shard_commit { # via wq_io_do
send($op_p, "shard_done $self->{shard}", 0);
}
-sub assoc_max_init ($) {
+sub assoc_max_args ($) {
my ($self) = @_;
my $max = $self->{-opt}->{'associate-max'} // $ASSOC_MAX;
- $max = $ASSOC_MAX if !$max;
- $max < 0 ? ((2 ** 31) - 1) : $max;
+ $max <= 0 ? () : ('-m', $max);
}
sub start_xhc () {
@@ -538,7 +537,7 @@ sub dump_roots_start {
run_await(\@sort, $CMD_ENV, $sort_opt, \&cmd_done, $associate);
run_await(\@UNIQ_FOLD, $fold_env, $fold_opt, \&cmd_done, $associate);
my @arg = ((map { ('-A', $_) } @ASSOC_PFX), '-c',
- '-m', assoc_max_init($self), $root2id, $QRY_STR);
+ assoc_max_args($self), $root2id, $QRY_STR);
for my $d ($self->shard_dirs) {
pipe(my $err_r, my $err_w);
$XHC->mkreq([$sort_w, $err_w], qw(dump_roots -d), $d, @arg);
@@ -556,6 +555,8 @@ sub dump_ibx { # sends to xap_helper.h
my $srch = $ibx->isrch or return warn <<EOM;
W: $ekey not indexed for search
EOM
+ # note: we don't send associate_max_args to dump_ibx since we
+ # have to post-filter non-patch messages
my @cmd = ('dump_ibx', $srch->xh_args,
(map { ('-A', $_) } @ASSOC_PFX), $ibx_id, $QRY_STR);
pipe(my $r, my $w);
next prev parent reply other threads:[~2023-11-13 13:15 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-13 13:15 [PATCH 00/18] cindex: some --associate work Eric Wong
2023-11-13 13:15 ` [PATCH 01/18] cindex: check `say' errors w/ close or ->flush Eric Wong
2023-11-13 13:15 ` [PATCH 02/18] tmpfile: check `stat' errors, use autodie for unlink Eric Wong
2023-11-13 13:15 ` [PATCH 03/18] cindex: use `local' for pipes between processes Eric Wong
2023-11-13 13:15 ` [PATCH 04/18] xap_helper_cxx: use write_file helper Eric Wong
2023-11-13 13:15 ` [PATCH 05/18] xap_helper_cxx: make the build process ccache-friendly Eric Wong
2023-11-13 13:15 ` [PATCH 06/18] xap_helper_cxx: use -pipe by default in CXXFLAGS Eric Wong
2023-11-13 13:15 ` [PATCH 07/18] xap_client: spawn C++ xap_helper directly Eric Wong
2023-11-13 13:15 ` [PATCH 08/18] treewide: update read_all to avoid eof|close checks Eric Wong
2023-11-13 13:15 ` [PATCH 09/18] spawn: don't append to scalarrefs on stdout/stderr Eric Wong
2023-11-13 13:15 ` [PATCH 10/18] cindex: imply --all with --associate w/o -I/--only Eric Wong
2023-11-13 13:15 ` [PATCH 11/18] cindex: delay associate until prune+indexing finish Eric Wong
2023-11-13 13:15 ` [PATCH 12/18] xap_helper: Perl dump_ibx respects `-m MAX' Eric Wong
2023-11-13 13:15 ` [PATCH 13/18] cidx_xap_helper_aux: complain about truncated inputs Eric Wong
2023-11-13 13:15 ` [PATCH 14/18] xap_helper: stricter and harsher error handling Eric Wong
2023-11-13 13:15 ` [PATCH 15/18] xap_helper: better variable naming for key buffer Eric Wong
2023-11-13 13:15 ` Eric Wong [this message]
2023-11-13 13:15 ` [PATCH 17/18] cindex: rename associate-max => window Eric Wong
2023-11-13 13:15 ` [PATCH 18/18] cindex: support --associate-aggressive shortcut Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231113131551.843230-17-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).