* [PATCH 18/18] cindex: support --associate-aggressive shortcut
2023-11-13 13:15 7% [PATCH 00/18] cindex: some --associate work Eric Wong
@ 2023-11-13 13:15 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2023-11-13 13:15 UTC (permalink / raw)
To: meta
This is shorthand to enabling --associate with the most
aggressive (and time-consuming) options available, starting from
the Unix epoch and having an unlimited window to join on.
---
lib/PublicInbox/CodeSearchIdx.pm | 5 +++++
script/public-inbox-cindex | 1 +
2 files changed, 6 insertions(+)
diff --git a/lib/PublicInbox/CodeSearchIdx.pm b/lib/PublicInbox/CodeSearchIdx.pm
index 54ddb68e..4ed5ea64 100644
--- a/lib/PublicInbox/CodeSearchIdx.pm
+++ b/lib/PublicInbox/CodeSearchIdx.pm
@@ -1146,6 +1146,11 @@ sub cidx_run { # main entry point
local $self->{ASSOC_PFX} = \@ASSOC_PFX;
local $self->{PENDING} = {};
local $self->{-pi_cfg};
+ if ($self->{-opt}->{'associate-aggressive'}) { # shortcut
+ $self->{-opt}->{'associate-date-range'} //= '19700101000000..';
+ $self->{-opt}->{'associate-window'} //= -1;
+ $self->{-opt}->{associate} //= 1;
+ }
if (grep { $_ } @{$self->{-opt}}{qw(prune associate)}) {
require File::Temp;
$TMPDIR = File::Temp->newdir('cidx-all-git-XXXX', TMPDIR => 1);
diff --git a/script/public-inbox-cindex b/script/public-inbox-cindex
index b8133806..feb4a7f4 100755
--- a/script/public-inbox-cindex
+++ b/script/public-inbox-cindex
@@ -27,6 +27,7 @@ EOF
my $opt = { fsync => 1, scan => 1 }; # --no-scan is hidden
GetOptions($opt, qw(quiet|q verbose|v+ reindex jobs|j=i fsync|sync! dangerous
indexlevel|index-level|L=s associate associate-window=i
+ associate-aggressive
associate-date-range=s associate-prefixes=s@
batch_size|batch-size=s max_size|max-size=s
include|I=s@ only=s@ all show-roots
^ permalink raw reply related [relevance 6%]
* [PATCH 00/18] cindex: some --associate work
@ 2023-11-13 13:15 7% Eric Wong
2023-11-13 13:15 6% ` [PATCH 18/18] cindex: support --associate-aggressive shortcut Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2023-11-13 13:15 UTC (permalink / raw)
To: meta
Still very much in flux, but some treewide cleanups in there...
And I've been wondering if "join" is a better word than
"associate" to denote the relationship between inboxes
and coderepos.
But "join" (even if we use join(1) internally) probably
implies strict relationships, whereas our current "associate"
is always going to be fuzzy due to patchids being fuzzy
and blobs OIDs being abbreviated in patches.
I'm also thinking about moving --associate-* CLI switches
into suboptions (e.g. what getsubopt(3) supports), so:
--associate=aggressive,prefixes=patchid+dfblob
But Perl doesn't ship with getsubopt(3) emulation
out-of-the-box
Eric Wong (18):
cindex: check `say' errors w/ close or ->flush
tmpfile: check `stat' errors, use autodie for unlink
cindex: use `local' for pipes between processes
xap_helper_cxx: use write_file helper
xap_helper_cxx: make the build process ccache-friendly
xap_helper_cxx: use -pipe by default in CXXFLAGS
xap_client: spawn C++ xap_helper directly
treewide: update read_all to avoid eof|close checks
spawn: don't append to scalarrefs on stdout/stderr
cindex: imply --all with --associate w/o -I/--only
cindex: delay associate until prune+indexing finish
xap_helper: Perl dump_ibx respects `-m MAX'
cidx_xap_helper_aux: complain about truncated inputs
xap_helper: stricter and harsher error handling
xap_helper: better variable naming for key buffer
cindex: do not guess integer maximum for Xapian
cindex: rename associate-max => window
cindex: support --associate-aggressive shortcut
lib/PublicInbox/CidxComm.pm | 6 +-
lib/PublicInbox/CidxXapHelperAux.pm | 6 +-
lib/PublicInbox/CodeSearchIdx.pm | 122 ++++++++++-----
lib/PublicInbox/Gcf2.pm | 3 +-
lib/PublicInbox/IO.pm | 18 ++-
lib/PublicInbox/LeiInput.pm | 10 +-
lib/PublicInbox/LeiMirror.pm | 10 +-
lib/PublicInbox/LeiToMail.pm | 3 +-
lib/PublicInbox/Spawn.pm | 4 +-
lib/PublicInbox/TestCommon.pm | 6 +-
lib/PublicInbox/Tmpfile.pm | 10 +-
lib/PublicInbox/XapClient.pm | 28 ++--
lib/PublicInbox/XapHelper.pm | 30 ++--
lib/PublicInbox/XapHelperCxx.pm | 55 +++----
lib/PublicInbox/xap_helper.h | 233 ++++++++++++----------------
script/public-inbox-cindex | 3 +-
script/public-inbox-learn | 2 +-
script/public-inbox-mda | 2 +-
script/public-inbox-purge | 2 +-
t/spawn.t | 2 +-
t/xap_helper.t | 27 ++--
21 files changed, 287 insertions(+), 295 deletions(-)
Yay, less code!
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2023-11-13 13:15 7% [PATCH 00/18] cindex: some --associate work Eric Wong
2023-11-13 13:15 6% ` [PATCH 18/18] cindex: support --associate-aggressive shortcut Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).