* [PATCH 00/14] more indexing related improvements
@ 2020-08-10 2:11 5% Eric Wong
2020-08-10 2:12 7% ` [PATCH 10/14] searchidx: use singular `$opt' for consistency with v2 Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-08-10 2:11 UTC (permalink / raw)
To: meta
publicInbox.indexSequentialShard now works incrementally
-convert also learned all the options -index learned,
so it can be less painful on HDDs.
Eric Wong (14):
index: require --reindex when using --xapian-only
index: --sequential-shard works incrementally
doc: index: some more notes about latest changes
doc: add some notes around -xcpdb / -edit / -purge
index+xcpdb: improve SIG{INT,TERM,HUP,PIPE} behavior
msgmap: tmp_clone: simplify + meaningful filename
avoid File::Temp::tempfile in more places
admin: use a generic veriable name
index: cleanup internal variables
searchidx: use singular `$opt' for consistency with v2
convert: support new -index options
convert: speed up --help
convert: check ARGV more correctly
convert: set No_COW on copied SQLite files
Documentation/public-inbox-convert.pod | 19 ++++
Documentation/public-inbox-edit.pod | 14 +++
Documentation/public-inbox-index.pod | 68 +++++++------
Documentation/public-inbox-init.pod | 2 +-
Documentation/public-inbox-purge.pod | 14 +++
Documentation/public-inbox-xcpdb.pod | 15 ++-
lib/PublicInbox/Admin.pm | 71 ++++++++++++--
lib/PublicInbox/Msgmap.pm | 19 ++--
lib/PublicInbox/SearchIdx.pm | 34 +++----
lib/PublicInbox/V2Writable.pm | 77 ++++++++-------
lib/PublicInbox/Xapcmd.pm | 28 ++++--
script/public-inbox-convert | 131 ++++++++++++++++---------
script/public-inbox-index | 69 ++++---------
script/public-inbox-init | 17 ++--
t/import.t | 5 +-
15 files changed, 357 insertions(+), 226 deletions(-)
^ permalink raw reply [relevance 5%]
* [PATCH 10/14] searchidx: use singular `$opt' for consistency with v2
2020-08-10 2:11 5% [PATCH 00/14] more indexing related improvements Eric Wong
@ 2020-08-10 2:12 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-08-10 2:12 UTC (permalink / raw)
To: meta
The rest of our indexing code uses `$opt' instead of `$opts'.
---
lib/PublicInbox/SearchIdx.pm | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
index 7f2447fe..5c39f3d6 100644
--- a/lib/PublicInbox/SearchIdx.pm
+++ b/lib/PublicInbox/SearchIdx.pm
@@ -539,11 +539,11 @@ sub unindex_both { # git->cat_async callback
# called by public-inbox-index
sub index_sync {
- my ($self, $opts) = @_;
- delete $self->{lock_path} if $opts->{-skip_lock};
- $self->{ibx}->with_umask(\&_index_sync, $self, $opts);
- if ($opts->{reindex}) {
- my %again = %$opts;
+ my ($self, $opt) = @_;
+ delete $self->{lock_path} if $opt->{-skip_lock};
+ $self->{ibx}->with_umask(\&_index_sync, $self, $opt);
+ if ($opt->{reindex}) {
+ my %again = %$opt;
delete @again{qw(rethread reindex)};
index_sync($self, \%again);
}
@@ -745,15 +745,15 @@ sub reindex_from ($$) {
# indexes all unindexed messages (v1 only)
sub _index_sync {
- my ($self, $opts) = @_;
- my $tip = $opts->{ref} || 'HEAD';
+ my ($self, $opt) = @_;
+ my $tip = $opt->{ref} || 'HEAD';
my $git = $self->{ibx}->git;
- $self->{batch_bytes} = $opts->{batch_size} // $BATCH_BYTES;
+ $self->{batch_bytes} = $opt->{batch_size} // $BATCH_BYTES;
$git->batch_prepare;
- my $pr = $opts->{-progress};
- my $sync = { reindex => $opts->{reindex}, -opt => $opts };
+ my $pr = $opt->{-progress};
+ my $sync = { reindex => $opt->{reindex}, -opt => $opt };
my $xdb = $self->begin_txn_lazy;
- $self->{over}->rethread_prepare($opts);
+ $self->{over}->rethread_prepare($opt);
my $mm = _msgmap_init($self);
if ($sync->{reindex}) {
my $last = $mm->last_commit;
^ permalink raw reply related [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-08-10 2:11 5% [PATCH 00/14] more indexing related improvements Eric Wong
2020-08-10 2:12 7% ` [PATCH 10/14] searchidx: use singular `$opt' for consistency with v2 Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).