* [PATCH 2/5] index: --compact respects --sequential-shard
2020-08-07 10:52 6% [PATCH 0/5] more indexing improvements Eric Wong
@ 2020-08-07 10:52 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-08-07 10:52 UTC (permalink / raw)
To: meta
Since the --compact switch works on Xapian shards,
it makes sense that --sequential-shard affects our
usage of xapian-compact(1).
---
script/public-inbox-index | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/script/public-inbox-index b/script/public-inbox-index
index dc9bdde1..56df5bfe 100755
--- a/script/public-inbox-index
+++ b/script/public-inbox-index
@@ -83,5 +83,8 @@ EOL
$ibx_opt = { %$opt, sequentialshard => $v };
}
PublicInbox::Admin::index_inbox($ibx, undef, $ibx_opt);
- PublicInbox::Xapcmd::run($ibx, 'compact', $compact_opt) if $compact_opt;
+ if ($compact_opt) {
+ local $compact_opt->{jobs} = 0 if $ibx_opt->{sequentialshard};
+ PublicInbox::Xapcmd::run($ibx, 'compact', $compact_opt);
+ }
}
^ permalink raw reply related [relevance 7%]
* [PATCH 0/5] more indexing improvements
@ 2020-08-07 10:52 6% Eric Wong
2020-08-07 10:52 7% ` [PATCH 2/5] index: --compact respects --sequential-shard Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-08-07 10:52 UTC (permalink / raw)
To: meta
VERY big batch sizes seem helpful on HDDs.. And I also blew up
a run because --compact ran in parallel with 32 shards :x
And --help should exist for all commands users may run from
the CLI.
Eric Wong (5):
v2writable: fix batch size accounting
index: --compact respects --sequential-shard
index: max out XAPIAN_FLUSH_THRESHOLD if using --batch-size
searchidx: use Perl truthiness to detect XAPIAN_FLUSH_THRESHOLD
index: add built-in --help / -?
Documentation/public-inbox-index.pod | 4 +-
lib/PublicInbox/SearchIdx.pm | 3 +-
lib/PublicInbox/V2Writable.pm | 10 ++++-
script/public-inbox-index | 58 ++++++++++++++++++++++------
4 files changed, 58 insertions(+), 17 deletions(-)
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-08-07 10:52 6% [PATCH 0/5] more indexing improvements Eric Wong
2020-08-07 10:52 7% ` [PATCH 2/5] index: --compact respects --sequential-shard Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).