diff options
author | Eric Wong <e@yhbt.net> | 2020-08-07 10:52:16 +0000 |
---|---|---|
committer | Eric Wong <e@yhbt.net> | 2020-08-07 23:46:13 +0000 |
commit | 9030233f83bc2e30f9529ea484da123ea9d98119 (patch) | |
tree | dded614092ac9042509add800d916b999df51e84 /script | |
parent | 8a66a8d396c6cff82e6041d107d7fb842db6d602 (diff) | |
download | public-inbox-9030233f83bc2e30f9529ea484da123ea9d98119.tar.gz |
If XAPIAN_FLUSH_THRESHOLD is unset, Xapian will default to 10000. That limits the effectiveness of users specifying extremely large values of --batch-size. While we're at it, localize the changes to globals since -index may be eval-ed in tests (and perhaps production code in the future).
Diffstat (limited to 'script')
-rwxr-xr-x | script/public-inbox-index | 9 |
1 files changed, 7 insertions, 2 deletions
diff --git a/script/public-inbox-index b/script/public-inbox-index index 56df5bfe..e2bca16e 100755 --- a/script/public-inbox-index +++ b/script/public-inbox-index @@ -42,11 +42,16 @@ if (defined $max_size) { die "`publicInbox.indexMaxSize=$max_size' not parsed\n"; } -if (my $bs = $opt->{batchsize} // $cfg->{lc('publicInbox.indexBatchSize')}) { +my $bs = $opt->{batchsize} // $cfg->{lc('publicInbox.indexBatchSize')}; +if (defined $bs) { PublicInbox::Admin::parse_unsigned(\$bs) or die "`publicInbox.indexBatchSize=$bs' not parsed\n"; - $PublicInbox::SearchIdx::BATCH_BYTES = $bs; } +local $PublicInbox::SearchIdx::BATCH_BYTES = $bs if defined($bs); + +# out-of-the-box builds of Xapian 1.4.x are still limited to 32-bit +# https://getting-started-with-xapian.readthedocs.io/en/latest/concepts/indexing/limitations.html +local $ENV{XAPIAN_FLUSH_THRESHOLD} ||= '4294967295' if defined($bs); my $s = $opt->{sequentialshard} // $cfg->{lc('publicInbox.indexSequentialShard')}; |