* [PATCH 6/7] v2writable: don't verify tip if reindexing
2020-12-25 10:21 5% [PATCH 0/7] index + extindex interaction improvements Eric Wong
@ 2020-12-25 10:21 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-12-25 10:21 UTC (permalink / raw)
To: meta
We only rely on git-rev-parse to resolve symbolic names ("HEAD")
to a SHA-* git commit ID. We'll assume any git commit IDs we
get from SQLite DBs are valid and let "git-log" fail if it
isn't.
---
lib/PublicInbox/V2Writable.pm | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm
index ca52874b..f20b5c7f 100644
--- a/lib/PublicInbox/V2Writable.pm
+++ b/lib/PublicInbox/V2Writable.pm
@@ -1104,12 +1104,14 @@ sub sync_prepare ($$) {
-d $git_dir or next; # missing epochs are fine
my $git = PublicInbox::Git->new($git_dir);
my $unit = { git => $git, epoch => $i };
+ my $tip;
if ($reindex_heads) {
- $head = $reindex_heads->[$i] or next;
+ $tip = $head = $reindex_heads->[$i] or next;
+ } else {
+ $tip = $git->qx(qw(rev-parse -q --verify), $head);
+ next if $?; # new repo
+ chomp $tip;
}
- chomp(my $tip = $git->qx(qw(rev-parse -q --verify), $head));
- next if $?; # new repo
-
my $range = log_range($sync, $unit, $tip) or next;
# can't use 'rev-list --count' if we use --diff-filter
$pr->("$pfx $i.git counting $range ... ") if $pr;
^ permalink raw reply related [relevance 7%]
* [PATCH 0/7] index + extindex interaction improvements
@ 2020-12-25 10:21 5% Eric Wong
2020-12-25 10:21 7% ` [PATCH 6/7] v2writable: don't verify tip if reindexing Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-12-25 10:21 UTC (permalink / raw)
To: meta
Some things which make -index less painful when auto-updating
external indices.
"public-inbox-extindex --all" itself is still painfully slow
with 50K inboxes, but I think that can only be used once for
initialization and -index can be relied on for all incremental
updates.
Eric Wong (7):
index: disable --fast-noop on --reindex
extsearchidx: delay SQLite availability checks
extsearchidx: close DB handles after use if FD constrained
index: do not attach inbox to extindex unless updated
index: fix --no-fsync flag propagation to extindex
v2writable: don't verify tip if reindexing
index: filter out indexlevel=basic from extindex
lib/PublicInbox/Admin.pm | 1 +
lib/PublicInbox/ExtSearchIdx.pm | 96 +++++++++++++++++++++------------
lib/PublicInbox/SearchIdx.pm | 2 +
lib/PublicInbox/V2Writable.pm | 36 +++++++++----
script/public-inbox-index | 27 ++++++----
5 files changed, 109 insertions(+), 53 deletions(-)
^ permalink raw reply [relevance 5%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-12-25 10:21 5% [PATCH 0/7] index + extindex interaction improvements Eric Wong
2020-12-25 10:21 7% ` [PATCH 6/7] v2writable: don't verify tip if reindexing Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).