From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 2E7CE1F66E for ; Sat, 22 Aug 2020 19:51:36 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH] index: --sequential-shard checkpoints after each shard Date: Sat, 22 Aug 2020 19:51:36 +0000 Message-Id: <20200822195136.936-1-e@yhbt.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: There's no reason we'd want Xapian to defer flushing once we've indexed everything belonging to a particular shard. --- lib/PublicInbox/V2Writable.pm | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm index b0148dba..3d6f19cb 100644 --- a/lib/PublicInbox/V2Writable.pm +++ b/lib/PublicInbox/V2Writable.pm @@ -1264,8 +1264,11 @@ sub xapian_only { $sync->{art_end} = $art_end; if ($seq || !$self->{parallel}) { my $shard_end = $self->{shards} - 1; - for (0..$shard_end) { - index_xap_step($self, $sync, $art_beg + $_) + for my $i (0..$shard_end) { + index_xap_step($self, $sync, $art_beg + $i); + if ($i != $shard_end) { + reindex_checkpoint($self, $sync); + } } } else { # parallel (maybe) index_xap_step($self, $sync, $art_beg, 1);