From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id A8A101F86C for ; Fri, 13 Nov 2020 11:11:44 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 2/4] *index: avoid per-epoch --batch-check processes Date: Fri, 13 Nov 2020 11:11:42 +0000 Message-Id: <20201113111144.23038-3-e@80x24.org> In-Reply-To: <20201113111144.23038-1-e@80x24.org> References: <20201113111144.23038-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Since all.git (v2) and ALL.git (extindex) encompass every single epoch or indexed inbox; and we is_ancestor() only uses hexadecimal OIDs, there is no good reason to to use $unit->{git} for an epoch-local $git->check avoids redundant long-lived processes. This prevents dozens/hundreds of --batch-check processes from being left running after indexing and can improve locality if size checks are being done (since that uses --batch-check, too). Theoretically an several epochs may have conflicting OIDs, but we're screwed in those cases, anyways, so we might as well detect it earlier (though I'm not sure what the behavior would be :x). --- lib/PublicInbox/ExtSearchIdx.pm | 2 +- lib/PublicInbox/V2Writable.pm | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/PublicInbox/ExtSearchIdx.pm b/lib/PublicInbox/ExtSearchIdx.pm index 14ffdadb..2d230dc1 100644 --- a/lib/PublicInbox/ExtSearchIdx.pm +++ b/lib/PublicInbox/ExtSearchIdx.pm @@ -361,7 +361,7 @@ sub update_last_commit { # overrides V2Writable die "Unsupported inbox version: $v"; } my $last = $self->{oidx}->eidx_meta($meta_key); - if (defined $last && is_ancestor($unit->{git}, $last, $latest_cmt)) { + if (defined $last && is_ancestor($self->git, $last, $latest_cmt)) { my @cmd = (qw(rev-list --count), "$last..$latest_cmt"); chomp(my $n = $unit->{git}->qx(@cmd)); return if $n ne '' && $n == 0; diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm index 87b76501..cf44c95b 100644 --- a/lib/PublicInbox/V2Writable.pm +++ b/lib/PublicInbox/V2Writable.pm @@ -966,7 +966,7 @@ sub update_last_commit { my $latest_cmt = $stk ? $stk->{latest_cmt} : ${$sync->{latest_cmt}}; defined($latest_cmt) or return; my $last = last_epoch_commit($self, $unit->{epoch}); - if (defined $last && is_ancestor($unit->{git}, $last, $latest_cmt)) { + if (defined $last && is_ancestor($self->git, $last, $latest_cmt)) { my @cmd = (qw(rev-list --count), "$last..$latest_cmt"); chomp(my $n = $unit->{git}->qx(@cmd)); return if $n ne '' && $n == 0; @@ -1003,7 +1003,7 @@ sub log_range ($$$) { my $range = "$cur..$tip"; $pr->("$i.git checking contiguity... ") if $pr; my $git = $unit->{git}; - if (is_ancestor($git, $cur, $tip)) { # common case + if (is_ancestor($sync->{self}->git, $cur, $tip)) { # common case $pr->("OK\n") if $pr; my $n = $git->qx(qw(rev-list --count), $range); chomp($n);