From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 08/12] miscidx: cleanup git processes after manifest indexing
Date: Mon, 23 Nov 2020 07:05:58 +0000 [thread overview]
Message-ID: <20201123070602.9698-9-e@80x24.org> (raw)
In-Reply-To: <20201123070602.9698-1-e@80x24.org>
We shouldn't leave "cat-file --batch" processes around when
we're done with an epoch or inbox, since there could be
many thousands.
---
lib/PublicInbox/ExtSearchIdx.pm | 1 +
lib/PublicInbox/MiscIdx.pm | 1 +
2 files changed, 2 insertions(+)
diff --git a/lib/PublicInbox/ExtSearchIdx.pm b/lib/PublicInbox/ExtSearchIdx.pm
index 708f8a3e..890ac282 100644
--- a/lib/PublicInbox/ExtSearchIdx.pm
+++ b/lib/PublicInbox/ExtSearchIdx.pm
@@ -311,6 +311,7 @@ sub _sync_inbox ($$$) {
}
index_todo($self, $sync, $_) for @{delete($sync->{todo}) // []};
$self->{midx}->index_ibx($ibx);
+ $ibx->git->cleanup; # done with this inbox, now
}
sub eidx_sync { # main entry point
diff --git a/lib/PublicInbox/MiscIdx.pm b/lib/PublicInbox/MiscIdx.pm
index 9dcc96b7..acb49ce7 100644
--- a/lib/PublicInbox/MiscIdx.pm
+++ b/lib/PublicInbox/MiscIdx.pm
@@ -107,6 +107,7 @@ EOF
if (my $ent = $git->manifest_entry($epoch, $desc)) {
$data->{"$pfx$epoch.git"} = $ent;
}
+ $git->cleanup; # ->modified starts cat-file --batch
}
} elsif (my $ent = $ibx->git->manifest_entry) { # v1
$data->{"/$ibx->{name}"} = $ent;
next prev parent reply other threads:[~2020-11-23 7:06 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-23 7:05 [PATCH 00/12] extindex: speed up manifest.js.gz generation Eric Wong
2020-11-23 7:05 ` [PATCH 01/12] miscsearch: a new Xapian sub-DB for extindex Eric Wong
2020-11-23 7:05 ` [PATCH 02/12] move JSON module portability into PublicInbox::Config Eric Wong
2020-11-23 7:05 ` [PATCH 03/12] git: add manifest_entry method Eric Wong
2020-11-23 7:05 ` [PATCH 04/12] manifest: use ibx->git_epoch method for v2 Eric Wong
2020-11-23 7:05 ` [PATCH 05/12] inbox: git_epoch: remove ->version check Eric Wong
2020-11-23 7:05 ` [PATCH 06/12] miscidx: put grokmirror manifest entries in Xapian docdata Eric Wong
2020-11-23 7:05 ` [PATCH 07/12] extsearch: fix remaining "eindex" references Eric Wong
2020-11-23 7:05 ` Eric Wong [this message]
2020-11-23 7:05 ` [PATCH 09/12] miscidx: store absolute git_dir of each epoch in docdata Eric Wong
2020-11-23 7:06 ` [PATCH 10/12] extsearchidx: do not short-circuit MiscIdx on no-op v2 prepare Eric Wong
2020-11-23 7:06 ` [PATCH 11/12] manifest: support faster generation via [extindex "all"] Eric Wong
2020-11-23 7:06 ` [PATCH 12/12] *search: simplify retry_reopen users Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201123070602.9698-9-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).