From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 1/2] git: unconditional expiry
Date: Sat, 1 Jun 2019 03:37:05 +0000 [thread overview]
Message-ID: <20190601033706.18113-2-e@80x24.org> (raw)
In-Reply-To: <20190601033706.18113-1-e@80x24.org>
A constant stream of traffic to either httpd/nntpd would mean
git-cat-file processes never expire. Things can go bad after a
full repack, as a full repack will unlink old pack indices and
git-cat-file does not currently detect unlinked files.
We could do something complicated by recursively stat-ing
objects/pack of every git directory and alternate;
but that's probably not worth the trouble compared to
occasionally restarting the cat-file process.
So simplify the code and let httpd/nntpd expire them
periodically, since spawning a "git-cat-file --batch" process
isn't too expensive. We already spawn for every request which
hits git-http-backend, cgit, and git-apply.
In the future, we may optionally support the Git::Raw module
to avoid IPC; but we must remain careful to not leave lingering
FDs open to unlinked files after repack.
---
lib/PublicInbox/Git.pm | 20 +++++---------------
lib/PublicInbox/Inbox.pm | 5 ++---
t/git.t | 3 +--
3 files changed, 8 insertions(+), 20 deletions(-)
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index a4daaa4..9a38d7c 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -211,19 +211,9 @@ sub check {
}
sub _destroy {
- my ($self, $in, $out, $pid, $expire) = @_;
- my $rfh = $self->{$in} or return;
- if (defined $expire) {
- # at least FreeBSD 11.2 and Linux 4.20 update mtime of the
- # read end of a pipe when the pipe is written to; dunno
- # about other OSes.
- my $mtime = (stat($rfh))[9];
- return if $mtime > $expire;
- }
+ my ($self, $in, $out, $pid) = @_;
my $p = delete $self->{$pid} or return;
- foreach my $f ($in, $out) {
- delete $self->{$f};
- }
+ delete @$self{($in, $out)};
waitpid $p, 0;
}
@@ -251,9 +241,9 @@ sub qx {
# returns true if there are pending "git cat-file" processes
sub cleanup {
- my ($self, $expire) = @_;
- _destroy($self, qw(in out pid), $expire);
- _destroy($self, qw(in_c out_c pid_c), $expire);
+ my ($self) = @_;
+ _destroy($self, qw(in out pid));
+ _destroy($self, qw(in_c out_c pid_c));
!!($self->{pid} || $self->{pid_c});
}
diff --git a/lib/PublicInbox/Inbox.pm b/lib/PublicInbox/Inbox.pm
index 2771a24..b3178b9 100644
--- a/lib/PublicInbox/Inbox.pm
+++ b/lib/PublicInbox/Inbox.pm
@@ -32,13 +32,12 @@ sub cleanup_task () {
# refcnt is zero when tmp is out-of-scope
}
}
- my $expire = time - 60;
if (my $git = $ibx->{git}) {
- $again = $git->cleanup($expire);
+ $again = $git->cleanup;
}
if (my $gits = $ibx->{-repo_objs}) {
foreach my $git (@$gits) {
- $again = 1 if $git->cleanup($expire);
+ $again = 1 if $git->cleanup;
}
}
if ($have_devel_peek) {
diff --git a/t/git.t b/t/git.t
index 7edf82b..913f6e5 100644
--- a/t/git.t
+++ b/t/git.t
@@ -143,8 +143,7 @@ if ('alternates reloaded') {
my $config = eval { local $/; <$fh> };
is($$found, $config, 'alternates reloaded');
- ok($gcf->cleanup(time - 30), 'cleanup did not expire');
- ok(!$gcf->cleanup(time + 30), 'cleanup can expire');
+ ok(!$gcf->cleanup, 'cleanup can expire');
ok(!$gcf->cleanup, 'cleanup idempotent');
my $t = $gcf->modified;
--
EW
next prev parent reply other threads:[~2019-06-01 3:37 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-01 3:37 [PATCH 0/2] git: a few minor cleanup tweaks to daemons Eric Wong
2019-06-01 3:37 ` Eric Wong [this message]
2019-06-01 3:37 ` [PATCH 2/2] git: drop the deleted err_c file Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190601033706.18113-2-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).