[PATCH 0/2] git: a few minor cleanup tweaks to daemons

[PATCH 0/2] git: a few minor cleanup tweaks to daemons

[Eric Wong]

These probably don't affect anybody running servers atm, as I'm
probably the most hardware-deficient of public-inbox admins out
there...

Eric Wong (2):
  git: unconditional expiry
  git: drop the deleted err_c file

 lib/PublicInbox/Git.pm   | 21 ++++++---------------
 lib/PublicInbox/Inbox.pm |  5 ++---
 t/git.t                  |  3 +--
 3 files changed, 9 insertions(+), 20 deletions(-)

-- 
EW

[PATCH 1/2] git: unconditional expiry

[Eric Wong]

A constant stream of traffic to either httpd/nntpd would mean
git-cat-file processes never expire.  Things can go bad after a
full repack, as a full repack will unlink old pack indices and
git-cat-file does not currently detect unlinked files.

We could do something complicated by recursively stat-ing
objects/pack of every git directory and alternate;
but that's probably not worth the trouble compared to
occasionally restarting the cat-file process.

So simplify the code and let httpd/nntpd expire them
periodically, since spawning a "git-cat-file --batch" process
isn't too expensive.  We already spawn for every request which
hits git-http-backend, cgit, and git-apply.

In the future, we may optionally support the Git::Raw module
to avoid IPC; but we must remain careful to not leave lingering
FDs open to unlinked files after repack.
---
 lib/PublicInbox/Git.pm   | 20 +++++---------------
 lib/PublicInbox/Inbox.pm |  5 ++---
 t/git.t                  |  3 +--
 3 files changed, 8 insertions(+), 20 deletions(-)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index a4daaa4..9a38d7c 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -211,19 +211,9 @@ sub check {
 }
 
 sub _destroy {
-	my ($self, $in, $out, $pid, $expire) = @_;
-	my $rfh = $self->{$in} or return;
-	if (defined $expire) {
-		# at least FreeBSD 11.2 and Linux 4.20 update mtime of the
-		# read end of a pipe when the pipe is written to; dunno
-		# about other OSes.
-		my $mtime = (stat($rfh))[9];
-		return if $mtime > $expire;
-	}
+	my ($self, $in, $out, $pid) = @_;
 	my $p = delete $self->{$pid} or return;
-	foreach my $f ($in, $out) {
-		delete $self->{$f};
-	}
+	delete @$self{($in, $out)};
 	waitpid $p, 0;
 }
 
@@ -251,9 +241,9 @@ sub qx {
 
 # returns true if there are pending "git cat-file" processes
 sub cleanup {
-	my ($self, $expire) = @_;
-	_destroy($self, qw(in out pid), $expire);
-	_destroy($self, qw(in_c out_c pid_c), $expire);
+	my ($self) = @_;
+	_destroy($self, qw(in out pid));
+	_destroy($self, qw(in_c out_c pid_c));
 	!!($self->{pid} || $self->{pid_c});
 }
 
diff --git a/lib/PublicInbox/Inbox.pm b/lib/PublicInbox/Inbox.pm
index 2771a24..b3178b9 100644
--- a/lib/PublicInbox/Inbox.pm
+++ b/lib/PublicInbox/Inbox.pm
@@ -32,13 +32,12 @@ sub cleanup_task () {
 				# refcnt is zero when tmp is out-of-scope
 			}
 		}
-		my $expire = time - 60;
 		if (my $git = $ibx->{git}) {
-			$again = $git->cleanup($expire);
+			$again = $git->cleanup;
 		}
 		if (my $gits = $ibx->{-repo_objs}) {
 			foreach my $git (@$gits) {
-				$again = 1 if $git->cleanup($expire);
+				$again = 1 if $git->cleanup;
 			}
 		}
 		if ($have_devel_peek) {
diff --git a/t/git.t b/t/git.t
index 7edf82b..913f6e5 100644
--- a/t/git.t
+++ b/t/git.t
@@ -143,8 +143,7 @@ if ('alternates reloaded') {
 	my $config = eval { local $/; <$fh> };
 	is($$found, $config, 'alternates reloaded');
 
-	ok($gcf->cleanup(time - 30), 'cleanup did not expire');
-	ok(!$gcf->cleanup(time + 30), 'cleanup can expire');
+	ok(!$gcf->cleanup, 'cleanup can expire');
 	ok(!$gcf->cleanup, 'cleanup idempotent');
 
 	my $t = $gcf->modified;
-- 
EW

[PATCH 2/2] git: drop the deleted err_c file

[Eric Wong]

No reason to leave that (usually) empty file open after killing off
"cat-file --batch-check".  This wasn't an unbound leak, though,
as respawning the --batch-check process would've clobbered the
old err_c file.
---
 lib/PublicInbox/Git.pm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 9a38d7c..9014e02 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -211,9 +211,10 @@ sub check {
 }
 
 sub _destroy {
-	my ($self, $in, $out, $pid) = @_;
+	my ($self, $in, $out, $pid, $err) = @_;
 	my $p = delete $self->{$pid} or return;
 	delete @$self{($in, $out)};
+	delete $self->{$err} if $err; # `err_c'
 	waitpid $p, 0;
 }
 
@@ -243,7 +244,7 @@ sub qx {
 sub cleanup {
 	my ($self) = @_;
 	_destroy($self, qw(in out pid));
-	_destroy($self, qw(in_c out_c pid_c));
+	_destroy($self, qw(in_c out_c pid_c err_c));
 	!!($self->{pid} || $self->{pid_c});
 }
 
-- 
EW