user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: "Eric Wong (Contractor, The Linux Foundation)" <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 06/12] git: reload alternates file on missing blob
Date: Thu, 22 Feb 2018 21:42:16 +0000	[thread overview]
Message-ID: <20180222214222.1086-7-e@80x24.org> (raw)
In-Reply-To: <20180222214222.1086-1-e@80x24.org>

Since we'll be adding new repositories to the `alternates' file
in git, we must restart the `git cat-file --batch' process as
git currently does not detect changes to the alternates file
in long-running cat-file processes.

Don't bother with the `--batch-check' process since we won't be
using it with v2.
---
 lib/PublicInbox/Git.pm | 29 +++++++++++++++++++++++++----
 t/git.t                | 21 +++++++++++++++++++++
 2 files changed, 46 insertions(+), 4 deletions(-)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 6437643..95df52e 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -15,7 +15,19 @@ use PublicInbox::Spawn qw(spawn popen_rd);
 
 sub new {
 	my ($class, $git_dir) = @_;
-	bless { git_dir => $git_dir }, $class
+	my @st;
+	$st[7] = $st[10] = 0;
+	bless { git_dir => $git_dir, st => \@st }, $class
+}
+
+sub alternates_changed {
+	my ($self) = @_;
+	my $alt = "$self->{git_dir}/objects/info/alternates";
+	my @st = stat($alt) or return 0;
+	my $old_st = $self->{st};
+	# 10 - ctime, 7 - size
+	return 0 if ($st[10] == $old_st->[10] && $st[7] == $old_st->[7]);
+	$self->{st} = \@st;
 }
 
 sub _bidi_pipe {
@@ -38,14 +50,23 @@ sub _bidi_pipe {
 
 sub cat_file {
 	my ($self, $obj, $ref) = @_;
+	my ($retried, $in, $head);
 
+again:
 	batch_prepare($self);
 	$self->{out}->print($obj, "\n") or fail($self, "write error: $!");
 
-	my $in = $self->{in};
+	$in = $self->{in};
 	local $/ = "\n";
-	my $head = $in->getline;
-	$head =~ / missing$/ and return undef;
+	$head = $in->getline;
+	if ($head =~ / missing$/) {
+		if (!$retried && alternates_changed($self)) {
+			$retried = 1;
+			cleanup($self);
+			goto again;
+		}
+		return;
+	}
 	$head =~ /^[0-9a-f]{40} \S+ (\d+)$/ or
 		fail($self, "Unexpected result from git cat-file: $head");
 
diff --git a/t/git.t b/t/git.t
index ab588a1..7f96293 100644
--- a/t/git.t
+++ b/t/git.t
@@ -6,6 +6,7 @@ use Test::More;
 use File::Temp qw/tempdir/;
 my $dir = tempdir('pi-git-XXXXXX', TMPDIR => 1, CLEANUP => 1);
 use Cwd qw/getcwd/;
+use PublicInbox::Spawn qw(popen_rd);
 
 use_ok 'PublicInbox::Git';
 {
@@ -142,4 +143,24 @@ if (1) {
 	ok($gcf->packed_bytes > 0, 'packed size is positive');
 }
 
+if ('alternates reloaded') {
+	my $alt = tempdir('pi-git-XXXXXX', TMPDIR => 1, CLEANUP => 1);
+	my @cmd = ('git', "--git-dir=$alt", qw(hash-object -w --stdin));
+	is(system(qw(git init -q --bare), $alt), 0, 'create alt directory');
+	open my $fh, '<', "$alt/config" or die "open failed: $!\n";
+	my $rd = popen_rd(\@cmd, {}, { 0 => fileno($fh) } );
+	close $fh or die "close failed: $!";
+	chomp(my $remote = <$rd>);
+	my $gcf = PublicInbox::Git->new($dir);
+	is($gcf->cat_file($remote), undef, "remote file not found");
+	open $fh, '>>', "$dir/objects/info/alternates" or
+			die "open failed: $!\n";
+	print $fh "$alt/objects" or die "print failed: $!\n";
+	close $fh or die "close failed: $!";
+	my $found = $gcf->cat_file($remote);
+	open $fh, '<', "$alt/config" or die "open failed: $!\n";
+	my $config = eval { local $/; <$fh> };
+	is($$found, $config, 'alternates reloaded');
+}
+
 done_testing();
-- 
EW


  parent reply	other threads:[~2018-02-22 21:42 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-22 21:42 [WIP PATCH 0/12] v2: git repo rotation + parallel Xapian indexing Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 01/12] import: allow the epoch (0s) as a valid time Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 02/12] extmsg: fix broken Xapian MID lookup Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 03/12] search: stop assuming Message-ID is unique Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 04/12] www: stop assuming mainrepo == git_dir Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 05/12] v2writable: initial cut for repo-rotation Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` Eric Wong (Contractor, The Linux Foundation) [this message]
2018-02-22 21:42 ` [PATCH 07/12] v2: support Xapian + SQLite indexing Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 08/12] import_vger_from_inbox: allow "-V" option Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 09/12] import_vger_from_mbox: use PublicInbox::MIME and avoid clobbering Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 10/12] v2: parallelize Xapian indexing Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 11/12] v2writable: round-robin to partitions based on article number Eric Wong (Contractor, The Linux Foundation)
2018-02-22 21:42 ` [PATCH 12/12] searchidxpart: increase pipe size for partitions Eric Wong (Contractor, The Linux Foundation)
2018-02-23  1:22 ` [WIP PATCH 0/12] v2: git repo rotation + parallel Xapian indexing Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180222214222.1086-7-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).