user/dev discussion of public-inbox itself
 help / color / Atom feed
* [PATCH 0/2] make "public-inbox-index" friendlier to clone mirrors
@ 2019-03-08 22:52 Eric Wong
  2019-03-08 22:52 ` [PATCH 1/2] v2writable: hoist out fill_alternates Eric Wong
  2019-03-08 22:52 ` [PATCH 2/2] v2writable: index_sync adds new epochs to alternates Eric Wong
  0 siblings, 2 replies; 3+ messages in thread
From: Eric Wong @ 2019-03-08 22:52 UTC (permalink / raw)
  To: meta

While I remembered to clone the 7.git epoch of LKML and
run "public-inbox-index"; new messages were inaccessible
via read-only PSGI and NNTP interfaces since I forgot to
add the new epoch to $INBOX_DIR/all.git/objects/info/alternates.

Eric Wong (2):
      v2writable: hoist out fill_alternates
      v2writable: index_sync adds new epochs to alternates

 lib/PublicInbox/V2Writable.pm | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)



^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH 1/2] v2writable: hoist out fill_alternates
  2019-03-08 22:52 [PATCH 0/2] make "public-inbox-index" friendlier to clone mirrors Eric Wong
@ 2019-03-08 22:52 ` Eric Wong
  2019-03-08 22:52 ` [PATCH 2/2] v2writable: index_sync adds new epochs to alternates Eric Wong
  1 sibling, 0 replies; 3+ messages in thread
From: Eric Wong @ 2019-03-08 22:52 UTC (permalink / raw)
  To: meta

We'll be using this sub to fill $GIT_DIR/objects/info/alternates
if somebody uses clone --mirror, too
---
 lib/PublicInbox/V2Writable.pm | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm
index b1d8095..bc31ffa 100644
--- a/lib/PublicInbox/V2Writable.pm
+++ b/lib/PublicInbox/V2Writable.pm
@@ -509,14 +509,12 @@ sub done {
 	$self->{-inbox}->git->cleanup;
 }
 
-sub git_init {
+sub fill_alternates ($$) {
 	my ($self, $epoch) = @_;
-	my $pfx = "$self->{-inbox}->{mainrepo}/git";
-	my $git_dir = "$pfx/$epoch.git";
-	my @cmd = (qw(git init --bare -q), $git_dir);
-	PublicInbox::Import::run_die(\@cmd);
 
+	my $pfx = "$self->{-inbox}->{mainrepo}/git";
 	my $all = "$self->{-inbox}->{mainrepo}/all.git";
+	my @cmd;
 	unless (-d $all) {
 		@cmd = (qw(git init --bare -q), $all);
 		PublicInbox::Import::run_die(\@cmd);
@@ -524,8 +522,7 @@ sub git_init {
 				'repack.writeBitmaps', 'true');
 		PublicInbox::Import::run_die(\@cmd);
 	}
-
-	@cmd = (qw/git config/, "--file=$git_dir/config",
+	@cmd = (qw/git config/, "--file=$pfx/$epoch.git/config",
 			'include.path', '../../all.git/config');
 	PublicInbox::Import::run_die(\@cmd);
 
@@ -540,12 +537,20 @@ sub git_init {
 		my $dir = "../../git/$i.git/objects";
 		push @add, $dir if !$alts{$dir} && -d "$pfx/$i.git";
 	}
-	return $git_dir unless @add;
+	return unless @add;
 	open my $fh, '>>', $alt or die "open >> $alt: $!\n";
 	foreach my $dir (@add) {
 		print $fh "$dir\n" or die "print >> $alt: $!\n";
 	}
 	close $fh or die "close $alt: $!\n";
+}
+
+sub git_init {
+	my ($self, $epoch) = @_;
+	my $git_dir = "$self->{-inbox}->{mainrepo}/git/$epoch.git";
+	my @cmd = (qw(git init --bare -q), $git_dir);
+	PublicInbox::Import::run_die(\@cmd);
+	fill_alternates($self, $epoch);
 	$git_dir
 }
 
-- 
EW


^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH 2/2] v2writable: index_sync adds new epochs to alternates
  2019-03-08 22:52 [PATCH 0/2] make "public-inbox-index" friendlier to clone mirrors Eric Wong
  2019-03-08 22:52 ` [PATCH 1/2] v2writable: hoist out fill_alternates Eric Wong
@ 2019-03-08 22:52 ` Eric Wong
  1 sibling, 0 replies; 3+ messages in thread
From: Eric Wong @ 2019-03-08 22:52 UTC (permalink / raw)
  To: meta

Newly-cloned epochs need to be in alternates file of
all.git for the web and NNTP interfaces to work.  So
allow invocations of "public-inbox-index" to idempotently
ensure the epoch is visible from the all.git repo.
---
 lib/PublicInbox/V2Writable.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm
index bc31ffa..6829a34 100644
--- a/lib/PublicInbox/V2Writable.pm
+++ b/lib/PublicInbox/V2Writable.pm
@@ -941,6 +941,7 @@ sub index_sync {
 		my $git_dir = git_dir_n($self, $i);
 		die "already reindexing!\n" if delete $self->{reindex_pipe};
 		-d $git_dir or next; # missing parts are fine
+		fill_alternates($self, $i);
 		my $git = PublicInbox::Git->new($git_dir);
 		my $unindex = delete $self->{"unindex-range.$i"};
 		$self->unindex($opts, $git, $unindex) if $unindex;
-- 
EW


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-08 22:52 [PATCH 0/2] make "public-inbox-index" friendlier to clone mirrors Eric Wong
2019-03-08 22:52 ` [PATCH 1/2] v2writable: hoist out fill_alternates Eric Wong
2019-03-08 22:52 ` [PATCH 2/2] v2writable: index_sync adds new epochs to alternates Eric Wong

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror http://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.org/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox