user/dev discussion of public-inbox itself
 help / color / Atom feed
* [PATCH 0/4] flesh out multi-URL support
@ 2020-02-01  9:12 Eric Wong
  2020-02-01  9:12 ` [PATCH 1/4] wwwtext: show multiple infourl values properly Eric Wong
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Eric Wong @ 2020-02-01  9:12 UTC (permalink / raw)
  To: meta

Better support for multiple URLs will hopefully discourage SPOFs
and give users more options when hosts go down.  It also gives
users and admins more agility when it comes to switching domain
names or having .onion mirrors.

Eric Wong (4):
  wwwtext: show multiple infourl values properly
  wwwtext: give "url" examples in sample config
  solver: join multiple URLs with "||"
  config: assume multiple cgit URLs, too

 lib/PublicInbox/Config.pm    |  8 ++++++--
 lib/PublicInbox/SolverGit.pm |  5 +++--
 lib/PublicInbox/WwwText.pm   | 24 +++++++++++++++---------
 3 files changed, 24 insertions(+), 13 deletions(-)

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/4] wwwtext: show multiple infourl values properly
  2020-02-01  9:12 [PATCH 0/4] flesh out multi-URL support Eric Wong
@ 2020-02-01  9:12 ` Eric Wong
  2020-02-01  9:12 ` [PATCH 2/4] wwwtext: give "url" examples in sample config Eric Wong
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-02-01  9:12 UTC (permalink / raw)
  To: meta

This is now an array, so ensure it's shown properly in the
sample config, instead of "ARRAY(0xI8BADBEEF)" or similar.

Fixes: 1988d730c0088e8b "config: support multi-value inbox.*.*url"
---
 lib/PublicInbox/WwwText.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/WwwText.pm b/lib/PublicInbox/WwwText.pm
index 2e4aeec0..ace7f9d7 100644
--- a/lib/PublicInbox/WwwText.pm
+++ b/lib/PublicInbox/WwwText.pm
@@ -148,12 +148,12 @@ sub inbox_config ($$$) {
 	; instead of "inboxdir", both remain supported after 1.2
 	mainrepo = /path/to/top-level-inbox
 EOS
-	for my $k (qw(address listid)) {
+	for my $k (qw(address listid infourl)) {
 		defined(my $v = $ibx->{$k}) or next;
 		$$txt .= "\t$k = $_\n" for @$v;
 	}
 
-	for my $k (qw(filter infourl newsgroup obfuscate replyto watchheader)) {
+	for my $k (qw(filter newsgroup obfuscate replyto watchheader)) {
 		defined(my $v = $ibx->{$k}) or next;
 		$$txt .= "\t$k = $v\n";
 	}

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 2/4] wwwtext: give "url" examples in sample config
  2020-02-01  9:12 [PATCH 0/4] flesh out multi-URL support Eric Wong
  2020-02-01  9:12 ` [PATCH 1/4] wwwtext: show multiple infourl values properly Eric Wong
@ 2020-02-01  9:12 ` Eric Wong
  2020-02-01  9:12 ` [PATCH 3/4] solver: join multiple URLs with "||" Eric Wong
  2020-02-01  9:12 ` [PATCH 4/4] config: assume multiple cgit URLs, too Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-02-01  9:12 UTC (permalink / raw)
  To: meta

inbox.$NAME.url is a common parameter and set by
public-inbox-init(1), so ensure we have lines for it and
emphasize it can be multi-value for .onion hidden services or
otherwise mirrored and available under multiple URLs.
---
 lib/PublicInbox/WwwText.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/PublicInbox/WwwText.pm b/lib/PublicInbox/WwwText.pm
index ace7f9d7..fbbe8e3b 100644
--- a/lib/PublicInbox/WwwText.pm
+++ b/lib/PublicInbox/WwwText.pm
@@ -147,6 +147,8 @@ sub inbox_config ($$$) {
 	; note: public-inbox before v1.2.0 used "mainrepo"
 	; instead of "inboxdir", both remain supported after 1.2
 	mainrepo = /path/to/top-level-inbox
+	url = https://example.com/$name/
+	url = http://example.onion/$name/
 EOS
 	for my $k (qw(address listid infourl)) {
 		defined(my $v = $ibx->{$k}) or next;

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 3/4] solver: join multiple URLs with "||"
  2020-02-01  9:12 [PATCH 0/4] flesh out multi-URL support Eric Wong
  2020-02-01  9:12 ` [PATCH 1/4] wwwtext: show multiple infourl values properly Eric Wong
  2020-02-01  9:12 ` [PATCH 2/4] wwwtext: give "url" examples in sample config Eric Wong
@ 2020-02-01  9:12 ` Eric Wong
  2020-02-01  9:12 ` [PATCH 4/4] config: assume multiple cgit URLs, too Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-02-01  9:12 UTC (permalink / raw)
  To: meta

It seems to make sense to the target audience that any of
the URLs displayed could work.
---
 lib/PublicInbox/SolverGit.pm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/SolverGit.pm b/lib/PublicInbox/SolverGit.pm
index 06b4cdc4..bf806f65 100644
--- a/lib/PublicInbox/SolverGit.pm
+++ b/lib/PublicInbox/SolverGit.pm
@@ -530,7 +530,8 @@ sub resolve_patch ($$) {
 	if (my $existing = solve_existing($self, $want)) {
 		my ($found_git, undef, $type, undef) = @$existing;
 		dbg($self, "found $cur_want in " .
-			join("\n", $found_git->pub_urls($self->{psgi_env})));
+			join(" ||\n\t",
+				$found_git->pub_urls($self->{psgi_env})));
 
 		if ($cur_want eq $self->{oid_want} || $type ne 'blob') {
 			eval { done($self, $existing) };
@@ -548,7 +549,7 @@ sub resolve_patch ($$) {
 
 		unshift @{$self->{patches}}, @$diffs;
 		dbg($self, "found $cur_want in ".
-			join("\n\t", map { di_url($self, $_) } @$diffs));
+			join(" ||\n\t", map { di_url($self, $_) } @$diffs));
 
 		# good, we can find a path to the oid we $want, now
 		# lets see if we need to apply more patches:

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 4/4] config: assume multiple cgit URLs, too
  2020-02-01  9:12 [PATCH 0/4] flesh out multi-URL support Eric Wong
                   ` (2 preceding siblings ...)
  2020-02-01  9:12 ` [PATCH 3/4] solver: join multiple URLs with "||" Eric Wong
@ 2020-02-01  9:12 ` Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-02-01  9:12 UTC (permalink / raw)
  To: meta

Since we support inboxes with multiple URLs and multiple
infourls to reduce reliance on SPOFs, we'll do the same with
cgit URLs.
---
 lib/PublicInbox/Config.pm  |  8 ++++++--
 lib/PublicInbox/WwwText.pm | 18 +++++++++++-------
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm
index 1ba1225e..e0ca7c5a 100644
--- a/lib/PublicInbox/Config.pm
+++ b/lib/PublicInbox/Config.pm
@@ -190,6 +190,9 @@ sub cgit_repo_merge ($$$) {
 		return unless -e "$path/$se";
 	}
 	return if -e "$path/noweb";
+	# this comes from the cgit config, and AFAIK cgit only allows
+	# repos to have one URL, but that's just the PATH_INFO component,
+	# not the Host: portion
 	# $repo = { url => 'foo.git', dir => '/path/to/foo.git' }
 	my $rel = $repo->{url};
 	unless (defined $rel) {
@@ -207,7 +210,7 @@ sub cgit_repo_merge ($$$) {
 			$rel =~ s!/?\.git\z!!;
 	}
 	$self->{"coderepo.$rel.dir"} //= $path;
-	$self->{"coderepo.$rel.cgiturl"} //= $rel;
+	$self->{"coderepo.$rel.cgiturl"} //= _array($rel);
 }
 
 sub is_git_dir ($) {
@@ -332,8 +335,9 @@ sub _fill_code_repo {
 				_array($self->{lc("$pfx.${t}UrlFormat")});
 	}
 
-	if (my $cgits = $self->{lc("$pfx.cgitUrl")}) {
+	if (defined(my $cgits = $self->{"$pfx.cgiturl"})) {
 		$git->{cgit_url} = $cgits = _array($cgits);
+		$self->{"$pfx.cgiturl"} = $cgits;
 
 		# cgit supports "/blob/?id=%s", but it's only a plain-text
 		# display and requires an unabbreviated id=
diff --git a/lib/PublicInbox/WwwText.pm b/lib/PublicInbox/WwwText.pm
index fbbe8e3b..c084b5bf 100644
--- a/lib/PublicInbox/WwwText.pm
+++ b/lib/PublicInbox/WwwText.pm
@@ -168,17 +168,21 @@ EOS
 
 		my $pi_config = $ctx->{www}->{pi_config};
 		for my $cr_name (@$cr) {
-			my $url = $pi_config->{"coderepo.$cr_name.cgiturl"};
+			my $urls = $pi_config->{"coderepo.$cr_name.cgiturl"};
 			my $path = "/path/to/$cr_name";
 			$cr_name = dq_escape($cr_name);
 
 			$$txt .= qq([coderepo "$cr_name"]\n);
-			if (defined($url)) {
-				my $cpath = $path;
-				if ($path !~ m![a-z0-9_/\.\-]!i) {
-					$cpath = dq_escape($cpath);
-				}
-				$$txt .= qq(\t; git clone $url "$cpath"\n);
+			if ($urls && scalar(@$urls)) {
+				$$txt .= "\t; ";
+				$$txt .= join(" ||\n\t;\t", map {;
+					my $cpath = $path;
+					if ($path !~ m![a-z0-9_/\.\-]!i) {
+						$cpath = dq_escape($cpath);
+					}
+					qq(git clone $_ "$cpath");
+				} @$urls);
+				$$txt .= "\n";
 			}
 			$$txt .= "\tdir = $path\n";
 			$$txt .= "\tcgiturl = https://example.com/";

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-01  9:12 [PATCH 0/4] flesh out multi-URL support Eric Wong
2020-02-01  9:12 ` [PATCH 1/4] wwwtext: show multiple infourl values properly Eric Wong
2020-02-01  9:12 ` [PATCH 2/4] wwwtext: give "url" examples in sample config Eric Wong
2020-02-01  9:12 ` [PATCH 3/4] solver: join multiple URLs with "||" Eric Wong
2020-02-01  9:12 ` [PATCH 4/4] config: assume multiple cgit URLs, too Eric Wong

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git