From 3596019278ef489f27e0659c752977f60f847903 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Fri, 24 Sep 2021 10:56:43 +0000 Subject: clone|--mirror: fix and test against pre-manifest WWW There may still be pre-manifest.js.gz versions of PublicInbox::WWW. running and serving v2 inboxes. Since $INBOX_URL/manifest.js.gz was not understood, it was assumed to be a Message-ID and 301-ed to "$INBOX_URL/manifest.js.gz/" with a trailing slash, so our 404 checks were invalid. Update our fallbacks to deal with 301 by catching JSON decoding errors to trigger HTML scraping. For HTML parsing, be sure to not be fooled by potential user-generated content and only scan the part after the last
. We also need to avoid propagating $? from curl unnecessarily when we can continue safely. Finally, update v2mirror.t with tests to use PublicInbox::WWW from our "v1.1.0-pre1" tag to ensure these code paths get tested --- lib/PublicInbox/TestCommon.pm | 1 + 1 file changed, 1 insertion(+) (limited to 'lib/PublicInbox/TestCommon.pm') diff --git a/lib/PublicInbox/TestCommon.pm b/lib/PublicInbox/TestCommon.pm index aff34853..cd706e0e 100644 --- a/lib/PublicInbox/TestCommon.pm +++ b/lib/PublicInbox/TestCommon.pm @@ -469,6 +469,7 @@ sub start_script { $ENV{LISTEN_PID} = $$; $ENV{LISTEN_FDS} = $fds; } + if ($opt->{-C}) { chdir($opt->{-C}) or die "chdir: $!" } $0 = join(' ', @$cmd); if ($sub) { eval { PublicInbox::DS->Reset }; -- cgit v1.2.3-24-ge0c7