* [PATCH] doc: update public-inbox-clone examples and help
@ 2023-03-05 22:18 7% Eric Wong
0 siblings, 0 replies; 1+ results
From: Eric Wong @ 2023-03-05 22:18 UTC (permalink / raw)
To: meta
Basically, public-inbox-clone has become grok-pull without
config files nor absolute paths.
---
Documentation/public-inbox-clone.pod | 62 ++++++++++++++++++++++------
script/public-inbox-clone | 12 ++++--
2 files changed, 57 insertions(+), 17 deletions(-)
diff --git a/Documentation/public-inbox-clone.pod b/Documentation/public-inbox-clone.pod
index 7774d4ed..af4e8e95 100644
--- a/Documentation/public-inbox-clone.pod
+++ b/Documentation/public-inbox-clone.pod
@@ -15,14 +15,16 @@ making the initial clone of a remote HTTP(S) public-inbox. It
allows cloning multi-epoch v2 inboxes with a single command and
zero configuration.
-In public-inbox 2.0+, public-inbox-clone can also mirror multiple
-inboxes or code repositories using manifest.js.gz files like
-L<grok-pull(1)> from grokmirror.
+In public-inbox 2.0+, public-inbox-clone can create and maintain
+a mirror of multiple inboxes or code repositories using manifest.js.gz
+files like L<grok-pull(1)> from grokmirror. L<public-inbox-fetch(1)> is
+NOT required when using this mode.
It does not run L<public-inbox-init(1)> nor
L<public-inbox-index(1)>. Those commands must be run separately
if serving/searching the mirror is required. As-is,
-public-inbox-clone is suitable for creating a git-only backup.
+public-inbox-clone is suitable for creating a git-only backup
+without Xapian and SQLite indices.
When cloning a single inbox, public-inbox-clone creates a Makefile
with handy targets to update the inbox once indexed.
@@ -31,11 +33,12 @@ not be rewritten by L<public-inbox-fetch(1)> unless it is removed
completely.
public-inbox-clone does not use nor require any extra
-configuration files (not even C<~/.public-inbox/config>).
+configuration files (not even C<~/.public-inbox/config>),
+but it can download snippets suitable for adding to any
+L<public-inbox-config(5)> file.
L<public-inbox-fetch(1)> may be used to keep a single C<INBOX_DIR>
-up-to-date. When using manifest.js.gz, public-inbox-clone is used
-to maintain the multi-repository mirror.
+up-to-date.
For v2 inboxes, it will create a C<$INBOX_DIR/manifest.js.gz>
file to speed up subsequent L<public-inbox-fetch(1)>.
@@ -63,17 +66,17 @@ Default: C<0..~0> or C<0..> or C<..~0>
=item --include=PATTERN
-When cloning a top-level with multiple inboxes, only clone inboxes and
-repositories matching a given wildcard pattern (using C<*?> and C<[]> is
-supported).
+When cloning a top-level with multiple inboxes via manifest,
+only clone inboxes and repositories matching a given wildcard pattern
+(using C<*?> and C<[]> is supported).
This is a new option in public-inbox 2.0+
=item --exclude=PATTERN
-When cloning a top-level with multiple inboxes, ignore inboxes and
-repositories matching the given wildcard pattern. Supports the same
-wildcards as L</--include>
+When cloning a top-level with multiple inboxes via manifest,
+ignore inboxes and repositories matching the given wildcard pattern.
+Supports the same wildcards as L</--include>
This is a new option in public-inbox 2.0+
@@ -204,6 +207,39 @@ Whether to wrap L<git(1)> and L<curl(1)> commands with L<torsocks(1)>.
Default: C<auto>
+=item -j JOBS
+=item --jobs=JOBS
+
+The number of parallel processes to spawn at once for various network
+operations using L<git(1)> and/or L<curl(1)>.
+
+=back
+
+=head1 EXAMPLES
+
+=for comment
+Sticking to smaller projects in examples to minimize load on servers
+
+=over
+
+=item To mirror the most recent epochs of dwarves and LTTng inboxes:
+
+ public-inbox-clone --epoch=~0 \
+ --include='*lttng*' --include='*dwarves' \
+ https://80x24.org/lore/ /path/to/inbox-mirror
+
+C<https://lore.kernel.org/> may be used instead of C<https://80x24.org/lore/>
+
+=item To mirror all code repos of the sparse project:
+
+ public-inbox-clone --objstore= --project-list= --prune \
+ --include='*sparse*' --inbox-config=never \
+ --remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \
+ https://80x24.org/lore/ /path/to/code-mirror
+
+C<https://git.kernel.org/> may be used instead of C<https://80x24.org/lore/>
+and the C<--remote-manifest> option can be omitted.
+
=back
=head1 CONTACT
diff --git a/script/public-inbox-clone b/script/public-inbox-clone
index 10ad3487..5b365df7 100755
--- a/script/public-inbox-clone
+++ b/script/public-inbox-clone
@@ -6,21 +6,25 @@ use v5.12;
use Getopt::Long qw(:config gnu_getopt no_ignore_case auto_abbrev);
my $opt = {};
my $help = <<EOF; # the following should fit w/o scrolling in 80x24 term:
-usage: public-inbox-clone INBOX_URL [DESTINATION]
+usage: public-inbox-clone [OPTIONS] INBOX_URL [INBOX_DIR]
+ public-inbox-clone [OPTIONS] ROOT_URL [DESTINATION]
- clone remote public-inboxes
+ clone remote public-inboxes or grokmirror manifests
options:
--epoch=RANGE range of v2 epochs to clone (e.g `2..5', `~0', `~1..')
- --objstore=DIR share storage for coderepos
--torsocks VAL whether or not to wrap git and curl commands with
torsocks (default: `auto')
Must be one of: `auto', `no' or `yes'
--dry-run | -n show what would be cloned without cloning
--verbose | -v increase verbosity (may be repeated)
- --quiet | -q increase verbosity (may be repeated)
+ --quiet | -q disable progress reporting
-C DIR chdir to specified directory
+
+See public-inbox-clone(1) man page for --manifest, --remote-manifest,
+--objstore, --project-list, --post-update-hook, --include, --exclude,
+--prune, --keep-going, --jobs, --inbox-config
EOF
# cgit calls it `project-list', grokmirror calls it `projectslist',
^ permalink raw reply related [relevance 7%]
Results 1-1 of 1 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2023-03-05 22:18 7% [PATCH] doc: update public-inbox-clone examples and help Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).