diff options
Diffstat (limited to 'Documentation/public-inbox-clone.pod')
-rw-r--r-- | Documentation/public-inbox-clone.pod | 211 |
1 files changed, 206 insertions, 5 deletions
diff --git a/Documentation/public-inbox-clone.pod b/Documentation/public-inbox-clone.pod index efee01ee..64ee3138 100644 --- a/Documentation/public-inbox-clone.pod +++ b/Documentation/public-inbox-clone.pod @@ -4,7 +4,9 @@ public-inbox-clone - "git clone --mirror" wrapper =head1 SYNOPSIS -public-inbox-clone INBOX_URL [INBOX_DIR] +public-inbox-clone [OPTIONS] INBOX_URL [INBOX_DIR] + +public-inbox-clone [OPTIONS] ROOT_URL [DESTINATION] # public-inbox 2.0+ =head1 DESCRIPTION @@ -13,15 +15,29 @@ making the initial clone of a remote HTTP(S) public-inbox. It allows cloning multi-epoch v2 inboxes with a single command and zero configuration. +In public-inbox 2.0+, public-inbox-clone can create and maintain +a mirror of multiple inboxes or code repositories using manifest.js.gz +files like L<grok-pull(1)> from grokmirror. L<public-inbox-fetch(1)> is +NOT required when using this mode. + It does not run L<public-inbox-init(1)> nor L<public-inbox-index(1)>. Those commands must be run separately if serving/searching the mirror is required. As-is, -public-inbox-clone is suitable for creating a git-only backup. +public-inbox-clone is suitable for creating a git-only backup +without Xapian and SQLite indices. + +When cloning a single inbox, public-inbox-clone creates a Makefile +with handy targets to update the inbox once indexed. +This Makefile may be edited by the user; it will +not be rewritten by L<public-inbox-fetch(1)> unless it is removed +completely. public-inbox-clone does not use nor require any extra -configuration files (not even C<~/.public-inbox/config>). +configuration files (not even C<~/.public-inbox/config>), +but it can download snippets suitable for adding to any +L<public-inbox-config(5)> file. -L<public-inbox-fetch(1)> may be used to keep C<INBOX_DIR> +L<public-inbox-fetch(1)> may be used to keep a single C<INBOX_DIR> up-to-date. For v2 inboxes, it will create a C<$INBOX_DIR/manifest.js.gz> @@ -46,6 +62,157 @@ C<--epoch=~2..> clones the three latest epochs. Default: C<0..~0> or C<0..> or C<..~0> (all epochs, all three examples are equivalent) +=item -I PATTERN + +=item --include=PATTERN + +When cloning a top-level with multiple inboxes via manifest, +only clone inboxes and repositories matching a given wildcard pattern +(using C<*?> and C<[]> is supported). + +This is a new option in public-inbox 2.0+ + +=item --exclude=PATTERN + +When cloning a top-level with multiple inboxes via manifest, +ignore inboxes and repositories matching the given wildcard pattern. +Supports the same wildcards as L</--include> + +This is a new option in public-inbox 2.0+ + +=item --inbox-config=always|v2|v1|never + +Whether or not to retrieve the C<$INBOX/_/text/config/raw> HTTP(S) +endpoint when cloning. + +Since we can't deduce v1 inboxes from code repositories, setting this +to C<v2> or C<never> can allow faster clones of code repositories if +no v1 inboxes are present. + +Default: C<always> + +This is a new option in public-inbox 2.0+ + +=item --inbox-version=NUM + +Force a remote public-inbox version (must be C<1> or C<2>). +This is auto-detected by default, and this option exists mainly +for testing. + +This is a new option in public-inbox 2.0+ + +=item --objstore=DIR + +Enables space savings when the remote C<manifest.js.gz> +includes C<forkgroup> entries as generated by grokmirror 2.x. + +If C<DIR> does not start with C</>, C<./>, or C<../>, it is treated +as relative to the C<DESTINATION> directory. If only C<--objstore=> +is specified where C<DIR> is an empty string (C<"">), then C<objstore> +(C<$DESTINATION/objstore>) is the implied value of C<DIR>. + +This is a new option in public-inbox 2.0+ + +=item --manifest=FILE + +When incrementally updating an existing mirror, load the given +manifest (typically C<manifest.js.gz>) to speed up updates. + +By default, public-inbox writes the retrieved manifest to +C<$DESTINATION/manifest.js.gz>, this directive also +changes the destination to the specified C<FILE> + +If C<FILE> does not start with C</>, C<./>, or C<../>, it is treated +as relative to the C<DESTINATION> directory. If only C<--manifest=> +is specified where C<FILE> is an empty string (C<"">), then C<manifest.js.gz> +(C<$DESTINATION/manifest.js.gz>) is the implied value of C<FILE>. + +When updating manifests with many forks using the same objstore, +git 2.41+ is highly recommended for performance as we automatically +use the C<fetch.hideRefs> feature to speed up negotiation. + +C<--manifest=> is a new option in public-inbox 2.0+ + +=item --remote-manifest=URL|RELATIVE_PATH + +Use an alternate location for the remote manifest.js.gz file. +This may be specified as a full absolute URL (e.g +C<--remote-manifest=https://80x24.org/lore/pub/manifest.js.gz>), +or a pathname relative to the ROOT_URL (e.g +C<--remote-manifest=pub/manifest.js.gz> when ROOT_URL is +C<https://80x24.org/lore/> + +By default, C<ROOT_URL/manifest.js.gz> is used. + +This is a new option in public-inbox 2.0+ + +=item --project-list=FILE + +When cloning code repos from a manifest, generate a cgit-compatible +project list. + +If C<FILE> does not start with C</>, C<./>, or C<../>, it is treated +as relative to the C<DESTINATION> directory. If only C<--project-list=> +is specified where C<FILE> is an empty string (C<"">), then C<projects.list> +(C<$DESTINATION/projects.list>) is the implied value of C<FILE>. + +This is a new option in public-inbox 2.0+ + +=item --post-update-hook=COMMAND + +Hooks to run after a repository is cloned or updated, C<COMMAND> will +have the bare git repository destination given as its first and only +argument. + +For v2 inboxes, this operates on a per-epoch basis. + +May be specified multiple times to run multiple commands in the +order specified on the command-line. + +This is a new option in public-inbox 2.0+ + +=item -p + +=item --prune + +Pass the C<--prune> and C<--prune-tags> flags to L<git-fetch(1)> +calls on incremental clones. + +This is a new option in public-inbox 2.0+ + +=item --purge + +Deletes entire repos which no longer exist in the remote manifest, +or are filtered out by C<--include=> or C<--exclude=>. + +This is only useful when using C<--manifest> + +This is a new option in public-inbox 2.0+ + +=item --exit-code + +Exit with C<127> if no updates are done when relying on a manifest. +Updates include fingerprint mismatches in the manifest, new symlinks, +new repositories, and removed repositories from the L<--project-list> + +This is a new option in public-inbox 2.0+ + +=item -k + +=item --keep-going + +Continue as much as possible after an error. + +This is a new option in public-inbox 2.0+ + +=item -n + +=item --dry-run + +Show what would be done, without making any changes. + +This is a new option in public-inbox 2.0+ + =item -q =item --quiet @@ -62,10 +229,44 @@ Increases verbosity, also passed to L<git-fetch(1)>. =item --no-torsocks -Whether to wrap L<git(1)> and L<curl(1)> commands with torsocks. +Whether to wrap L<git(1)> and L<curl(1)> commands with L<torsocks(1)>. Default: C<auto> +=item -j JOBS + +=item --jobs=JOBS + +The number of parallel processes to spawn at once for various network +operations using L<git(1)> and/or L<curl(1)>. + +=back + +=head1 EXAMPLES + +=for comment +Sticking to smaller projects in examples to minimize load on servers + +=over + +=item To mirror the most recent epochs of dwarves and LTTng inboxes: + + public-inbox-clone --epoch=~0 \ + --include='*lttng*' --include='*dwarves' \ + https://80x24.org/lore/ /path/to/inbox-mirror + +C<https://lore.kernel.org/> may be used instead of C<https://80x24.org/lore/> + +=item To mirror all code repos of the sparse project: + + public-inbox-clone --objstore= --project-list= --prune \ + --include='*sparse*' --inbox-config=never \ + --remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \ + https://80x24.org/lore/ /path/to/code-mirror + +C<https://git.kernel.org/> may be used instead of C<https://80x24.org/lore/> +and the C<--remote-manifest> option can be omitted. + =back =head1 CONTACT |