=head1 NAME public-inbox-clone - "git clone --mirror" wrapper =head1 SYNOPSIS public-inbox-clone [OPTIONS] INBOX_URL [INBOX_DIR] public-inbox-clone [OPTIONS] ROOT_URL [DESTINATION] # public-inbox 2.0+ =head1 DESCRIPTION public-inbox-clone is a wrapper around C for making the initial clone of a remote HTTP(S) public-inbox. It allows cloning multi-epoch v2 inboxes with a single command and zero configuration. In public-inbox 2.0+, public-inbox-clone can create and maintain a mirror of multiple inboxes or code repositories using manifest.js.gz files like L from grokmirror. L is NOT required when using this mode. It does not run L nor L. Those commands must be run separately if serving/searching the mirror is required. As-is, public-inbox-clone is suitable for creating a git-only backup without Xapian and SQLite indices. When cloning a single inbox, public-inbox-clone creates a Makefile with handy targets to update the inbox once indexed. This Makefile may be edited by the user; it will not be rewritten by L unless it is removed completely. public-inbox-clone does not use nor require any extra configuration files (not even C<~/.public-inbox/config>), but it can download snippets suitable for adding to any L file. L may be used to keep a single C up-to-date. For v2 inboxes, it will create a C<$INBOX_DIR/manifest.js.gz> file to speed up subsequent L. =head1 OPTIONS =over =item --epoch=RANGE Restrict clones of L inboxes to the given range of epochs. The range may be a single non-negative integer or a (possibly open-ended) C range of non-negative integers. C<~> may be prefixed to either (or both) integer values to represent the offset from the maximum possible value. For example, C<--epoch=~0> alone clones only the latest epoch, C<--epoch=~2..> clones the three latest epochs. Default: C<0..~0> or C<0..> or C<..~0> (all epochs, all three examples are equivalent) =item -I PATTERN =item --include=PATTERN When cloning a top-level with multiple inboxes via manifest, only clone inboxes and repositories matching a given wildcard pattern (using C<*?> and C<[]> is supported). This is a new option in public-inbox 2.0+ =item --exclude=PATTERN When cloning a top-level with multiple inboxes via manifest, ignore inboxes and repositories matching the given wildcard pattern. Supports the same wildcards as L This is a new option in public-inbox 2.0+ =item --inbox-config=always|v2|v1|never Whether or not to retrieve the C<$INBOX/_/text/config/raw> HTTP(S) endpoint when cloning. Since we can't deduce v1 inboxes from code repositories, setting this to C or C can allow faster clones of code repositories if no v1 inboxes are present. Default: C This is a new option in public-inbox 2.0+ =item --inbox-version=NUM Force a remote public-inbox version (must be C<1> or C<2>). This is auto-detected by default, and this option exists mainly for testing. This is a new option in public-inbox 2.0+ =item --objstore=DIR Enables space savings when the remote C includes C entries as generated by grokmirror 2.x. If C does not start with C, C<./>, or C<../>, it is treated as relative to the C directory. If only C<--objstore=> is specified where C is an empty string (C<"">), then C (C<$DESTINATION/objstore>) is the implied value of C. This is a new option in public-inbox 2.0+ =item --manifest=FILE When incrementally updating an existing mirror, load the given manifest (typically C) to speed up updates. By default, public-inbox writes the retrieved manifest to C<$DESTINATION/manifest.js.gz>, this directive also changes the destination to the specified C If C does not start with C, C<./>, or C<../>, it is treated as relative to the C directory. If only C<--manifest=> is specified where C is an empty string (C<"">), then C (C<$DESTINATION/manifest.js.gz>) is the implied value of C. When updating manifests with many forks using the same objstore, git 2.41+ is highly recommended for performance as we automatically use the C feature to speed up negotiation. C<--manifest=> is a new option in public-inbox 2.0+ =item --remote-manifest=URL|RELATIVE_PATH Use an alternate location for the remote manifest.js.gz file. This may be specified as a full absolute URL (e.g C<--remote-manifest=https://80x24.org/lore/pub/manifest.js.gz>), or a pathname relative to the ROOT_URL (e.g C<--remote-manifest=pub/manifest.js.gz> when ROOT_URL is C By default, C is used. This is a new option in public-inbox 2.0+ =item --project-list=FILE When cloning code repos from a manifest, generate a cgit-compatible project list. If C does not start with C, C<./>, or C<../>, it is treated as relative to the C directory. If only C<--project-list=> is specified where C is an empty string (C<"">), then C (C<$DESTINATION/projects.list>) is the implied value of C. This is a new option in public-inbox 2.0+ =item --post-update-hook=COMMAND Hooks to run after a repository is cloned or updated, C will have the bare git repository destination given as its first and only argument. For v2 inboxes, this operates on a per-epoch basis. May be specified multiple times to run multiple commands in the order specified on the command-line. This is a new option in public-inbox 2.0+ =item -p =item --prune Pass the C<--prune> and C<--prune-tags> flags to L calls on incremental clones. This is a new option in public-inbox 2.0+ =item --purge Deletes entire repos which no longer exist in the remote manifest, or are filtered out by C<--include=> or C<--exclude=>. This is only useful when using C<--manifest> This is a new option in public-inbox 2.0+ =item --exit-code Exit with C<127> if no updates are done when relying on a manifest. Updates include fingerprint mismatches in the manifest, new symlinks, new repositories, and removed repositories from the L<--project-list> This is a new option in public-inbox 2.0+ =item -k =item --keep-going Continue as much as possible after an error. This is a new option in public-inbox 2.0+ =item -n =item --dry-run Show what would be done, without making any changes. This is a new option in public-inbox 2.0+ =item -q =item --quiet Quiets down progress messages, also passed to L. =item -v =item --verbose Increases verbosity, also passed to L. =item --torsocks=auto|no|yes =item --no-torsocks Whether to wrap L and L commands with L. Default: C =item -j JOBS =item --jobs=JOBS The number of parallel processes to spawn at once for various network operations using L and/or L. =back =head1 EXAMPLES =for comment Sticking to smaller projects in examples to minimize load on servers =over =item To mirror the most recent epochs of dwarves and LTTng inboxes: public-inbox-clone --epoch=~0 \ --include='*lttng*' --include='*dwarves' \ https://80x24.org/lore/ /path/to/inbox-mirror C may be used instead of C =item To mirror all code repos of the sparse project: public-inbox-clone --objstore= --project-list= --prune \ --include='*sparse*' --inbox-config=never \ --remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \ https://80x24.org/lore/ /path/to/code-mirror C may be used instead of C and the C<--remote-manifest> option can be omitted. =back =head1 CONTACT Feedback welcome via plain-text mail to L The mail archives are hosted at L and L =head1 COPYRIGHT Copyright all contributors L License: AGPL-3.0+ L =head1 SEE ALSO L, L, L