From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 953BC1F452 for ; Sun, 6 Aug 2023 02:31:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1691289116; bh=KXq1VjK5pYXwxfUykFE/QmAzkMa0DU6sPAsY7akl1No=; h=From:To:Subject:Date:In-Reply-To:References:From; b=JK4OnLOVYzWY27HDymsjOB8/n4ipgdEa+rG9M41k1hc1mjA4Rl96k2qVh4anxDaGM iCd3GWr62VaUXRtfQZALn+qDj4Vyca0i5eFeAHbMFlT5kSFyiuqX7GCeBzQxMSZk++ +O7gIVj7d4azFZScBQVJhm4fLGvfzNbCL7a1wFYk= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 1/2] doc: add manpage for -cindex Date: Sun, 6 Aug 2023 02:31:55 +0000 Message-Id: <20230806023156.3483821-2-e@80x24.org> In-Reply-To: <20230806023156.3483821-1-e@80x24.org> References: <20230806023156.3483821-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: It's similar to a combination of -index and -extindex but perhaps more refined this time around... --- Documentation/public-inbox-cindex.pod | 141 ++++++++++++++++++++++++++ MANIFEST | 1 + 2 files changed, 142 insertions(+) create mode 100644 Documentation/public-inbox-cindex.pod diff --git a/Documentation/public-inbox-cindex.pod b/Documentation/public-inbox-cindex.pod new file mode 100644 index 00000000..93149e62 --- /dev/null +++ b/Documentation/public-inbox-cindex.pod @@ -0,0 +1,141 @@ +=head1 NAME + +public-inbox-index - create and update search for code repositories + +=head1 SYNOPSIS + +public-inbox-cindex [OPTIONS] GIT_DIR... + +public-inbox-cindex [OPTIONS] --update + +=head1 DESCRIPTION + +public-inbox-cindex creates and updates the Xapian search index for +git code repository (C) search. It can also associate +(fuzzy join) coderepos with Xapian-indexed inboxes. It only indexes +commit messages and diffs as they would show up in a an email. It +does not index the contents of blobs directly. + +Like inbox indices, coderepo indices can either be internal or external +to a coderepo. Either way, they're both created and updated through +public-inbox-cindex. + +Once the initial indices are created by public-inbox-cindex, +the L switch will incrementally update them. + +=head1 OPTIONS + +=over + +=item -d EXTDIR + +Use the given directory as an external index. External indices are +generally recommended to internal indices since they do not need +write access to any code repositories themselves. They are highly +recommended when many repositories share a common history or if +there is an M:N relationship between inboxes and coderepos. + +=item -j JOBS + +=item --jobs=JOBS + +Influences the number of Xapian indexing shards. + +If the repo has not been indexed or initialized, C +shards will be created. + +Default: the number of existing Xapian shards + +=item --reindex + +Forces a re-index of all commits. This can be used for in-place +upgrades and bugfixes while read-only processes are utilizing the index. + +=item --update + +=item -u + +Incrementally index all previously-indexed coderepos. + +=item --prune + +Removes commits which are no longer accessible via git. +Use this after L (or L). + +=item --no-fsync + +Disables L and L operations on SQLite +and Xapian. This is only effective with Xapian 1.4+. This is +primarily intended for systems with low RAM and the small +(default) C<--batch-size=1m>. Users of large C<--batch-size> +may even find disabling L causes too much dirty +data to accumulate, resulting on latency spikes from writeback. + +=item --max-size SIZE + +=item --batch-size SIZE + +These affect the coderepo index the same way they affect +inbox indices. See L. + +=back + +=head1 FILES + +For internal indices, the Xapian DB is stored in +C<$GIT_DIR/public-inbox-cindex>. + +External indices are stored wherever L EXTDIR points. + +=head1 CONFIGURATION + +=over 8 + +=item publicinbox.indexMaxSize + +=item publicinbox.indexBatchSize + +These configuration knobs affect the coderepo index the same way +they affect inbox indices. See L. + +=back + +=head1 ENVIRONMENT + +=over 8 + +=item PI_CONFIG + +Used to override the default "~/.public-inbox/config" value. + +=item XAPIAN_FLUSH_THRESHOLD + +The number of documents to update before committing changes to +disk. This environment is handled directly by Xapian, refer to +Xapian API documentation for more details. + +Use C instead. + +=back + +=head1 UPGRADING + +Occasionally, public-inbox will update it's schema version and +require a full reindex by running this command with L. + +=head1 CONTACT + +Feedback welcome via plain-text mail to L + +The mail archives are hosted at L and +L + +=head1 COPYRIGHT + +Copyright all contributors L + +License: AGPL-3.0+ L + +=head1 SEE ALSO + +L diff --git a/MANIFEST b/MANIFEST index 44eaa497..1001ca08 100644 --- a/MANIFEST +++ b/MANIFEST @@ -67,6 +67,7 @@ Documentation/lei.pod Documentation/lei_design_notes.txt Documentation/marketing.txt Documentation/mknews.perl +Documentation/public-inbox-cindex.pod Documentation/public-inbox-clone.pod Documentation/public-inbox-compact.pod Documentation/public-inbox-config.pod