NAME
    public-inbox-index - create and update search indices

SYNOPSIS
    public-inbox-index [OPTIONS] GIT_DIR

DESCRIPTION
    public-inbox-index creates and updates the search and NNTP article
    number database used by the read-only public-inbox HTTP and NNTP
    interfaces. Currently, this requires Search::Xapian and DBD::SQlite and
    DBI Perl modules.

    Once the initial indices are created by public-inbox-index,
    public-inbox-mda(1) and public-inbox-watch(1) will automatically
    maintain them.

    Running this manually to update indices is only required if relying on
    git-fetch(1) to mirror an existing public-inbox; or if upgrading to a
    new version of public-inbox using the "--reindex" option.

    Having a search and article number database is essential to running the
    NNTP interface, and strongly recommended for the HTTP interface as it
    provides thread grouping in addition to normal search functionality.

OPTIONS
    --reindex
        Forces a search engine re-index of all messages in the repository.
        This can be used for in-place upgrades while NNTP/HTTP server
        processes are utilizing the index. Keep in mind this roughly doubles
        the size of the already-large Xapian database.

        This does not touch the NNTP article number database.

FILES
    All public-inbox-specific files are contained within the
    "$GIT_DIR/public-inbox/" directory. All files are expected to grow in
    size as more messages are archived, so using compaction commands (e.g.
    xapian-compact(1)) is not recommended unless the list is no longer
    active.

    $GIT_DIR/public-inbox/msgmap.sqlite3
        The stable NNTP article number to Message-ID mapping is stored in an
        SQLite3 database.

        This is required for users of public-inbox-nntpd(1), but users of
        the PublicInbox::WWW interface will find it useful for attempting
        recovery from copy-paste truncations of URLs containing long
        Message-IDs.

        Avoid removing this file and regenerating it; it may cause existing
        NNTP readers to lose sync and miss (or see duplicate) messages.

        This file is relatively small, and typically less than 5% of the
        space of the mail stored in a packed git repository.

    $GIT_DIR/public-inbox/xapian*
        The database used by Search::Xapian. This directory name is followed
        by a number indicating the index schema version this installation of
        public-inbox uses.

        These directories may be safely deleted or removed in full while the
        NNTP and HTTP interfaces are no longer accessing them.

        In addition to providing a search interface for the HTTP interface,
        the Xapian database is used to group and combine related messages
        into threads. For NNTP servers, it also provides a cache of metadata
        and header information often requested by NNTP clients.

        This directory is large, often two to three times the size of the
        objects stored in a packed git repository. Using the "--reindex"
        option makes it larger, still.

ENVIRONMENT
    PI_CONFIG
            Used to override the default "~/.public-inbox/config" value.

CONTACT
    Feedback welcome via plain-text mail to <mailto:meta@public-inbox.org>

    The mail archives are hosted at <https://public-inbox.org/meta/> and
    <http://hjrcffqmbrq6wope.onion/meta/>

COPYRIGHT
    Copyright 2016 all contributors <mailto:meta@public-inbox.org>

    License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>

SEE ALSO
    Search::Xapian, DBD::SQLite