1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
| | =head1 NAME
public-inbox-index - create and update search indices
=head1 SYNOPSIS
public-inbox-index [OPTIONS] REPO_DIR
=head1 DESCRIPTION
public-inbox-index creates and updates the search and NNTP
article number database used by the read-only public-inbox HTTP
and NNTP interfaces. Currently, this requires L<Search::Xapian>
and L<DBD::SQlite> and L<DBI> Perl modules.
Once the initial indices are created by public-inbox-index,
L<public-inbox-mda(1)> and L<public-inbox-watch(1)> will
automatically maintain them.
Running this manually to update indices is only required if
relying on L<git-fetch(1)> to mirror an existing public-inbox;
or if upgrading to a new version of public-inbox using
the C<--reindex> option.
Having a search and article number database is essential to
running the NNTP interface, and strongly recommended for the
HTTP interface as it provides thread grouping in addition
to normal search functionality.
=head1 OPTIONS
=over
=item --reindex
Forces a search engine re-index of all messages in the
repository. This can be used for in-place upgrades while
NNTP/HTTP server processes are utilizing the index. Keep in
mind this roughly doubles the size of the already-large
Xapian database.
This does not touch the NNTP article number database.
=back
=head1 FILES
All public-inbox-specific files are contained within the
C<$REPO_DIR/public-inbox/> directory. All files are expected to
grow in size as more messages are archived, so using compaction
commands (e.g. L<xapian-compact(1)>) is not recommended unless
the list is no longer active.
=over
=item $REPO_DIR/public-inbox/msgmap.sqlite3
The stable NNTP article number to Message-ID mapping is
stored in an SQLite3 database.
This is required for users of L<public-inbox-nntpd(1)>, but
users of the L<PublicInbox::WWW> interface will find it
useful for attempting recovery from copy-paste truncations of
URLs containing long Message-IDs.
Avoid removing this file and regenerating it; it may cause
existing NNTP readers to lose sync and miss (or see duplicate)
messages.
This file is relatively small, and typically less than 5%
of the space of the mail stored in a packed git repository.
=item $REPO_DIR/public-inbox/xapian*
The database used by L<Search::Xapian>. This directory name is
followed by a number indicating the index schema version this
installation of public-inbox uses.
These directories may be safely deleted or removed in full
while the NNTP and HTTP interfaces are no longer accessing
them.
In addition to providing a search interface for the HTTP
interface, the Xapian database is used to group and combine
related messages into threads. For NNTP servers, it also
provides a cache of metadata and header information often
requested by NNTP clients.
This directory is large, often two to three times the size of
the objects stored in a packed git repository. Using the
C<--reindex> option makes it larger, still.
=back
=head1 ENVIRONMENT
=over 8
=item PI_CONFIG
Used to override the default "~/.public-inbox/config" value.
=back
=head1 CONTACT
Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>
The mail archives are hosted at L<https://public-inbox.org/meta/>
and L<http://hjrcffqmbrq6wope.onion/meta/>
=head1 COPYRIGHT
Copyright 2016-2018 all contributors L<mailto:meta@public-inbox.org>
License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>
=head1 SEE ALSO
L<Search::Xapian>, L<DBD::SQLite>
|