public-inbox.git  about / heads / tags
an "archives first" approach to mailing lists
blob e7c07ed38d4085c9c5b6bbdc7954f50aa8b2a68a 3929 bytes (raw)
$ git show HEAD:Documentation/public-inbox-xcpdb.pod	# shows this blob on the CLI

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
 
=head1 NAME

public-inbox-xcpdb - upgrade Xapian DB formats

=head1 SYNOPSIS

public-inbox-xcpdb [OPTIONS] INBOX_DIR

public-inbox-xcpdb [OPTIONS] --all

=head1 DESCRIPTION

public-inbox-xcpdb is similar to L<copydatabase(1)> for
upgrading to the latest database format supported by Xapian
(e.g. "glass" or "honey"), but is designed to tolerate and
accept parallel Xapian database modifications from
L<public-inbox-watch(1)>, L<public-inbox-mda(1)>,
L<public-inbox-learn(1)>, and L<public-inbox-index(1)>.

This command is rarely used, as Xapian DB formats rarely
change.

=head1 OPTIONS

=over

=item --all

Copy all inboxes configured in ~/.public-inbox/config.
This is an alternative to specifying individual inboxes directories
on the command-line.

=item -c

=item --compact

In addition to performing the copy operation, run L<xapian-compact(1)>
on each Xapian shard after copying but before finalizing it.
Compared to the cost of copying a Xapian database, compacting a
Xapian database takes only around 5% of the time required to copy.

Compared to L<public-inbox-compact(1)>, use of this option is
preferable for gigantic inboxes where the coarse-grained lock
currently required for L<public-inbox-compact(1)> can cause
the compaction to take hours at-a-time.

=item -R N

=item --reshard=N

Reshard the Xapian database on a L<v2|public-inbox-v2-format(5)>
inbox to C<N> shards .  Since L<xapian-compact(1)> is not suitable
for merging, users can rely on this switch to reshard the
existing Xapian database(s) to any positive value of C<N>.

This is useful in case the Xapian DB was created with too few or
too many shards given the capabilities of the current hardware.

=item --blocksize

=item --no-full

=item --fuller

These options are passed directly to L<xapian-compact(1)> when
used with C<--compact>.

=item --no-fsync

Disable L<fsync(2)> and L<fdatasync(2)>.
See L<public-inbox-index(1)/--no-fsync> for caveats.

Available in public-inbox 1.6.0+.

=item --sequential-shard

Copy each shard sequentially, ignoring C<--jobs>.  This also
affects indexing done at the end of a run.

=item --batch-size=BYTES

=item --max-size=BYTES

See L<public-inbox-index(1)> for a description of these options.

These indexing options indexing at the end of a run.
C<public-inbox-xcpdb> may run in parallel with with
L<public-inbox-index(1)>, and C<public-inbox-xcpdb> needs to
reindex changes made to the old Xapian DBs by
L<public-inbox-index(1)> while it was running.

=back

=head1 ENVIRONMENT

=over 8

=item PI_CONFIG

The default config file, normally "~/.public-inbox/config".
See L<public-inbox-config(5)>

=item XAPIAN_FLUSH_THRESHOLD

The number of documents to update before committing changes to
disk.  This environment is handled directly by Xapian, refer to
Xapian API documentation for more details.

Default: 10000

=back

=head1 UPGRADING

This tool is intended for admins upgrading Xapian search databases
used by public-inbox, NOT users upgrading public-inbox itself.

In particular, it DOES NOT upgrade the schema used by the
PSGI search interface (see L<public-inbox-index(1)>).

=head1 LIMITATIONS

Do not use L<public-inbox-purge(1)> or L<public-inbox-edit(1)>
while this is running; old (purged or edited data) may show up.

Normal invocations L<public-inbox-index(1)> can safely run
while this is running, too.  However, reindexing via the
L<public-inbox-index(1)/--reindex> switch will be a waste of
computing resources.

=head1 CONTACT

Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>

The mail archives are hosted at L<https://public-inbox.org/meta/> and
L<http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>

=head1 COPYRIGHT

Copyright 2019-2021 all contributors L<mailto:meta@public-inbox.org>

License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>

=head1 SEE ALSO

L<copydatabase(1)>, L<xapian-compact(1)>, L<public-inbox-index(1)>

git clone https://public-inbox.org/public-inbox.git
git clone http://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/public-inbox.git