user/dev discussion of public-inbox itself
 help / color / mirror / Atom feed
1397a7f4ef156d8527780eca99489de734e4b920 blob 3877 bytes (raw)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
 
=head1 NAME

public-inbox-xcpdb - upgrade Xapian DB formats

=head1 SYNOPSIS

public-inbox-xcpdb [OPTIONS] INBOX_DIR

public-inbox-xcpdb [OPTIONS] --all

=head1 DESCRIPTION

public-inbox-xcpdb is similar to L<copydatabase(1)> for
upgrading to the latest database format supported by Xapian
(e.g. "glass" or "honey"), but is designed to tolerate and
accept parallel Xapian database modifications from
L<public-inbox-watch(1)>, L<public-inbox-mda(1)>,
L<public-inbox-learn(1)>, and L<public-inbox-index(1)>.

This command is rarely used, as Xapian DB formats rarely
change.

=head1 OPTIONS

=over

=item --all

Copy all inboxes configured in ~/.public-inbox/config.
This is an alternative to specifying individual inboxes directories
on the command-line.

=item -c, --compact

In addition to performing the copy operation, run L<xapian-compact(1)>
on each Xapian shard after copying but before finalizing it.
Compared to the cost of copying a Xapian database, compacting a
Xapian database takes only around 5% of the time required to copy.

Compared to L<public-inbox-compact(1)>, use of this option is
preferable for gigantic inboxes where the coarse-grained lock
currently required for L<public-inbox-compact(1)> can cause
the compaction to take hours at-a-time.

=item --reshard=N / -R N

Reshard the Xapian database on a L<v2|public-inbox-v2-format(5)>
inbox to C<N> shards .  Since L<xapian-compact(1)> is not suitable
for merging, users can rely on this switch to reshard the
existing Xapian database(s) to any positive value of C<N>.

This is useful in case the Xapian DB was created with too few or
too many shards given the capabilities of the current hardware.

=item --blocksize / --no-full / --fuller

These options are passed directly to L<xapian-compact(1)> when
used with C<--compact>.

=item --no-fsync

Disable L<fsync(2)> and L<fdatasync(2)>.
See L<public-inbox-index(1)/--no-fsync> for caveats.

Available in public-inbox 1.6.0 (PENDING).

=item --sequential-shard

Copy each shard sequentially, ignoring C<--jobs>.  This also
affects indexing done at the end of a run.

=item --batch-size=BYTES

=item --max-size=BYTES

See L<public-inbox-index(1)> for a description of these options.

These indexing options indexing at the end of a run.
C<public-inbox-xcpdb> may run in parallel with with
L<public-inbox-index(1)>, and C<public-inbox-xcpdb> needs to
reindex changes made to the old Xapian DBs by
L<public-inbox-index(1)> while it was running.

=back

=head1 ENVIRONMENT

=over 8

=item PI_CONFIG

The default config file, normally "~/.public-inbox/config".
See L<public-inbox-config(5)>

=item XAPIAN_FLUSH_THRESHOLD

The number of documents to update before committing changes to
disk.  This environment is handled directly by Xapian, refer to
Xapian API documentation for more details.

Default: 10000

=back

=head1 UPGRADING

This tool is intended for admins upgrading Xapian search databases
used by public-inbox, NOT users upgrading public-inbox itself.

In particular, it DOES NOT upgrade the schema used by the
PSGI search interface (see L<public-inbox-index(1)>).

=head1 LIMITATIONS

Do not use L<public-inbox-purge(1)> or L<public-inbox-edit(1)>
while this is running; old (purged or edited data) may show up.

Normal invocations L<public-inbox-index(1)> can safely run
while this is running, too.  However, reindexing via the
L<public-inbox-index(1)/--reindex> switch will be a waste of
computing resources.

=head1 CONTACT

Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>

The mail archives are hosted at L<https://public-inbox.org/meta/>
and L<http://hjrcffqmbrq6wope.onion/meta/>

=head1 COPYRIGHT

Copyright 2019-2020 all contributors L<mailto:meta@public-inbox.org>

License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>

=head1 SEE ALSO

L<copydatabase(1)>, L<xapian-compact(1)>, L<public-inbox-index(1)>
debug log:

solving 1397a7f4 ...
found 1397a7f4 in https://80x24.org/public-inbox.git

user/dev discussion of public-inbox itself

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 meta meta/ http://public-inbox.org/meta \
		meta@public-inbox.org
	public-inbox-index meta

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git