diff options
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/RelNotes/v2.0.0.wip | 5 | ||||
-rw-r--r-- | Documentation/public-inbox-daemon.pod | 38 | ||||
-rw-r--r-- | Documentation/public-inbox-extindex.pod | 13 | ||||
-rw-r--r-- | Documentation/public-inbox-index.pod | 7 | ||||
-rw-r--r-- | Documentation/public-inbox-tuning.pod | 10 |
5 files changed, 64 insertions, 9 deletions
diff --git a/Documentation/RelNotes/v2.0.0.wip b/Documentation/RelNotes/v2.0.0.wip index 4d872fd7..f04d8144 100644 --- a/Documentation/RelNotes/v2.0.0.wip +++ b/Documentation/RelNotes/v2.0.0.wip @@ -54,8 +54,9 @@ treewide * SHA-256 coderepos are fully supported (but not inboxes, yet) - * jemalloc (tested as an LD_PRELOAD) is recommended to reduce fragmentation - in long-running daemon processes serving unpredictable traffic + * for daemons serving public traffic, MALLOC_MMAP_THRESHOLD_=131072 is + recommended to reduce fragmentation in glibc malloc, while jemalloc + (tested as an LD_PRELOAD) is another option (at least for 64-bit). PublicInbox::WWW diff --git a/Documentation/public-inbox-daemon.pod b/Documentation/public-inbox-daemon.pod index 6f1e3b53..092be667 100644 --- a/Documentation/public-inbox-daemon.pod +++ b/Documentation/public-inbox-daemon.pod @@ -79,9 +79,9 @@ C<err=> may also be specified on a per-listener basis. Default: /dev/null with C<--daemonize>, inherited otherwise -=item -W +=item -W INTEGER -=item --worker-processes +=item --worker-processes INTEGER Set the number of worker processes. @@ -96,6 +96,40 @@ the master on crashes. Default: 1 +=item -X INTEGER + +=item --xapian-helpers INTEGER + +Enables the use of Xapian helper processes to handle expensive, +non-deterministic Xapian search queries asynchronously without +blocking simple requests. + +With positive values, there is an additional manager process +that can be signaled to control the number of Xapian helper workers. + +* C<-X0> one worker, no manager process +* C<-X1> one worker, one manager process +... +* C<-X8> eight workers, one manager process + +As with the public-facing public-inbox-* daemons, sending C<SIGTTIN> +or C<SIGTTOU> to the Xapian helper manager process will increment or +decrement the number of workers. + +Both Xapian helper workers and managers automatically respawn if they +crash or are explicitly killed, even with C<-X0>. + +A C++ compiler, L<pkg-config(1)>, and Xapian development files (e.g. +C<libxapian-dev> or C<xapian*-core-dev*>) are required to gain access to +some expensive queries and significant memory savings. + +Xapian helper workers are shared by all C<--worker-processes> of the +Perl daemon for additional memory savings. + +New in public-inbox 2.0.0. + +Default: undefined, search queries are handled synchronously + =item --cert /path/to/cert The default TLS certificate for HTTPS, IMAPS, NNTPS, POP3S and/or STARTTLS diff --git a/Documentation/public-inbox-extindex.pod b/Documentation/public-inbox-extindex.pod index b53e45ed..2db7d7e9 100644 --- a/Documentation/public-inbox-extindex.pod +++ b/Documentation/public-inbox-extindex.pod @@ -80,6 +80,19 @@ doubles the size of the already-large Xapian database. Used with C<--reindex>, it will only look for new and stale entries and not touch already-indexed messages. +=item --no-multi-pack-index + +Disable writing a L<git-multi-pack-index(1)> file to save memory. +Normally, enabling multi-pack-index speeds up startup time of +subsequent L<git-cat-file(1)> processes by 3-4%, but generating +this file requires several GB of memory with large repos. + +Unlike the C<core.multiPackIndex> directive in git, it's still +possible to read existing multi-pack-index files if they are +created elsewhere. + +Available in public-inbox 2.0.0+ + =back =head1 FILES diff --git a/Documentation/public-inbox-index.pod b/Documentation/public-inbox-index.pod index 14f157a5..f1a2180a 100644 --- a/Documentation/public-inbox-index.pod +++ b/Documentation/public-inbox-index.pod @@ -192,6 +192,13 @@ external indices are configured. Do not update the C<all> external index by default. This negates all uses of C<-E> / C<--update-extindex=> on the command-line. +=item --no-multi-pack-index + +Disables writing the multi-pack-index when using L</--update-extindex>. +See L<public-inbox-extindex(1)/--no-multi-pack-index> for details. + +Available in public-inbox 2.0.0+ + =item --since=DATESTRING =item --after=DATESTRING diff --git a/Documentation/public-inbox-tuning.pod b/Documentation/public-inbox-tuning.pod index 7d0690b4..b56c2b10 100644 --- a/Documentation/public-inbox-tuning.pod +++ b/Documentation/public-inbox-tuning.pod @@ -165,11 +165,11 @@ capacity planning. Bursts of small object allocations late in process life contribute to fragmentation of the heap due to arenas (slabs) used internally by Perl. -jemalloc (tested as an LD_PRELOAD on GNU/Linux) reduces -overall fragmentation compared to glibc malloc in long-lived processes. -glibc malloc users may try setting C<MALLOC_MMAP_THRESHOLD_> to a lower -value (e.g. 131072) but that may require increasing the -C<sys.vm.max_map_count> sysctl. +glibc malloc users should use C<MALLOC_MMAP_THRESHOLD_=131072> to reduce +fragmentation from the sliding mmap window. On 64-bit systems, jemalloc +(tested as an LD_PRELOAD on GNU/Linux) reduces fragmentation at the +expense of VM space. 32-bit systems may be better off sticking with +glibc and MALLOC_MMAP_THRESHOLD_. =head2 Other OS tuning knobs |