diff options
| author | Eric Wong <e@80x24.org> | 2022-11-14 08:07:02 +0000 |
|---|---|---|
| committer | Eric Wong <e@80x24.org> | 2022-11-16 02:42:39 +0000 |
| commit | f7ac353a6e88abdae687cc30d7562a2171684d51 (patch) | |
| tree | e0fb36cd0e51a6c01a002039ee6a4810dc90f54c | |
| parent | 4d78e55e9d6ca77f8b1ef0f19e44cc33cc2cb981 (diff) | |
| download | public-inbox-f7ac353a6e88abdae687cc30d7562a2171684d51.tar.gz | |
Eric Wong <e@80x24.org> wrote: > Thanks for confirming things work as intended. I think the > default should be clamped, though... 15 seems a bit high for > smaller IMAP servers *shrug* --------8<------- Subject: [PATCH] lei q|up: limit default write --jobs for IMAP(S) IMAP(S) servers often limit per-user connections, so avoid bumping into limits to improve the out-of-the-box experience. 4 seems like a conservative default, since we already chose that number for remote HTTP(S) endpoints. Link: https://public-inbox.org/meta/20220910201958.GA12212@dcvr/
| -rw-r--r-- | Documentation/lei-q.pod | 4 | ||||
| -rw-r--r-- | lib/PublicInbox/LeiQuery.pm | 7 |
2 files changed, 7 insertions, 4 deletions
diff --git a/Documentation/lei-q.pod b/Documentation/lei-q.pod index 8134223e..d52c5b04 100644 --- a/Documentation/lei-q.pod +++ b/Documentation/lei-q.pod @@ -135,8 +135,8 @@ Set the number of query and write worker processes for parallelism. C<QUERY_WORKERS> defaults to the number of CPUs available, but 4 per remote (HTTP/HTTPS) host. -C<WRITE_WORKERS> defaults to the number of CPUs available for Maildir, -IMAP/IMAPS, and mbox* destinations. +C<WRITE_WORKERS> defaults to 75% of the number of CPUs available for +Maildir and mbox* destinations, but 4 per IMAP/IMAPS host. Omitting C<QUERY_WORKERS> but leaving the comma (C<,>) allows one to only set C<WRITE_WORKERS> diff --git a/lib/PublicInbox/LeiQuery.pm b/lib/PublicInbox/LeiQuery.pm index df9c32b3..0f839236 100644 --- a/lib/PublicInbox/LeiQuery.pm +++ b/lib/PublicInbox/LeiQuery.pm @@ -39,8 +39,11 @@ sub _start_query { # used by "lei q" and "lei up" $lms->lms_write_prepare->lms_pause; # just create } } - $l2m and $l2m->{-wq_nr_workers} //= $mj // - int($nproc * 0.75 + 0.5); # keep some CPU for git + $l2m and $l2m->{-wq_nr_workers} //= $mj // do { + # keep some CPU for git, and don't overload IMAP destinations + my $n = int($nproc * 0.75 + 0.5); + $self->{net} && $n > 4 ? 4 : $n; + }; # descending docid order is cheapest, MUA controls sorting order $self->{mset_opt}->{relevance} //= -2 if $l2m || $opt->{threads}; |