| Date | Commit message (Collapse) |
|---|
|
Sigfd->event_step needs priority over WQWorkers (and everything
else). Do that by running once per event_loop iteration rather
than looping inside event_step. This lowers throughput since it
requires more syscalls, but that's the price of fairness.
|
|
Since some lei worker classes only use a single worker,
there's no sense in having broadcast for those cases.
|
|
WQWorkers are limited roughly to MAX_ARG_STRLEN (the kernel
limit of argv + environ) to avoid excessive memory growth.
Occasionally, we need to send larger messages via workqueues
that are too small to hit EMSGSIZE on the sender.
This fixes "lei q" when using HTTP(S) externals, since that
code path sends large Eml objects from lei_xsearch workers
directly to lei2mail WQ workers.
|
|
We'll give workqueues a broadcast mechanism to ensure all
workers see a certain message. We'll also tag each worker
with {-wq_worker_nr} in preparation for work distribution.
This is intended to avoid extra connection and fork() costs
from LeiAuth in a future commit.
|
|
This will let us to maximize the capability of our asynchronous
git API. This lets us avoid relying on EOF to notify lei2mail
workers; thus giving us the option of running fewer lei_xsearch
worker processes in parallel than local sources.
I tried using a synchronous git API; and even with libgit2 in
the same process to avoid the IPC cost failed to match the
throughput afforded by this change. This is because libgit2 is
built (at least on Debian) with the SHA-1 collision code enabled
and ubc_check stuff was dominating my profiles.
|