about summary refs log tree commit homepage
path: root/lib/PublicInbox/LeiRemote.pm
DateCommit message (Collapse)
2021-10-15lei: TSTP affects all curl and related subprocesses
By relying more on pgroups for remaining remaining processes, this lets us pause all curl+tail subprocesses with a single kill(2) to avoid cluttering stderr. We won't bother pausing the pigz/gzip/bzip2/xz compressor process not cat-file processes, though, since those don't write to the terminal and they idle soon after the workers react to SIGSTOP. AutoReap is hoisted out from TestCommon.pm. CLONE_SKIP is gone since we won't be using Perl threads any time soon (they're discouraged by the maintainers of Perl).
2021-10-13lei: use standard warn() in more places
warn() is easier to augment with context information, and frankly unavoidable in the presence of 3rd-party libraries we don't control.
2021-09-19lei/store: use SOCK_SEQPACKET rather than pipe
This has several advantages: * no need to use ipc.lock to protect a pipe for non-atomic writes * ability to pass FDs. In another commit, this will let us simplify lei->sto_done_request and pass newly-created sockets to lei/store directly. disadvantages: - an extra pipe is required for rare messages over several hundred KB, this is probably a non-issue, though The performance delta is unknown, but I expect shards (which remain pipes) to be the primary bottleneck IPC-wise for lei/store.
2021-09-16lei: git_oid: replace git_blob_id
We'll be using binary SHA-1 and SHA-256 in-memory since that's what mail_sync.sqlite3 stores.
2021-08-14lei: hexdigest mocks account for unwanted headers
PublicInbox::Import never imports @UNWANTED_HEADERS, so ensure our mock blob OIDs do the same. This ought to prevent duplicates if the PSGI mboxrd download starts setting "X-Status: F" like "lei q -tt .."
2021-08-14lei <q|up>: wait on remote mboxrd imports synchronously
This ought to avoid /Document \d+ not found/ errors from Xapian when seeing a message for the first time by not attempting to read keywords for totally unseen messages.
2021-03-29lei: use IO::Uncompress::Gunzip MultiStream
This is compatible with default gunzip(1) behavior and future-proofs us against potential changes in PublicInbox::WWW to save memory on public-inbox-httpd instances.
2021-03-28lei blob: add remote external support
Introduce a new LeiRemote wrapper to provide an internal API which SolverGit expects. This lets us use HTTP/HTTPS endpoints to reconstruct blobs off patches as we would with local endpoints, just more slowly...