From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 63EFE1F406 for ; Wed, 25 Oct 2023 00:29:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1698193789; bh=7CqxGVr6gLJsgUSqIaLsBmwtbOCjzNVm2XsXBoLfSdI=; h=From:To:Subject:Date:From; b=HtD7vyiBvDi5Tpy6P4zAgXcbnA7+pHfRaqjanPNrulCQ2DRnBPz6XDG6+v1DJFtuG cg0D/udncj3fInzNhwm3ZgPfPHC5dmAKCRdb/uFgoBliLmHWv/uXSF9S0W1eKo5dEC 1s86+Qut28RJBOMKQI6KUQiukDyGSuUvJbdoQRNw= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 00/26] process management simplifications Date: Wed, 25 Oct 2023 00:29:23 +0000 Message-ID: <20231025002949.3092193-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: The convoluted HTTPD/Async code and is now gone and replaced with the simpler InputPipe (originally developed for lei). Fortunately, the s/psgi_return/psgi_yield/ change went more smoothly than I expected. cindex gets some simplifications, too; which will be helpful since more work is required on that code. I didn't get a chance to use the split out Limiter+Qspawn, though... 3/26 is a fairly large philosophy change to use temporary files over pipes, but it's probably fine as it's simpler and more straightforward data flow and only used for small outputs which can fit in memory, so unlikely to hit disk. 9/26 is the major change which was enabled by a change made 4 years ago. Eric Wong (26): limiter: split out from qspawn spawn: support synchronous run_qx psgi_qx: use a temporary file rather than pipe www_coderepo: capture uses a flattened list qspawn: psgi_return allows list for callback args qspawn: drop unused err arg for ->event_step httpd/async: require IO arg xt/check-run: call DS->Reset after all tests qspawn: introduce new psgi_yield API repo_atom: switch to psgi_yield repo_snapshot: psgi_yield viewvcs: psgi_yield www_altid: switch to psgi_yield cgit: switch to psgi_yield www_coderepo: psgi_yield drop psgi_return, httpd/async and GetlineBody qspawn: use WwwStatic for fallbacks and error code qspawn: simplify internal argument passing cidx_log_p: don't bother with F_SETPIPE_SZ cindex: avoid awaitpid for popen cindex: use timer for inits cindex: start using run_await to simplify code cindex: use run_await to read extensions.objectFormat cindex: drop XH_PID global cindex: use run_await wrapper for git commands cindex: use sysread for generating fingerprint MANIFEST | 5 +- lib/PublicInbox/Aspawn.pm | 34 +++ lib/PublicInbox/Cgit.pm | 2 +- lib/PublicInbox/CidxLogP.pm | 3 +- lib/PublicInbox/CodeSearchIdx.pm | 236 +++++++++---------- lib/PublicInbox/Config.pm | 4 +- lib/PublicInbox/GetlineBody.pm | 46 ---- lib/PublicInbox/GetlineResponse.pm | 40 ++++ lib/PublicInbox/Git.pm | 6 + lib/PublicInbox/GitHTTPBackend.pm | 17 +- lib/PublicInbox/GzipFilter.pm | 5 +- lib/PublicInbox/HTTP.pm | 8 +- lib/PublicInbox/HTTPD.pm | 5 +- lib/PublicInbox/HTTPD/Async.pm | 109 --------- lib/PublicInbox/Inbox.pm | 4 +- lib/PublicInbox/InputPipe.pm | 12 +- lib/PublicInbox/LEI.pm | 2 +- lib/PublicInbox/Limiter.pm | 47 ++++ lib/PublicInbox/MailDiff.pm | 8 +- lib/PublicInbox/Qspawn.pm | 349 +++++++++++------------------ lib/PublicInbox/RepoAtom.pm | 6 +- lib/PublicInbox/RepoSnapshot.pm | 2 +- lib/PublicInbox/SearchIdx.pm | 19 +- lib/PublicInbox/SolverGit.pm | 10 +- lib/PublicInbox/Spawn.pm | 69 ++++-- lib/PublicInbox/ViewVCS.pm | 7 +- lib/PublicInbox/WwwAltId.pm | 6 +- lib/PublicInbox/WwwCoderepo.pm | 12 +- t/dir_idle.t | 1 - t/fake_inotify.t | 2 - t/httpd-corner.psgi | 14 +- t/httpd-corner.t | 12 +- t/qspawn.t | 3 +- t/spawn.t | 13 +- xt/check-run.t | 2 + 35 files changed, 515 insertions(+), 605 deletions(-) create mode 100644 lib/PublicInbox/Aspawn.pm delete mode 100644 lib/PublicInbox/GetlineBody.pm create mode 100644 lib/PublicInbox/GetlineResponse.pm delete mode 100644 lib/PublicInbox/HTTPD/Async.pm create mode 100644 lib/PublicInbox/Limiter.pm