From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id ACDE01F463 for ; Wed, 25 Dec 2019 07:51:04 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 00/30] www: eliminate most per-request closures Date: Wed, 25 Dec 2019 07:50:34 +0000 Message-Id: <20191225075104.22184-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Closures (aka "anonymous subs") tack several KB of memory onto every WWW request/response, decreasing scalability and performance of our WWW endpoints. They also increase human review time to check for reference cycles. Similar changes to -nntpd and the generic parts of -httpd were also done recently: https://public-inbox.org/meta/20191221235319.27082-1-e@80x24.org/ https://public-inbox.org/meta/20191221080007.27810-1-e@80x24.org/ These could still use some naming improvements, and it's been pretty tiring writing the same-ish commit message over and over. All these changes around eliminating closures also make it easier to port our codebase to languages which lack closures. Fwiw, I've been brainstorming ideas to create a new, refcounted language where cyclic references are impossible by design. Such a design would not be possible if closures were implemented; but doable otherwise by taking a hint from *nix FSes. Eric Wong (30): git: allow async_cat to pass arg to callback httpd/async: support passing arg to callbacks qspawn: remove some anonymous subs for psgi_qx qspawn: disambiguate command vs PSGI env qspawn: replace anonymous $end callbacks w/ event_step msg_iter: provide means to stop using anonymous subs qspawn: reduce local vars, de-anonymize rd_hdr httpd/async: get rid of ephemeral main_cb qspawn: psgi_return: initial cb can be named qspawn: psgi_return_start: hoist out from psgi_return qspawn: psgi_qx: eliminate anonymous subs qspawn: drop "qspawn.filter" support, for now qspawn: psgi_return: allow non-anon parse_hdr callback githttpbackend: split out wwwstatic www: lazy load Plack::Util mboxgz: pass $ctx to callback to avoid anon subs feed: avoid anonymous subs config: each_inbox: pass user arg to callback view: avoid anon sub in stream_thread view: msg_html: stop using an anonymous sub contentid: no anonymous sub wwwtext: avoid anonymous sub in response searchview: pass named subs to Www*Stream view: thread_html: pass named sub to WwwStream searchview: remove anonymous sub when sorting threads by relevance view: msg_iter calls add_body_text directly wwwattach: avoid anonymous sub for msg_iter viewvcs: avoid anonymous sub for HTML response solvergit: allow passing arg to user-supplied callback search: retry_reopen passes user arg to callback MANIFEST | 1 + lib/PublicInbox/Cgit.pm | 19 +- lib/PublicInbox/Config.pm | 11 +- lib/PublicInbox/ContentId.pm | 53 +++--- lib/PublicInbox/ExtMsg.pm | 58 +++--- lib/PublicInbox/Feed.pm | 51 +++-- lib/PublicInbox/GetlineBody.pm | 12 +- lib/PublicInbox/Git.pm | 14 +- lib/PublicInbox/GitHTTPBackend.pm | 99 +--------- lib/PublicInbox/HTTPD/Async.pm | 56 +++--- lib/PublicInbox/Mbox.pm | 131 +++++++------ lib/PublicInbox/MboxGz.pm | 2 +- lib/PublicInbox/MsgIter.pm | 8 +- lib/PublicInbox/NewsWWW.pm | 16 +- lib/PublicInbox/Qspawn.pm | 296 +++++++++++++++--------------- lib/PublicInbox/Search.pm | 16 +- lib/PublicInbox/SearchMsg.pm | 9 +- lib/PublicInbox/SearchView.pm | 100 +++++----- lib/PublicInbox/SolverGit.pm | 149 ++++++++------- lib/PublicInbox/View.pm | 187 ++++++++++--------- lib/PublicInbox/ViewVCS.pm | 111 ++++++----- lib/PublicInbox/WWW.pm | 2 +- lib/PublicInbox/WwwAtomStream.pm | 2 +- lib/PublicInbox/WwwAttach.pm | 49 ++--- lib/PublicInbox/WwwListing.pm | 37 ++-- lib/PublicInbox/WwwStatic.pm | 105 +++++++++++ lib/PublicInbox/WwwText.pm | 20 +- t/git.t | 21 +++ t/qspawn.t | 19 +- 29 files changed, 882 insertions(+), 772 deletions(-) create mode 100644 lib/PublicInbox/WwwStatic.pm