user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 00/30] www: eliminate most per-request closures
Date: Wed, 25 Dec 2019 07:50:34 +0000	[thread overview]
Message-ID: <20191225075104.22184-1-e@80x24.org> (raw)

Closures (aka "anonymous subs") tack several KB of memory onto
every WWW request/response, decreasing scalability and
performance of our WWW endpoints.  They also increase human
review time to check for reference cycles.

Similar changes to -nntpd and the generic parts of -httpd were
also done recently:
https://public-inbox.org/meta/20191221235319.27082-1-e@80x24.org/
https://public-inbox.org/meta/20191221080007.27810-1-e@80x24.org/

These could still use some naming improvements, and it's been
pretty tiring writing the same-ish commit message over and over.

All these changes around eliminating closures also make it
easier to port our codebase to languages which lack closures.

Fwiw, I've been brainstorming ideas to create a new, refcounted
language where cyclic references are impossible by design.  Such
a design would not be possible if closures were implemented; but
doable otherwise by taking a hint from *nix FSes.

Eric Wong (30):
  git: allow async_cat to pass arg to callback
  httpd/async: support passing arg to callbacks
  qspawn: remove some anonymous subs for psgi_qx
  qspawn: disambiguate command vs PSGI env
  qspawn: replace anonymous $end callbacks w/ event_step
  msg_iter: provide means to stop using anonymous subs
  qspawn: reduce local vars, de-anonymize rd_hdr
  httpd/async: get rid of ephemeral main_cb
  qspawn: psgi_return: initial cb can be named
  qspawn: psgi_return_start: hoist out from psgi_return
  qspawn: psgi_qx: eliminate anonymous subs
  qspawn: drop "qspawn.filter" support, for now
  qspawn: psgi_return: allow non-anon parse_hdr callback
  githttpbackend: split out wwwstatic
  www: lazy load Plack::Util
  mboxgz: pass $ctx to callback to avoid anon subs
  feed: avoid anonymous subs
  config: each_inbox: pass user arg to callback
  view: avoid anon sub in stream_thread
  view: msg_html: stop using an anonymous sub
  contentid: no anonymous sub
  wwwtext: avoid anonymous sub in response
  searchview: pass named subs to Www*Stream
  view: thread_html: pass named sub to WwwStream
  searchview: remove anonymous sub when sorting threads by relevance
  view: msg_iter calls add_body_text directly
  wwwattach: avoid anonymous sub for msg_iter
  viewvcs: avoid anonymous sub for HTML response
  solvergit: allow passing arg to user-supplied callback
  search: retry_reopen passes user arg to callback

 MANIFEST                          |   1 +
 lib/PublicInbox/Cgit.pm           |  19 +-
 lib/PublicInbox/Config.pm         |  11 +-
 lib/PublicInbox/ContentId.pm      |  53 +++---
 lib/PublicInbox/ExtMsg.pm         |  58 +++---
 lib/PublicInbox/Feed.pm           |  51 +++--
 lib/PublicInbox/GetlineBody.pm    |  12 +-
 lib/PublicInbox/Git.pm            |  14 +-
 lib/PublicInbox/GitHTTPBackend.pm |  99 +---------
 lib/PublicInbox/HTTPD/Async.pm    |  56 +++---
 lib/PublicInbox/Mbox.pm           | 131 +++++++------
 lib/PublicInbox/MboxGz.pm         |   2 +-
 lib/PublicInbox/MsgIter.pm        |   8 +-
 lib/PublicInbox/NewsWWW.pm        |  16 +-
 lib/PublicInbox/Qspawn.pm         | 296 +++++++++++++++---------------
 lib/PublicInbox/Search.pm         |  16 +-
 lib/PublicInbox/SearchMsg.pm      |   9 +-
 lib/PublicInbox/SearchView.pm     | 100 +++++-----
 lib/PublicInbox/SolverGit.pm      | 149 ++++++++-------
 lib/PublicInbox/View.pm           | 187 ++++++++++---------
 lib/PublicInbox/ViewVCS.pm        | 111 ++++++-----
 lib/PublicInbox/WWW.pm            |   2 +-
 lib/PublicInbox/WwwAtomStream.pm  |   2 +-
 lib/PublicInbox/WwwAttach.pm      |  49 ++---
 lib/PublicInbox/WwwListing.pm     |  37 ++--
 lib/PublicInbox/WwwStatic.pm      | 105 +++++++++++
 lib/PublicInbox/WwwText.pm        |  20 +-
 t/git.t                           |  21 +++
 t/qspawn.t                        |  19 +-
 29 files changed, 882 insertions(+), 772 deletions(-)
 create mode 100644 lib/PublicInbox/WwwStatic.pm


             reply	other threads:[~2019-12-25  7:51 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-25  7:50 Eric Wong [this message]
2019-12-25  7:50 ` [PATCH 01/30] git: allow async_cat to pass arg to callback Eric Wong
2019-12-25  7:50 ` [PATCH 02/30] httpd/async: support passing arg to callbacks Eric Wong
2019-12-26  7:53   ` Eric Wong
2019-12-25  7:50 ` [PATCH 03/30] qspawn: remove some anonymous subs for psgi_qx Eric Wong
2019-12-25  7:50 ` [PATCH 04/30] qspawn: disambiguate command vs PSGI env Eric Wong
2019-12-25  7:50 ` [PATCH 05/30] qspawn: replace anonymous $end callbacks w/ event_step Eric Wong
2019-12-25  7:50 ` [PATCH 06/30] msg_iter: provide means to stop using anonymous subs Eric Wong
2019-12-25  7:50 ` [PATCH 07/30] qspawn: reduce local vars, de-anonymize rd_hdr Eric Wong
2019-12-25  7:50 ` [PATCH 08/30] httpd/async: get rid of ephemeral main_cb Eric Wong
2019-12-25  7:50 ` [PATCH 09/30] qspawn: psgi_return: initial cb can be named Eric Wong
2019-12-25  7:50 ` [PATCH 10/30] qspawn: psgi_return_start: hoist out from psgi_return Eric Wong
2019-12-25  7:50 ` [PATCH 11/30] qspawn: psgi_qx: eliminate anonymous subs Eric Wong
2019-12-25  7:50 ` [PATCH 12/30] qspawn: drop "qspawn.filter" support, for now Eric Wong
2019-12-25  7:50 ` [PATCH 13/30] qspawn: psgi_return: allow non-anon parse_hdr callback Eric Wong
2019-12-25  7:50 ` [PATCH 14/30] githttpbackend: split out wwwstatic Eric Wong
2019-12-26 12:50   ` Eric Wong
2019-12-27 10:36     ` Eric Wong
2019-12-25  7:50 ` [PATCH 15/30] www: lazy load Plack::Util Eric Wong
2019-12-25  7:50 ` [PATCH 16/30] mboxgz: pass $ctx to callback to avoid anon subs Eric Wong
2019-12-25  7:50 ` [PATCH 17/30] feed: avoid anonymous subs Eric Wong
2019-12-25  7:50 ` [PATCH 18/30] config: each_inbox: pass user arg to callback Eric Wong
2019-12-26  6:48   ` Eric Wong
2019-12-25  7:50 ` [PATCH 19/30] view: avoid anon sub in stream_thread Eric Wong
2019-12-25  7:50 ` [PATCH 20/30] view: msg_html: stop using an anonymous sub Eric Wong
2019-12-25  7:50 ` [PATCH 21/30] contentid: no " Eric Wong
2019-12-25  7:50 ` [PATCH 22/30] wwwtext: avoid anonymous sub in response Eric Wong
2019-12-25  7:50 ` [PATCH 23/30] searchview: pass named subs to Www*Stream Eric Wong
2019-12-25  7:50 ` [PATCH 24/30] view: thread_html: pass named sub to WwwStream Eric Wong
2019-12-25  7:50 ` [PATCH 25/30] searchview: remove anonymous sub when sorting threads by relevance Eric Wong
2019-12-25  7:51 ` [PATCH 26/30] view: msg_iter calls add_body_text directly Eric Wong
2019-12-25  7:51 ` [PATCH 27/30] wwwattach: avoid anonymous sub for msg_iter Eric Wong
2019-12-25  7:51 ` [PATCH 28/30] viewvcs: avoid anonymous sub for HTML response Eric Wong
2019-12-25  7:51 ` [PATCH 29/30] solvergit: allow passing arg to user-supplied callback Eric Wong
2019-12-28  9:17   ` Eric Wong
2019-12-25  7:51 ` [PATCH 30/30] search: retry_reopen passes user arg to callback Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191225075104.22184-1-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).