From: Eric Wong <e@80x24.org> To: meta@public-inbox.org Subject: [PATCH 00/11] httpd: further reduce event loop monopolization Date: Wed, 9 Sep 2020 06:26:07 +0000 Message-ID: <20200909062618.5940-1-e@80x24.org> (raw) A couple more things to mitigate the effects of slow storage with many inboxes. Mostly solver-related, and still more to come... (Hoping the electrical grid stays up and dust bunny removal solved overheating problems). Eric Wong (11): xt/solver: test with public-inbox-httpd, too solver: drop warnings, modernize use v5.10.1, use SEEK_SET use "\&" where possible when referring to subroutines www: manifest.js.gz generation no longer hogs event loop config: flatten each_inbox and iterate_start args config: split out iterator into separate object t/cgi.t: show stderr on failures extmsg: prevent cross-inbox matches from hogging event loop wwwlisting: avoid hogging event loop solver: check one git coderepo and inbox at a time solver: break apart inbox blob retrieval MANIFEST | 2 + lib/PublicInbox/Cgit.pm | 5 +- lib/PublicInbox/Config.pm | 22 +-- lib/PublicInbox/ConfigIter.pm | 40 +++++ lib/PublicInbox/ExtMsg.pm | 102 ++++++++---- lib/PublicInbox/IMAPD.pm | 6 +- lib/PublicInbox/Inbox.pm | 2 +- lib/PublicInbox/ManifestJsGz.pm | 135 ++++++++++++++++ lib/PublicInbox/SolverGit.pm | 190 +++++++++++++--------- lib/PublicInbox/TestCommon.pm | 4 +- lib/PublicInbox/WWW.pm | 21 +-- lib/PublicInbox/Watch.pm | 13 +- lib/PublicInbox/WwwListing.pm | 279 ++++++++------------------------ t/cgi.t | 2 +- t/replace.t | 8 +- t/solver_git.t | 7 +- t/www_listing.t | 7 +- xt/msgtime_cmp.t | 2 +- xt/solver.t | 31 +++- 19 files changed, 499 insertions(+), 379 deletions(-) create mode 100644 lib/PublicInbox/ConfigIter.pm create mode 100644 lib/PublicInbox/ManifestJsGz.pm
next reply other threads:[~2020-09-09 6:26 UTC|newest] Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-09-09 6:26 Eric Wong [this message] 2020-09-09 6:26 ` [PATCH 01/11] xt/solver: test with public-inbox-httpd, too Eric Wong 2020-09-09 6:26 ` [PATCH 02/11] solver: drop warnings, modernize use v5.10.1, use SEEK_SET Eric Wong 2020-09-09 6:26 ` [PATCH 03/11] use "\&" where possible when referring to subroutines Eric Wong 2020-09-09 6:26 ` [PATCH 04/11] www: manifest.js.gz generation no longer hogs event loop Eric Wong 2020-09-09 6:26 ` [PATCH 05/11] config: flatten each_inbox and iterate_start args Eric Wong 2020-09-09 6:26 ` [PATCH 06/11] config: split out iterator into separate object Eric Wong 2020-09-09 6:26 ` [PATCH 07/11] t/cgi.t: show stderr on failures Eric Wong 2020-09-09 6:26 ` [PATCH 08/11] extmsg: prevent cross-inbox matches from hogging event loop Eric Wong 2020-09-09 6:26 ` [PATCH 09/11] wwwlisting: avoid " Eric Wong 2020-09-09 6:26 ` [PATCH 10/11] solver: check one git coderepo and inbox at a time Eric Wong 2020-09-09 6:26 ` [PATCH 11/11] solver: break apart inbox blob retrieval Eric Wong 2020-09-10 1:51 ` [PATCH 12/11] solver: async blob retrieval for diff extraction Eric Wong
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://public-inbox.org/README * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200909062618.5940-1-e@80x24.org \ --to=e@80x24.org \ --cc=meta@public-inbox.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
user/dev discussion of public-inbox itself This inbox may be cloned and mirrored by anyone: git clone --mirror http://public-inbox.org/meta git clone --mirror http://czquwvybam4bgbro.onion/meta git clone --mirror http://hjrcffqmbrq6wope.onion/meta git clone --mirror http://ou63pmih66umazou.onion/meta # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 meta meta/ http://public-inbox.org/meta \ meta@public-inbox.org public-inbox-index meta Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta nntp://news.gmane.io/gmane.mail.public-inbox.general note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/public-inbox.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git