* [PATCH 2/4] httpd/async: remove useless undef
@ 2022-12-23 11:05 7% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2022-12-23 11:05 UTC (permalink / raw)
To: meta
Assigning `undef' to a scalar doesn't free it's memory,
we need to call `undef($var)' in the caller. It's also
been pointless since we simplified ->async_pass in commit
b7fbffd1f8c12556 (httpd/async: get rid of ephemeral main_cb, 2019-12-25)
---
lib/PublicInbox/HTTPD/Async.pm | 4 ----
1 file changed, 4 deletions(-)
diff --git a/lib/PublicInbox/HTTPD/Async.pm b/lib/PublicInbox/HTTPD/Async.pm
index 9e592f47..75b3bd50 100644
--- a/lib/PublicInbox/HTTPD/Async.pm
+++ b/lib/PublicInbox/HTTPD/Async.pm
@@ -84,10 +84,6 @@ sub async_pass {
# *_wcb methods respond to ->write (and ->close), not ->print
$fh->write($$bref);
- # we're done with this, free this memory up ASAP since the
- # calls after this may use much memory:
- $$bref = undef;
-
$self->{http} = $http;
$self->{fh} = $fh;
}
^ permalink raw reply related [relevance 7%]
* [PATCH 08/30] httpd/async: get rid of ephemeral main_cb
2019-12-25 7:50 5% [PATCH 00/30] www: eliminate most per-request closures Eric Wong
@ 2019-12-25 7:50 6% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2019-12-25 7:50 UTC (permalink / raw)
To: meta
Cheaper to use up two hash table slots than creating a new sub.
---
lib/PublicInbox/HTTPD/Async.pm | 42 ++++++++++++++++------------------
1 file changed, 20 insertions(+), 22 deletions(-)
diff --git a/lib/PublicInbox/HTTPD/Async.pm b/lib/PublicInbox/HTTPD/Async.pm
index ac0ca3df..9494bc10 100644
--- a/lib/PublicInbox/HTTPD/Async.pm
+++ b/lib/PublicInbox/HTTPD/Async.pm
@@ -10,7 +10,7 @@ package PublicInbox::HTTPD::Async;
use strict;
use warnings;
use base qw(PublicInbox::DS);
-use fields qw(cb arg end_obj);
+use fields qw(http fh cb arg end_obj);
use Errno qw(EAGAIN);
use PublicInbox::Syscall qw(EPOLLIN EPOLLET);
@@ -31,21 +31,24 @@ sub new {
my $self = fields::new($class);
IO::Handle::blocking($io, 0);
$self->SUPER::new($io, EPOLLIN | EPOLLET);
- $self->{cb} = $cb; # initial read callback, later replaced by main_cb
+ $self->{cb} = $cb; # initial read callback
$self->{arg} = $arg; # arg for $cb
$self->{end_obj} = $end_obj; # like END{}, can ->event_step
$self;
}
-sub main_cb ($$) {
- my ($http, $fh) = @_;
- sub {
- my ($self) = @_;
+sub event_step {
+ my ($self) = @_;
+ if (my $cb = delete $self->{cb}) {
+ # this may call async_pass when headers are done
+ $cb->(delete $self->{arg});
+ } elsif (my $sock = $self->{sock}) {
+ my $http = $self->{http};
# $self->{sock} is a read pipe for git-http-backend or cgit
# and 65536 is the default Linux pipe size
- my $r = sysread($self->{sock}, my $buf, 65536);
+ my $r = sysread($sock, my $buf, 65536);
if ($r) {
- $fh->write($buf); # may call $http->close
+ $self->{fh}->write($buf); # may call $http->close
if ($http->{sock}) { # !closed
$self->requeue;
# let other clients get some work done, too
@@ -57,11 +60,11 @@ sub main_cb ($$) {
return; # EPOLLET means we'll be notified
}
- # Done! Error handling will happen in $fh->close
- # called by the {end} handler
+ # Done! Error handling will happen in $self->{fh}->close
+ # called by end_obj->event_step handler
delete $http->{forward};
- $self->close; # queues ->{end} to be called
- }
+ $self->close; # queues end_obj->event_step to be called
+ } # else { # we may've been requeued but closed by $http
}
# once this is called, all data we read is passed to the
@@ -79,21 +82,16 @@ sub async_pass {
# calls after this may use much memory:
$$bref = undef;
- # replace the header read callback with the main one
- my $cb = $self->{cb} = main_cb($http, $fh);
- $cb->($self); # either hit EAGAIN or ->requeue to keep EPOLLET happy
-}
+ $self->{http} = $http;
+ $self->{fh} = $fh;
-sub event_step {
- # {cb} may be undef after ->requeue due to $http->close happening
- my $cb = $_[0]->{cb} or return;
- $cb->(@_);
+ # either hit EAGAIN or ->requeue to keep EPOLLET happy
+ event_step($self);
}
-# may be called as $forward->close in PublicInbox::HTTP or EOF (main_cb)
+# may be called as $forward->close in PublicInbox::HTTP or EOF (event_step)
sub close {
my $self = $_[0];
- delete $self->{cb};
$self->SUPER::close; # DS::close
# we defer this to the next timer loop since close is deferred
^ permalink raw reply related [relevance 6%]
* [PATCH 00/30] www: eliminate most per-request closures
@ 2019-12-25 7:50 5% Eric Wong
2019-12-25 7:50 6% ` [PATCH 08/30] httpd/async: get rid of ephemeral main_cb Eric Wong
0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2019-12-25 7:50 UTC (permalink / raw)
To: meta
Closures (aka "anonymous subs") tack several KB of memory onto
every WWW request/response, decreasing scalability and
performance of our WWW endpoints. They also increase human
review time to check for reference cycles.
Similar changes to -nntpd and the generic parts of -httpd were
also done recently:
https://public-inbox.org/meta/20191221235319.27082-1-e@80x24.org/
https://public-inbox.org/meta/20191221080007.27810-1-e@80x24.org/
These could still use some naming improvements, and it's been
pretty tiring writing the same-ish commit message over and over.
All these changes around eliminating closures also make it
easier to port our codebase to languages which lack closures.
Fwiw, I've been brainstorming ideas to create a new, refcounted
language where cyclic references are impossible by design. Such
a design would not be possible if closures were implemented; but
doable otherwise by taking a hint from *nix FSes.
Eric Wong (30):
git: allow async_cat to pass arg to callback
httpd/async: support passing arg to callbacks
qspawn: remove some anonymous subs for psgi_qx
qspawn: disambiguate command vs PSGI env
qspawn: replace anonymous $end callbacks w/ event_step
msg_iter: provide means to stop using anonymous subs
qspawn: reduce local vars, de-anonymize rd_hdr
httpd/async: get rid of ephemeral main_cb
qspawn: psgi_return: initial cb can be named
qspawn: psgi_return_start: hoist out from psgi_return
qspawn: psgi_qx: eliminate anonymous subs
qspawn: drop "qspawn.filter" support, for now
qspawn: psgi_return: allow non-anon parse_hdr callback
githttpbackend: split out wwwstatic
www: lazy load Plack::Util
mboxgz: pass $ctx to callback to avoid anon subs
feed: avoid anonymous subs
config: each_inbox: pass user arg to callback
view: avoid anon sub in stream_thread
view: msg_html: stop using an anonymous sub
contentid: no anonymous sub
wwwtext: avoid anonymous sub in response
searchview: pass named subs to Www*Stream
view: thread_html: pass named sub to WwwStream
searchview: remove anonymous sub when sorting threads by relevance
view: msg_iter calls add_body_text directly
wwwattach: avoid anonymous sub for msg_iter
viewvcs: avoid anonymous sub for HTML response
solvergit: allow passing arg to user-supplied callback
search: retry_reopen passes user arg to callback
MANIFEST | 1 +
lib/PublicInbox/Cgit.pm | 19 +-
lib/PublicInbox/Config.pm | 11 +-
lib/PublicInbox/ContentId.pm | 53 +++---
lib/PublicInbox/ExtMsg.pm | 58 +++---
lib/PublicInbox/Feed.pm | 51 +++--
lib/PublicInbox/GetlineBody.pm | 12 +-
lib/PublicInbox/Git.pm | 14 +-
lib/PublicInbox/GitHTTPBackend.pm | 99 +---------
lib/PublicInbox/HTTPD/Async.pm | 56 +++---
lib/PublicInbox/Mbox.pm | 131 +++++++------
lib/PublicInbox/MboxGz.pm | 2 +-
lib/PublicInbox/MsgIter.pm | 8 +-
lib/PublicInbox/NewsWWW.pm | 16 +-
lib/PublicInbox/Qspawn.pm | 296 +++++++++++++++---------------
lib/PublicInbox/Search.pm | 16 +-
lib/PublicInbox/SearchMsg.pm | 9 +-
lib/PublicInbox/SearchView.pm | 100 +++++-----
lib/PublicInbox/SolverGit.pm | 149 ++++++++-------
lib/PublicInbox/View.pm | 187 ++++++++++---------
lib/PublicInbox/ViewVCS.pm | 111 ++++++-----
lib/PublicInbox/WWW.pm | 2 +-
lib/PublicInbox/WwwAtomStream.pm | 2 +-
lib/PublicInbox/WwwAttach.pm | 49 ++---
lib/PublicInbox/WwwListing.pm | 37 ++--
lib/PublicInbox/WwwStatic.pm | 105 +++++++++++
lib/PublicInbox/WwwText.pm | 20 +-
t/git.t | 21 +++
t/qspawn.t | 19 +-
29 files changed, 882 insertions(+), 772 deletions(-)
create mode 100644 lib/PublicInbox/WwwStatic.pm
^ permalink raw reply [relevance 5%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-12-25 7:50 5% [PATCH 00/30] www: eliminate most per-request closures Eric Wong
2019-12-25 7:50 6% ` [PATCH 08/30] httpd/async: get rid of ephemeral main_cb Eric Wong
2022-12-23 11:05 [PATCH 0/4] httpd/async fixes for coderepos Eric Wong
2022-12-23 11:05 7% ` [PATCH 2/4] httpd/async: remove useless undef Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).