From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 2/2] httpd: disable Deflater middleware by default on Perl <5.18
Date: Thu, 26 Sep 2019 01:50:38 +0000 [thread overview]
Message-ID: <20190926015038.29770-3-e@80x24.org> (raw)
In-Reply-To: <20190926015038.29770-1-e@80x24.org>
Testing with perl-5.16.3-294.el7_6 RPM package on RHEL/CentOS 7,
the Deflater middleware triggers a leak when used in conjunction
with our push-based responses from PublicInbox::Qspawn.
I could not find another solution to workaround the memory leak
in this case, and I could not find a specific leak fix in
the perl5180delta manpage[1] which looked like it would
solve our problem.
Attempting to workaround the issue proved futile. Using
internal Deflater-specific keys to prevent deflating in
GitHTTPBackend and Qspawn did not solve the problem:
$env->{"plack.skip-deflater"} = 1;
$env->{"psgix.no-compress"} = 1;
Nor did forcing an invalid encoding via "git fetch":
git -c http.extraheader=Accept-Encoding:gzap fetch
So this appears to be a problem with Plack::Util::response_cb
somewhere.
This does NOT appear to be a problem with ref() leaking as in
DS::next_tick[2], since I couldn't find where
Plack::Middleware::Deflater or Plack::Util::response_cb would be
calling ref() on a blessed reference to trigger a leak.
Also, oddly enough, the ref() use for backwards compatibility at
the top of PublicInbox::GitHTTPBackend::serve does NOT seem to
trigger a leak on 5.16.3 due to [2]:
# XXX compatibility... ugh, can we stop supporting this?
$git = PublicInbox::Git->new($git) unless ref($git);
[1] https://perldoc.perl.org/perl5180delta.html
[2] https://rt.perl.org/Public/Bug/Display.html?id=114340
---
script/public-inbox-httpd | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/script/public-inbox-httpd b/script/public-inbox-httpd
index b2464f4..9b869f9 100755
--- a/script/public-inbox-httpd
+++ b/script/public-inbox-httpd
@@ -24,7 +24,13 @@ my $refresh = sub {
my $www = PublicInbox::WWW->new;
$www->preload;
$app = builder {
- eval {
+ # Perl 5.16.3 leaks in our "push" response code path
+ # (e.g. Qspawn) due to something in
+ # Plack::Util::response_cb, regardless of whether the
+ # client is sending Accept-Encoding:gzip requests.
+ # perl5180delta documents many leak fixes, so assume
+ # 5.18+ is safe for now and bump the check as-need:
+ $] >= 5.018000 and eval {
enable 'Deflater',
content_type => [ qw(
text/html
--
EW
next prev parent reply other threads:[~2019-09-26 1:50 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-26 1:50 [PATCH 0/2] leak workarounds for Perl 5.16 on CentOS/RHEL 7 Eric Wong
2019-09-26 1:50 ` [PATCH 1/2] ds: workaround a memory leak in Perl 5.16.x Eric Wong
2019-09-26 1:50 ` Eric Wong [this message]
2019-09-26 12:36 ` [PATCH 0/2] leak workarounds for Perl 5.16 on CentOS/RHEL 7 Konstantin Ryabitsev
2019-09-26 21:16 ` Eric Wong
2019-09-27 21:01 ` [PATCH 3/2] qspawn: workaround Perl 5.16.3 leak, re-enable Deflater Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190926015038.29770-3-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).