From: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
To: Eric Wong <e@80x24.org>
Cc: meta@public-inbox.org
Subject: Re: how's memory usage on public-inbox-httpd?
Date: Thu, 6 Jun 2019 17:45:09 -0400 [thread overview]
Message-ID: <20190606214509.GA4087@chatter.i7.local> (raw)
In-Reply-To: <20190606203752.7wpdla5ynemjlshs@dcvr>
On Thu, Jun 06, 2019 at 08:37:52PM +0000, Eric Wong wrote:
>Do you have commit 7d02b9e64455831d3bda20cd2e64e0c15dc07df5?
>("view: stop storing all MIME objects on large threads")
>That was most significant.
Yes. We're running 743ac758 with a few cherry-picked patches on top of
that (like epoch roll-over fix).
>Otherwise it's probably a combination of several things...
>httpd and nntpd both supports streaming, arbitrarily large
>endpoints (all.mbox.gz, and /T/, /t/, /t.mbox.gz threads with
>thousands of messages, giant NNTP BODY/ARTICLE ranges).
>
>All those endpoints should detect backpressure from a slow
>client (varnish/nginx in your case) using the ->getline method.
Wouldn't that spike up and down? The size I'm seeing stays pretty
constant without any significant changes across requests.
>Also, are you only using the default of -W/--worker-process=1
>on a 16-core machine? Just checked public-inbox-httpd(8), the
>-W switch is documented :) You can use SIGTTIN/TTOU to
>increase, decrease workers w/o restarting, too.
D'oh, yes... though it's not been a problem yet. :) I'm not sure I want
to bump that up, though, if that means we're going to have multiple
19GB-sized processes instead of one. :)
>Do you have any stats on the number of simultaneous connections
>public-inbox-httpd/nginx/varnish handles (and logging of that
>info at peek)? (perhaps running "ss -tan" periodically)(*)
We don't collect that info, but I'm not sure it's the number of
concurrent connections that's the culprit, as there is no fluctuation in
RSS size based on the number of responses.
To answer the questions in your follow-up:
It would appear to be all in anon memory. Mem_usage [1] reports:
# ./Mem_usage 18275
Backed by file:
Executable r-x 16668
Write/Exec (jump tables) rwx 0
RO data r-- 106908
Data rw- 232
Unreadable --- 94072
Unknown 0
Anonymous:
Writable code (stack) rwx 0
Data (malloc, mmap) rw- 19988892
RO data r-- 0
Unreadable --- 0
Unknown 12
I've been looking at lsof -p of that process and I see sqlite and xapian
showing up and disappearing. The lkml ones are being accessed almost all
the time, but even there I see them showing up with different FD
entries, so they are being closed and reopened properly.
Hope this helps.
-K
.. [1] https://elinux.org/images/d/d3/Mem_usage
next prev parent reply other threads:[~2019-06-06 21:45 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-01 19:44 how's memory usage on public-inbox-httpd? Eric Wong
2019-06-06 19:04 ` Konstantin Ryabitsev
2019-06-06 20:37 ` Eric Wong
2019-06-06 21:45 ` Konstantin Ryabitsev [this message]
2019-06-06 22:10 ` Eric Wong
2019-06-06 22:19 ` Konstantin Ryabitsev
2019-06-06 22:29 ` Eric Wong
2019-06-10 10:09 ` [RFC] optionally support glibc malloc_info via SIGCONT Eric Wong
2019-06-09 8:39 ` how's memory usage on public-inbox-httpd? Eric Wong
2019-06-12 17:08 ` Eric Wong
2019-06-06 20:54 ` Eric Wong
2019-10-16 22:10 ` Eric Wong
2019-10-18 19:23 ` Konstantin Ryabitsev
2019-10-19 0:11 ` Eric Wong
2019-10-22 17:28 ` Konstantin Ryabitsev
2019-10-22 19:11 ` Eric Wong
2019-10-28 23:24 ` Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190606214509.GA4087@chatter.i7.local \
--to=konstantin@linuxfoundation.org \
--cc=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).