Re: how's memory use? May 2020 edition

[Eric Wong <e@yhbt.net>]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Tue, May 12, 2020 at 08:37:34AM +0000, Eric Wong wrote:
> > Hey all, if possible; I'd like to know the memory use of your
> > daemons (particularly -httpd), relevant pmap(1) (or equivalent)
> > output, and version of public-inbox in use.
> 
> This is on lore.kernel.org. We upgraded to 1.5.0 yesterday, so this is 
> only after a day of running, but this usually covers a lot of traffic.
> We run with -W4, hence 4 different outputs:

Thanks for the info!  I forgot to ask in the original post, but
having an idea of active connections per worker might be useful
too.  I rarely see more than a few dozen, myself.

> # pgrep -f public-inbox-httpd | xargs pmap | grep anon

Would've been easier for humans to read the output if each
process were individually broken out, but I can figure it out
from addresses below :>

> 0000000002093000  23568K rw---   [ anon ]

OK, that looks like the heap of the master.

> 00007f981c6dc000     84K rw---   [ anon ]
> 00007f981d2c5000      4K rw---   [ anon ]
> 00007f982802f000     20K rw---   [ anon ]
> 00007f982824c000     16K rw---   [ anon ]
> 00007f982865c000    184K rw---   [ anon ]
> 00007f9828da8000      8K rw---   [ anon ]
> 00007f9828fc2000      8K rw---   [ anon ]
> 00007f9829351000      4K rw---   [ anon ]
> 00007f982953f000    160K rw---   [ anon ]
> 00007f9829572000      4K rw---   [ anon ]
> 00007f9829575000      4K rw---   [ anon ]
> 00007fffddbe2000      8K r-x--   [ anon ]
> ffffffffff600000      4K r-x--   [ anon ]

Probably somethings used by glibc internally, or maybe
SQLite, Xapian.  Good thing is the above mappings now
get shared with children and are copy-on-write

OK, onto another process:

> 0000000002093000  23568K rw---   [ anon ]

That looks inherited with the parent.

> 0000000003797000 235060K rw---   [ anon ]

Ah, so that's probably the main heap after forking (I forget
my 64-bit process uses -W0, so no workers in that setup).

Anyways, 250-300MB seems a lot better than things were for lore
few months ago (closer to ~1G per worker, IIRC?).

I've still got a some pure Perl ideas (and plenty with
Inline::C), though I'll probably prioritize other things, first,
such as IMAP.

> 00007f981a8fd000   2736K rw---   [ anon ]

Not sure where the above comes from, but it's an odd allocation
that seems to get pulled in by most other workers, independently.

> 00007f981c6dc000     84K rw---   [ anon ]
> 00007f981d2c5000      4K rw---   [ anon ]
> 00007f982802f000     20K rw---   [ anon ]
> 00007f982824c000     16K rw---   [ anon ]
> 00007f982865c000    184K rw---   [ anon ]
> 00007f9828da8000      8K rw---   [ anon ]
> 00007f9828fc2000      8K rw---   [ anon ]
> 00007f9829351000      4K rw---   [ anon ]
> 00007f982953f000    160K rw---   [ anon ]
> 00007f9829572000      4K rw---   [ anon ]
> 00007f9829575000      4K rw---   [ anon ]
> 00007fffddbe2000      8K r-x--   [ anon ]
> ffffffffff600000      4K r-x--   [ anon ]

That all looks shared from the parent, good.

> 0000000002093000  23568K rw---   [ anon ]
> 0000000003797000 216568K rw---   [ anon ]

OK, similar to the other worker.

> 00007f98196cc000   4724K rw---   [ anon ]
> 00007f9819b69000   4876K rw---   [ anon ]
> 00007f981a02c000   2736K rw---   [ anon ]
> 00007f981aeb5000   3496K rw---   [ anon ]
> 00007f981b350000   1628K rw---   [ anon ]

Weird, going to need to source dive into other dependencies to
figure this out, but also not a lot compared to the main 200MB+
heap, either.

I've got some odd ones like those, too, and they seem to
persist...

<snip 00007f981c6dc000 - ffffffffff600000>

> 0000000002093000  23568K rw---   [ anon ]
> 0000000003797000 241724K rw---   [ anon ]

> 00007f981a8e2000   2736K rw---   [ anon ]
> 00007f981b16f000   1964K rw---   [ anon ]
> 00007f981b35a000   1300K rw---   [ anon ]

Ditto for these mysterious allocations

<snip 00007f981c6dc000 - ffffffffff600000>

> 0000000002093000  23568K rw---   [ anon ]
> 0000000003797000 202632K rw---   [ anon ]

Probably the least busy process, and no odd >1MB mappings.

Anyways, things seem looking much better than they were in the
past.  Regexp matching and split() for MIME is still a problem,
and some lists like linux-mtd having some giant multi-MB spam
that gets crawled...

Thanks again for the info!