From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00
	shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 580551F4C0;
	Sat, 19 Oct 2019 00:11:44 +0000 (UTC)
Date: Sat, 19 Oct 2019 00:11:44 +0000
From: Eric Wong <e@80x24.org>
To: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Cc: meta@public-inbox.org
Subject: Re: how's memory usage on public-inbox-httpd?
Message-ID: <20191019001144.GA20824@dcvr>
References: <20181201194429.d5aldesjkb56il5c@dcvr>
 <20190606190455.GA17362@chatter.i7.local>
 <20191016221045.GA6828@dcvr>
 <20191018192352.GH25456@chatter.i7.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20191018192352.GH25456@chatter.i7.local>
List-Id: <meta.public-inbox.org>

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Wed, Oct 16, 2019 at 10:10:45PM +0000, Eric Wong wrote:
> > Btw, has this gotten better since the Perl 5.16.3 workarounds?
> > 
> > My 32-bit instance which sees the most HTTP traffic hasn't
> > exceeded 80M per-process in a while.
> 
> It's been definitely dramatically better. We keep adding lists to lore, so I
> haven't really been able to watch memory usage after a long period of daemon
> uptime, but it's never really gone very much above 1GB. In fact, we're
> downgrading lore to a smaller instance in the near future since we don't
> need to worry about running out of RAM any more.

Cool, but 1GB is still an order of magnitude worse that what
I'd expect :<   I remember Email::MIME had huge explosions with
some 30MB+ spam messages:
  https://public-inbox.org/meta/20190609083918.gfr2kurah7f2hysx@dcvr/
  (maybe gmime can help)

Depending on your storage speed/latency, more RAM can still help
significantly with Xapian.  The NVME stuff has amazing numbers,
but my mobos are too old and I'm still stuck on SATA 2.

Is nntpd better?  That only uses Email::Simple and not MIME;
so less explosions.