From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id C53C01F45A; Tue, 21 Apr 2020 08:52:08 +0000 (UTC) Date: Tue, 21 Apr 2020 08:52:08 +0000 From: Eric Wong To: meta@public-inbox.org Subject: Encode preloading Message-ID: <20200421085208.GA28087@dcvr> References: <20200319083256.15593-1-e@yhbt.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20200319083256.15593-1-e@yhbt.net> List-Id: [was: Subject: [PATCH 0/6] daemon: reduce fragmentation via preload Eric Wong wrote: > For long-lived daemons, perform immortal allocations as early as > possible to reduce the likelyhood of heap fragmentation due to > mixed-lifetime allocations happening once the process is fully > loaded and serving requests, since per-request allocations > should all be short-lived. Encode also loads lazily... I thought about having public-inbox-{index,mda,watch} track a list of modules for WWW to preload, but that can't predict the future. There's no telling what emails will show up in the future which trigger autoloading, so I think the only way to avoid fragmentation is to preload everything which would get autoloaded... Ugh, that even means Encode::EBCDIC and Encode::Symbol (for dingbats!) > On a side note, I'm wondering if WWW should just preload by > default. I'm not sure if anybody uses public-inbox.cgi (or > should be using it :P). It's not like we don't ship > public-inbox-httpd; and any PSGI implementation could be used > for smaller inboxes (or powerful-enough hardware). Yeah, likely to happen. Trying to optimize a CGI script for startup time while depending on git, SQLite, and Xapian just doesn't seem worth it.