From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 5CF0F1F44D for ; Mon, 15 Apr 2024 19:50:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1713210656; bh=zzx4u9MTNWV2BoTTHZiuOHRD7AKvjF8OwLg5Gl25zSI=; h=From:To:Subject:Date:From; b=XOx6so+/XW/774ku5d7JojOugl3jq8KVfL6vMQC4u1RqWrePy2UU+iTbsGsFYFr9Q lHIwNPfkDb/zHny6XwsJ7rwJIT3A4ci7HW+YlM9eS4muzeSR19AJCLrEnuroJVy/Xn VXL/2RLB/R45rnUDcVtYlT/WiAHma6BxgMSOYnMo= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH] doc: note MALLOC_MMAP_THRESHOLD_ as a potential workaround Date: Mon, 15 Apr 2024 19:50:56 +0000 Message-ID: <20240415195056.2017968-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Large string processing + concurrency + caching/memoization really brings out the worst in glibc malloc :< --- Documentation/public-inbox-tuning.pod | 5 ++++- examples/public-inbox-netd@.service | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Documentation/public-inbox-tuning.pod b/Documentation/public-inbox-tuning.pod index 73246144..7d0690b4 100644 --- a/Documentation/public-inbox-tuning.pod +++ b/Documentation/public-inbox-tuning.pod @@ -165,8 +165,11 @@ capacity planning. Bursts of small object allocations late in process life contribute to fragmentation of the heap due to arenas (slabs) used internally by Perl. -jemalloc (tested as an LD_PRELOAD on GNU/Linux) appears to reduce +jemalloc (tested as an LD_PRELOAD on GNU/Linux) reduces overall fragmentation compared to glibc malloc in long-lived processes. +glibc malloc users may try setting C to a lower +value (e.g. 131072) but that may require increasing the +C sysctl. =head2 Other OS tuning knobs diff --git a/examples/public-inbox-netd@.service b/examples/public-inbox-netd@.service index 83d2e995..7437f086 100644 --- a/examples/public-inbox-netd@.service +++ b/examples/public-inbox-netd@.service @@ -12,8 +12,11 @@ Wants = public-inbox-netd.socket After = public-inbox-netd.socket [Service] -# An LD_PRELOAD for libjemalloc can be added here. It currently seems +# An LD_PRELOAD for libjemalloc can be added here. It is # more resistant to fragmentation in long-lived daemons than glibc. +# If you're unable to use jemalloc, setting MALLOC_MMAP_THRESHOLD_ +# to a lower value (e.g. 131072) but that may also require increasing +# the sys.vm.max_map_count sysctl. Environment = PI_CONFIG=/home/pi/.public-inbox/config \ PATH=/usr/local/bin:/usr/bin:/bin \ TZ=UTC \