From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-4.1 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id AE6241F8C6 for ; Fri, 13 Aug 2021 21:05:52 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E681B3985C5E for ; Fri, 13 Aug 2021 21:05:51 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E681B3985C5E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1628888751; bh=fLi5CLlxkvnLcH7KfYOhUCZf4O5gpRbn9DcBfGpMQIQ=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=FPNyjg5yzDHbVhPcAL/UBOzF9ZFPU2zxdSVKo3MGp1TGcokdeeKM7gRtbF+qVArN7 F38YpOVVjjLhLbNuawtADuuynIxHxL4pyiCXjTpKkKPlREm+RnXNsXUEFu5Wu5T0SP PMUMwmTSvx9K1IV7/s4k9uTeewi48TfUz2IaHqck= Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by sourceware.org (Postfix) with ESMTPS id A6897386184A for ; Fri, 13 Aug 2021 21:04:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A6897386184A Received: by mail-pj1-x102b.google.com with SMTP id u13-20020a17090abb0db0290177e1d9b3f7so22524124pjr.1 for ; Fri, 13 Aug 2021 14:04:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fLi5CLlxkvnLcH7KfYOhUCZf4O5gpRbn9DcBfGpMQIQ=; b=Y8ttn0UVIuVyiIBfEZ0br48yVfAk9trTmOS5ZNs3MgD9cCESIQs06vo+sJ6KG1JvnG 0SiVqtMJpIG69dFD5xce8DpVOoY6kkae1LoFEZW5m5V/aU82IxEomZHtHlrQtsReFQ5H hu/4La6H9pM8bsld7xHhR3yk6HRsOFhPST+ggtdP3d3/+ARTRF73KQ+T3quVjVA5dlrz 2HYSjTDPBTpNzbCL8vvdVI6GS3M5VgsMXCV1osgR52MNzN2c7nFrxcz0GvN3FbYY+H5e r/q/ja8Hw3WCCJLmhVykKk+NSDgJS2aD73rF7R6JRkGz3Tv0hNq2GIvBuB6JiHWvM7/D F0Vw== X-Gm-Message-State: AOAM531rDf7lLT/EIO7mnvW7Cu8zK/dsht4g0OCsg8QFJKsszZ0B1fr3 qJXjMSrLSDp+Gh3ZvoZVLgg14Zy7Ee5KEA== X-Google-Smtp-Source: ABdhPJyKWmEFpSf3incQfYjNe9ol04wATnNePEKOClQojF6sZ6vkqJQjXI/3h0LPbbZ3agEeNiIOBQ== X-Received: by 2002:a17:902:c950:b029:12d:2ada:9ef7 with SMTP id i16-20020a170902c950b029012d2ada9ef7mr3525259pla.61.1628888677639; Fri, 13 Aug 2021 14:04:37 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:9dce:7cf8:4c28:b9e7:2767]) by smtp.gmail.com with ESMTPSA id v15sm3259462pff.105.2021.08.13.14.04.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 14:04:37 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH 2/3] malloc: Add THP/madvise support for sbrk Date: Fri, 13 Aug 2021 18:04:28 -0300 Message-Id: <20210813210429.1147112-3-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210813210429.1147112-1-adhemerval.zanella@linaro.org> References: <20210813210429.1147112-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Adhemerval Zanella via Libc-alpha Reply-To: Adhemerval Zanella Cc: Norbert Manthey , Siddhesh Poyarekar , Guillaume Morin Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Sender: "Libc-alpha" For the main arena, the sbrk() might the preferable syscall instead of mmap(). And the granularity used when increasing the program segment is the default page size. To increase effectiveness with Transparent Huge Page with madvise, the large page size is use instead. This is enabled with the new tunable 'glibc.malloc.thp_pagesize'. Checked on x86_64-linux-gnu. --- malloc/malloc.c | 40 +++++++++++++++++++++++++++++++++++----- 1 file changed, 35 insertions(+), 5 deletions(-) diff --git a/malloc/malloc.c b/malloc/malloc.c index 52ea84a63d..7cd586c866 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -2028,6 +2028,38 @@ sysmadvise_thp (void *p, INTERNAL_SIZE_T size) #endif } +static inline long int +thp_brk_align_up (long int size) +{ + INTERNAL_SIZE_T r = size; +#if HAVE_TUNABLES && defined (MADV_HUGEPAGE) + /* Defined in brk.c. */ + extern void *__curbrk; + if (mp_.thp_pagesize != 0) + { + uintptr_t top = ALIGN_UP ((uintptr_t)__curbrk + size, mp_.thp_pagesize); + r = top - (uintptr_t)__curbrk; + } + else +#endif + r = ALIGN_UP (size, GLRO(dl_pagesize)); + return r; +} + +static inline long +thp_brk_align_down (long int top) +{ + long r; +#if HAVE_TUNABLES && defined (MADV_HUGEPAGE) + if (mp_.thp_pagesize != 0) + r = ALIGN_DOWN (top, mp_.thp_pagesize); + else +#endif + r = ALIGN_DOWN (top, GLRO(dl_pagesize)); + return r; +} + + /* ------------------- Support for multiple arenas -------------------- */ #include "arena.c" @@ -2610,14 +2642,14 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av) size -= old_size; /* - Round to a multiple of page size. + Round to a multiple of page size or huge page size. If MORECORE is not contiguous, this ensures that we only call it with whole-page arguments. And if MORECORE is contiguous and this is not first time through, this preserves page-alignment of previous calls. Otherwise, we correct to page-align below. */ - size = ALIGN_UP (size, pagesize); + size = thp_brk_align_up (size); /* Don't try to call MORECORE if argument is so big as to appear @@ -2900,10 +2932,8 @@ systrim (size_t pad, mstate av) long released; /* Amount actually released */ char *current_brk; /* address returned by pre-check sbrk call */ char *new_brk; /* address returned by post-check sbrk call */ - size_t pagesize; long top_area; - pagesize = GLRO (dl_pagesize); top_size = chunksize (av->top); top_area = top_size - MINSIZE - 1; @@ -2911,7 +2941,7 @@ systrim (size_t pad, mstate av) return 0; /* Release in pagesize units and round down to the nearest page. */ - extra = ALIGN_DOWN(top_area - pad, pagesize); + extra = thp_brk_align_down (top_area - pad); if (extra == 0) return 0; -- 2.30.2