From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-6.0 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI,NICE_REPLY_A, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id DE2BF1F8C6 for ; Mon, 16 Aug 2021 20:55:37 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E77833848010 for ; Mon, 16 Aug 2021 20:55:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E77833848010 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1629147336; bh=LsiBCl1ID/4S8rmNe1NMo23JQMQ/nppwUttGA3lWDMU=; h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XK/UZgJlQtg7zLlo8v40Miw9YcVu0ieCFE+jILyyF8dC1E8mDclwmnISsrGL6v+wS 3ELcOv8RY8OYZnFhXrTVj8mmXcoXVSz6vNw4UfVK5juzQEcwe80wHLZKpnSG7qV2ft +ThwKc8rR8J/kMbMbOnpSJgJz3zbzaIHUGoB6UFI= Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) by sourceware.org (Postfix) with ESMTPS id BE4FF38515E8 for ; Mon, 16 Aug 2021 20:55:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org BE4FF38515E8 Received: by mail-qk1-x730.google.com with SMTP id t190so7254837qke.7 for ; Mon, 16 Aug 2021 13:55:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=LsiBCl1ID/4S8rmNe1NMo23JQMQ/nppwUttGA3lWDMU=; b=tG5Sboo0jBAI4YOSXbPmEUblG5qdfxyHqj9fJx1lJYkgg9GCJmo0pawoYiOcfPeDhV DKD/ELGobLuNt7wTwqnAhgEwt61GKTFCd5aUa6Zhy/6GDYlodVK5duVER7DjpKGCtBY9 kHGHejXsXJlDmEuZf2xVk8doNp5hGiVdQYJEuTSxw9KO+iCAxX1OWeT2OTY61n1mBD8P kA8XNP7SbgKLzoILQDesd8YPvGqIpeSc0B2+ngPGeE5SNd5C+cUp70dcNzP74fZiAvwQ WdrHK+lstTP2UJVIGi6lRAemwX1B59DLST+0vaqQxJgKyBe83+rLrxce/IupIEcxeJ86 cyHg== X-Gm-Message-State: AOAM531exB706gOaFIlZuuN2OowzOYM+jGdzADuj0e2Ux8Ast0yjHHs8 7Kpu2Ypy/armKl5+6ZEJmqicC23MbznwuQ== X-Google-Smtp-Source: ABdhPJwam5cWeT/zqYiYHyTB3rdV5aubGyHPG3Z+MsMAtmdnF5ycKMeC7DaRSFXcSIql1HjWt1SRLg== X-Received: by 2002:a05:620a:8c9:: with SMTP id z9mr247649qkz.225.1629147307225; Mon, 16 Aug 2021 13:55:07 -0700 (PDT) Received: from ?IPv6:2804:431:c7cb:9dce:aa23:db4f:6574:1da3? ([2804:431:c7cb:9dce:aa23:db4f:6574:1da3]) by smtp.gmail.com with ESMTPSA id y67sm14525qkd.58.2021.08.16.13.55.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 16 Aug 2021 13:55:06 -0700 (PDT) Subject: Re: [PATCH 0/3] malloc: improve THP effectiveness To: libc-alpha@sourceware.org, Norbert Manthey , Siddhesh Poyarekar , Guillaume Morin References: <20210813210429.1147112-1-adhemerval.zanella@linaro.org> <20210813213725.GA9384@bender.morinfr.org> Message-ID: <8a9c54a5-d0ad-440c-75e5-3390e9cc8a44@linaro.org> Date: Mon, 16 Aug 2021 17:55:04 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210813213725.GA9384@bender.morinfr.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Adhemerval Zanella via Libc-alpha Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Sender: "Libc-alpha" On 13/08/2021 18:37, Guillaume Morin wrote: > Hello Adhemerval, > > On 13 Aug 18:04, Adhemerval Zanella wrote: >> Although it does improve THP effectiveness, it does not provide the same >> features from libhugetlsfs morecore implementation [2], since it does >> use MAP_HUGETLB explicit on mmap. And I think this is not what we want >> for glibc, it requires additional setup from the admin to mount the >> hugetlsfs and reserve the pages with it outside from glibc scope. > > I certainly do appreciate the effort. But unfortunately this is not a > usable replacement for most libhugetlblfs users (who actually want to use > hugetlbfs). Yes, that's why I explicit stated this is not a replacement. But I had the misconception that MAP_HUGETLB would require to to use solely with mmap files opened on libhugetls filesystem and that's why I wrote that I think it is not meant to glibc. However reading the kernel documentation properly and after some experiment, I think we add another tunable to use MAP_HUGETLB as first allocation option. > > First, I'll argue to have THP supported directly in the allocator is > probably a nice-to-have feature for THP users but probably not that > critical considering you can just madvise() the memory *after* > it's been allocated. Alternatively any malloc interposition scheme can > do this trivially: afaik there were never an actual *need* for a > morecore implementation in this case. > There is no such possibility with hugetlbfs. It's either mmap() with > MAP_HUGETLB or not. Yeah, I am aware. The idea is mainly to abstract to requirement to query the kernel or handle the multiple pagesize from different architectures and also possible handle the sbrk() calls for main arena. We can also add more tuning in the future if we find some scenarios where THP need tuning. > > Second, THP is not a drop-in replacement for hugetblfs. hugetlbfs has > desirable properties that simply do not exist for THP. Just a few > examples: 1) A hugetlbfs allocation gives you a huge page or not at > allocation time but this is forever. There is no splitting, re-merging > by the VM: no TLB shootdowns for running processes etc. 2) Fast > allocation: there is a dedicated pool of these pages. There is no > competition with the rest of the processes unlike THP 3) No swapping all > hugetlbfs pages. > > I would really like to discuss and/or maybe implement some schemable > that allows to optionally use MAP_HUGETLB for all allocations (which > would be a definitive improvement over what libhugetlbfs was doing) if > that's workable for you. I am reworking this patchset and I intend to add an option to use MAP_HUGETLB as well.