From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Carlos O'Donell Newsgroups: gmane.comp.lib.glibc.alpha Subject: Re: [PATCH v2] Add malloc micro benchmark Date: Wed, 28 Feb 2018 10:21:20 -0800 Message-ID: References: <6ad98d83-d49b-25a3-ef01-e93e18f4740b@redhat.com> <96b76d58-d2f0-2176-51e5-f6338ed079e1@redhat.com> <165192eb-d815-867f-e1ba-0f9972eb19cd@redhat.com> <20180228141126.GA13073@domone> <808ed1e2-5945-0712-87a5-a0408de55fb0@redhat.com> <20180228164621.GA10445@domone> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1519841970 2415 195.159.176.226 (28 Feb 2018 18:19:30 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 28 Feb 2018 18:19:30 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 Cc: Joseph Myers , "libc-alpha@sourceware.org" , nd To: Wilco Dijkstra , =?UTF-8?B?T25kxZllaiBCw61sa2E=?= , Florian Weimer Original-X-From: libc-alpha-return-90686-glibc-alpha=m.gmane.org@sourceware.org Wed Feb 28 19:19:25 2018 Return-path: Envelope-to: glibc-alpha@blaine.gmane.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:cc:references:from:message-id:date :mime-version:in-reply-to:content-type :content-transfer-encoding; q=dns; s=default; b=H5LlRr9F643qGn7d G5z1h2DLLitJ066SYsk8d2Uhlk519wprg2EBBP3F1ZMzUmenFQb/W2PnqIY5XAem bmPJczwpradUQRxYip/mgkY4Fr0nRAg9f3fo1EimLf74qUF90N/cN2D/T2gIklA0 uFp/EtKltwBDGssUgsEO23qKieg= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:cc:references:from:message-id:date :mime-version:in-reply-to:content-type :content-transfer-encoding; s=default; bh=j2VRsjwdPEXZFxgy+Rd4TS aS7tE=; b=b/Xg9v6ALzLPA+Y6nu675SrulssZ8Cc+rMeusCR3B8NRqQ7lMvcwPm Z8ERBXNEG7bWu62BayNURxrDboKckwnQcJyl0KwAErMShaJ0z4DiHS+szCxabogx 8KYF7Ti9zXjM54oN0t32osX3jKf0wLp6HNwr2tBqdWpFURqasOW5Y= Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Original-Sender: libc-alpha-owner@sourceware.org Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 spammy=luxury, Hx-languages-length:2146, HX-Gm-Message-State:APf1xPDN X-HELO: mail-qk0-f172.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=NDibqkLVc40LgnuCBxuPAIa3J7fiQwh8OhLCG5UFQaU=; b=U5jxsVhGjspVXns7qx51pYK3xhCmUIAOhuW/3KYAfvvQg/yVIMW2I87hPUqiRMqjLD jOsDlNPJmEELhawzLhJg/tnE0y1ZQTRRx9yRMW7F+MCnv3RB31NwdMCVOC0TLmDakKO6 dvw0QA2rx368LibhR+b+nbs2pwLymKl40iWNvXGMX3J9kcXK09jg/0+fCu9DkebH3J6U ajZ7zYOQ7+WyThalBzhc6h7Tm7mPnUWS+m0Vbw9jwbB7ldrWbDQt+2dG4BWjwlQAFV5V 8YbWsaqKOgS2TOgycnbodQqPn/WG9TSRCjxDvbGdfzlqMGQvtTbcuInB1Bin4cfxdN2C gpxA== X-Gm-Message-State: APf1xPDN/Nowo96gIOPbiX91hTZqeRuAQrhlrxZbCO8RBR6ZfK6J9kb2 0Twf4+AsbuR5NXn/V5U+VJKjmOsgL/k= X-Google-Smtp-Source: AG47ELuu0la6UQlc0YsqB6sy5vn2beFZ/bvxTSEEkCPBCytrrJtQKzuUZCeGOmFxBM0izXA+qSy38g== X-Received: by 10.55.100.135 with SMTP id y129mr29559207qkb.209.1519842084313; Wed, 28 Feb 2018 10:21:24 -0800 (PST) In-Reply-To: Xref: news.gmane.org gmane.comp.lib.glibc.alpha:83017 Archived-At: Received: from server1.sourceware.org ([209.132.180.131] helo=sourceware.org) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er6K5-0008Tv-3O for glibc-alpha@blaine.gmane.org; Wed, 28 Feb 2018 19:19:25 +0100 Received: (qmail 73572 invoked by alias); 28 Feb 2018 18:21:28 -0000 Received: (qmail 73563 invoked by uid 89); 28 Feb 2018 18:21:27 -0000 On 02/28/2018 09:01 AM, Wilco Dijkstra wrote: > Ondřej Bílka wrote: >   >>> I think a heap-style allocator which does not segregate allocations >>> of different sizes still has its place, and why not provide one in >>> glibc? >>> >> That isn't case for any allocator and it is asking for trouble. You want >> to avoid sitation where two big chunks couldn't be merged because of >> tiny chunk between them. > > Agreed, you always want to special case small blocks. I don't believe there is > any advantage in using a single big heap. > >> For larger size this representation is still problematic and you could >> improve performance with another representation that also avoids >> alignment problem by placing metadata elsewhere(for mine only 4 bytes are needed). > > Larger sizes would be helped a lot once small blocks are dealt with separately. > So I don't think we need complicated balanced binary trees when dealing with a > small number of large blocks. You won't need an unsorted list either, large blocks > can be merged in O(1) time. > > There may be an advantage to place meta data elsewhere, for example it could make > adding/removing/walking free lists much faster (spatial locality) or to make heap > overflow attacks almost impossible, I agree with many of the things you and Ondrej are proposing. The outcome of our current discussions should be an incremental plan to go from where we are today, to where we want to be tomorrow. However, I do *not* believe it is a good plan to simply throw away the present allocator and claim it should be replaced from scratch. We do not have that luxury as a core project, we must remain answerable to our users. A high-level concrete problem today with glibc's malloc, and the only problem being reported by our users is that it consumes too much RSS. Solving that problem in the abstract is what we should be looking at. If we think that having multiple heaps for different sized objects is the way to do this, then we should think about how to go down that path with an experiment. Any cleanup we do before that is a win. -- Cheers, Carlos.