From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-4.0 required=3.0 tests=AWL,BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 0F7CB1F45E for ; Thu, 13 Feb 2020 18:08:20 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:cc:references:from:message-id:date :mime-version:in-reply-to:content-type :content-transfer-encoding; q=dns; s=default; b=rHScblaVPikY6Cj7 q+iT/HBjKNgCknCF6ovgri2kcIHoKvNiigGVAXmMvvnAv2y5yjus/nK1W3HJjGZ4 KPYkikIk8YJTSe4wXjIDNT3nx1CS9o7TmPG1klrkB1fBYoOfnH/ffmLef/mXf52Z GG82eAMvIWqRoQiySmjNC4p7Fo4= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:cc:references:from:message-id:date :mime-version:in-reply-to:content-type :content-transfer-encoding; s=default; bh=3xMTcgvrxI6Ajiwk3AiLKO rOGYE=; b=JkL7IgGzrFpc0t/ejxpCfvXtjHqT+piXHxOK6GF6Z7vaeCjwZ8JJE6 hrI3Y19quVTiEwx0DfFqvyLzr+kpQqglHbyM7jsA1j80rrXaOmAkypLPG7KMxxOt EIKIKfJNHZle+oKtMgCy/RCyF1vQ9b9WJptdLTbivlmHXawywnkOs= Received: (qmail 80385 invoked by alias); 13 Feb 2020 18:08:10 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 80307 invoked by uid 89); 13 Feb 2020 18:08:10 -0000 Authentication-Results: sourceware.org; auth=none X-HELO: us-smtp-delivery-1.mimecast.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581617286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2lUKHL/vEyLHpwlqMTvksmPrzlVNqyD3+FrZy7lSVQI=; b=GZeYVwSFW+9iZKitWPRRNmBGrBn9u1A0O19vjkBQOyAHpbjELOEwHUKrU2pXp6h9ATJkhM aJIU8MD2QN1/X3hy3wpwsO4W/f5bYdvcVOXYofzL8+AVQ+okT4WBaCPU5cxhtarkvMragd UpjtVHGMMgACl2Hpp/urftt7Cjri3UA= Subject: Re: [PATCH] Reserve static TLS for dynamically loaded initial-exec TLS only [BZ #25051] To: Szabolcs Nagy , Florian Weimer Cc: nd@arm.com, GNU C Library References: <44eaccc2-f760-88c0-989a-e413e328b051@arm.com> <87lfqjv56h.fsf@oldenburg2.str.redhat.com> <5533fb62-c336-45b3-71ec-30c5e7d7ac4c@arm.com> From: Carlos O'Donell Message-ID: <5474f68c-b093-8791-ca0b-b4b715174e2c@redhat.com> Date: Thu, 13 Feb 2020 13:07:56 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 2/13/20 11:38 AM, Szabolcs Nagy wrote: > On 07/01/2020 12:50, Szabolcs Nagy wrote: >> On 07/01/2020 12:09, Florian Weimer wrote: >>> * Szabolcs Nagy: >>> >>>> This patch reserves 128 bytes of the surplus TLS that is not used >>>> opportunistically. TLS_STATIC_SURPLUS is currently 1664, so this still >>>> allows 1536 bytes for opportunistic use. A new test is added to verify >>>> this ABI contract: dynamic loading of libraries with initial-exec TLS >>>> is supported up to 128 bytes in total on all targets. This should be >>>> enough for system libraries such as libgomp. >>> >>> I'm not sure if it's enough for loading another libc.so.6 via dlmopen. >>> Have you tested this, by chance? >> >> i haven't tested, but that wont work reliably with >> this patch. >> >> libc.so on aarch64 has 144 byte TLS (8byte alignmed), >> so the reserved 128byte surplus TLS is not enough >> (can be increased to 144 though). >> >> however if a lib with ie TLS is loaded before the static >> TLS runs out then that works: the 128byte reserve will be >> kept available until the 'opportunistic' part of TLS runs >> out and then the reserve can only be used for ie TLS. >> (so early dlmopen of libc.so.6 and later dlopen of libgomp >> works) > > i plan to work on this, but > > elf/dl-tls.c has > > /* Amount of excess space to allocate in the static TLS area > to allow dynamic loading of modules defining IE-model TLS data. */ > #define TLS_STATIC_SURPLUS 64 + DL_NNS * 100 > > with DL_NNS == 16 this gives 1664 byte surplus tls (+4 byte > extra in some cases because of alignment holes), but ideally > it should be > > 15 * 144 + // 2160 bytes for dlmopen of libc.so.6 > 16 * 128 + // 2048 bytes for dlopen/dlmopen of libgomp > 16 * 8 // 128 bytes for dlopen/dlmopen of libGL* ? > > = 4336 bytes. Yes. Each loaded library in dlmopen will get a unique map->l_tls_offset. Each library will take a block from the static tls. All of this is handled by _dl_try_allocate_static_tls in elf/dl-reloc.c. > (i think if rseq is committed libc tls might increase to > 192 bytes, because of the 32 byte alignment of __rseq_abi, > with that it's 5056 bytes.) Yes. > my understanding is that namespaces always need at least > surplus tls for libc (except for the first namespace) so > supporting DL_NNS == 16 would need 2160 bytes at least > (the current surplus only allows 12 namespaces at most). Yes. > i think either the supported namespaces should be reduced > (is that an option?) or the surplus tls size increased. > we also need a policy when non-ie tls libs or ie tls libs > other than libc/libgomp/libGL may use the surplus tls. Yes, we can reduce DL_NNS IMO, you could reduce it to 4 if you wanted, but I'll want a way to increase it too. See my suggestions. Let me take a stab at this: (1) File a bug to indicate all libraries must stop using TLS IE to fix dlmopen issues with TLS IE. (2) Set DL_NNS to 4, and add a tunable that lets you change the limit to reduce surplus memory allocated. (2.a) send linux man-pages a patch to make dlmopen(3) ambiguous about how many namespaces are supported and to point at the tunable to increase the memory used. (3) I think that the surplus should be split into two blocks: (3.a) Some reserved for TLS IE libs. (3.b) Some reserved for tlsdesc optimization. My opinion is that any library that is loaded and needs TLS IE should be able to use (3.a). We should reserve enough for (3.a) to be able to load glibc libraries, libgomp, and libgl (as you note). > example solution: > > #define DL_NNS 16 > #define DL_NNS_SUPPORTED 4 > #define LIBC_TLS 200 > #define IE_LIB_TLS 144 > #define NON_IE_LIB_TLS 1000 In the final patch these need way more comments to explain how you arrived at the magic numbers and the intent of your computation. > > #define TLS_STATIC_SURPLUS \ > (DL_NNS_SUPPORTED - 1) * LIBC_TLS /* 600 bytes. */ \ > + DL_NNS_SUPPORTED * IE_LIB_TLS /* 576 bytes. */ \ > + NON_IE_LIB_TLS > /* = 2176 bytes surplus tls. */ > > opportunistic tls use for non-ie tls libs is only allowed > up to 1000 bytes and 4 namespaces can be supported at all > times, beyond that dlmopen may not work (opportunistic tls > use is target dependent so the dlmopen limit will be so too). > > does this make sense? Yes, but I'm suggesting you just change DL_NNS to 4 and allow a tunable override. ... and as always MOAR COMMENTS. -- Cheers, Carlos.