From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 8E2CB1F990 for ; Wed, 5 Aug 2020 19:23:06 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5712A385C8B1; Wed, 5 Aug 2020 19:23:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5712A385C8B1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1596655385; bh=LnZLzxjZiUaPtYUVfCUyvjf9qO8/u/HnGCN7Tzhis68=; h=To:References:Subject:Date:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=wrUZzF9VkBXWvJkBgZa+eIeHsUTOxqz81UOi2hJaNBG33enPPGQVt3/zr9fxjyh9R wmWlyVHpyDZH7PaS3JXwuyN6+RxH5CQ07AvEQuV3uqqRqKkEazkGkj5ZMcStuYLphS Bs+mnlFhG/dXypu7O9QAh68ng6F2nWAnVHafHdS4= Received: from mail-io1-xd44.google.com (mail-io1-xd44.google.com [IPv6:2607:f8b0:4864:20::d44]) by sourceware.org (Postfix) with ESMTPS id A1D0E3857C44 for ; Wed, 5 Aug 2020 19:23:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A1D0E3857C44 Received: by mail-io1-xd44.google.com with SMTP id v6so31838509iow.11 for ; Wed, 05 Aug 2020 12:23:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:references:from:autocrypt:subject:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=LnZLzxjZiUaPtYUVfCUyvjf9qO8/u/HnGCN7Tzhis68=; b=k0cCR9L5zfSpiLoTBi8unnOgiR2rIG3NDYWmRgCwblJ2E1/MO93nZnG8+SfbU9moDn wzs9cIipy2D8akYpaOfC+tgBT+5NAQvkmzNClcQzXZxeFOTqQiuWkiDKi0oCfKPUlKuT pmDHRIIrHTIU8vMlgV1CP57dJdzqoJm2MPR/befmDSvwXF6cOelpEDVQonhiFmNIrvSc 5JNRMlgJ5jE3cn1rjiC1zmIPndirAbeMgdkl5s88IW3fFzVMNDB5Tww2c1XkA8bVgeyu VpJm9UnvUcq2BnjrD4Pmeofp5aY1Mjl5JzWEMiACPzRRC9m/egNtvvBQTWX57Zqr4+Lv Wcsg== X-Gm-Message-State: AOAM5339yXN7x/Zq8SHzIQLP5mP4FSaN9WMQd50jNMm7xOxF/hUmseQJ Is/depqutB2EtXcelO1Bh3XN0IPED//cfA== X-Google-Smtp-Source: ABdhPJx4u/DtdefvxGPs0BCGCx/imXmjv8PuR83vKreFTjwIUTEAdY9oUlBM9pmMkbpkSy2DYe3enw== X-Received: by 2002:ac8:70c3:: with SMTP id g3mr3376821qtp.164.1596655374700; Wed, 05 Aug 2020 12:22:54 -0700 (PDT) Received: from [192.168.1.4] ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id k2sm2484194qkf.127.2020.08.05.12.22.53 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 05 Aug 2020 12:22:53 -0700 (PDT) To: libc-alpha@sourceware.org References: <87imgx2tur.fsf@mid.deneb.enyo.de> Autocrypt: addr=adhemerval.zanella@linaro.org; prefer-encrypt=mutual; keydata= mQINBFcVGkoBEADiQU2x/cBBmAVf5C2d1xgz6zCnlCefbqaflUBw4hB/bEME40QsrVzWZ5Nq 8kxkEczZzAOKkkvv4pRVLlLn/zDtFXhlcvQRJ3yFMGqzBjofucOrmdYkOGo0uCaoJKPT186L NWp53SACXguFJpnw4ODI64ziInzXQs/rUJqrFoVIlrPDmNv/LUv1OVPKz20ETjgfpg8MNwG6 iMizMefCl+RbtXbIEZ3TE/IaDT/jcOirjv96lBKrc/pAL0h/O71Kwbbp43fimW80GhjiaN2y WGByepnkAVP7FyNarhdDpJhoDmUk9yfwNuIuESaCQtfd3vgKKuo6grcKZ8bHy7IXX1XJj2X/ BgRVhVgMHAnDPFIkXtP+SiarkUaLjGzCz7XkUn4XAGDskBNfbizFqYUQCaL2FdbW3DeZqNIa nSzKAZK7Dm9+0VVSRZXP89w71Y7JUV56xL/PlOE+YKKFdEw+gQjQi0e+DZILAtFjJLoCrkEX w4LluMhYX/X8XP6/C3xW0yOZhvHYyn72sV4yJ1uyc/qz3OY32CRy+bwPzAMAkhdwcORA3JPb kPTlimhQqVgvca8m+MQ/JFZ6D+K7QPyvEv7bQ7M+IzFmTkOCwCJ3xqOD6GjX3aphk8Sr0dq3 4Awlf5xFDAG8dn8Uuutb7naGBd/fEv6t8dfkNyzj6yvc4jpVxwARAQABtElBZGhlbWVydmFs IFphbmVsbGEgTmV0dG8gKExpbmFybyBWUE4gS2V5KSA8YWRoZW1lcnZhbC56YW5lbGxhQGxp bmFyby5vcmc+iQI3BBMBCAAhBQJXFRpKAhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJ EKqx7BSnlIjv0e8P/1YOYoNkvJ+AJcNUaM5a2SA9oAKjSJ/M/EN4Id5Ow41ZJS4lUA0apSXW NjQg3VeVc2RiHab2LIB4MxdJhaWTuzfLkYnBeoy4u6njYcaoSwf3g9dSsvsl3mhtuzm6aXFH /Qsauav77enJh99tI4T+58rp0EuLhDsQbnBic/ukYNv7sQV8dy9KxA54yLnYUFqH6pfH8Lly sTVAMyi5Fg5O5/hVV+Z0Kpr+ZocC1YFJkTsNLAW5EIYSP9ftniqaVsim7MNmodv/zqK0IyDB GLLH1kjhvb5+6ySGlWbMTomt/or/uvMgulz0bRS+LUyOmlfXDdT+t38VPKBBVwFMarNuREU2 69M3a3jdTfScboDd2ck1u7l+QbaGoHZQ8ZNUrzgObltjohiIsazqkgYDQzXIMrD9H19E+8fw kCNUlXxjEgH/Kg8DlpoYJXSJCX0fjMWfXywL6ZXc2xyG/hbl5hvsLNmqDpLpc1CfKcA0BkK+ k8R57fr91mTCppSwwKJYO9T+8J+o4ho/CJnK/jBy1pWKMYJPvvrpdBCWq3MfzVpXYdahRKHI ypk8m4QlRlbOXWJ3TDd/SKNfSSrWgwRSg7XCjSlR7PNzNFXTULLB34sZhjrN6Q8NQZsZnMNs TX8nlGOVrKolnQPjKCLwCyu8PhllU8OwbSMKskcD1PSkG6h3r0AquQINBFcVGkoBEACgAdbR Ck+fsfOVwT8zowMiL3l9a2DP3Eeak23ifdZG+8Avb/SImpv0UMSbRfnw/N81IWwlbjkjbGTu oT37iZHLRwYUFmA8fZX0wNDNKQUUTjN6XalJmvhdz9l71H3WnE0wneEM5ahu5V1L1utUWTyh VUwzX1lwJeV3vyrNgI1kYOaeuNVvq7npNR6t6XxEpqPsNc6O77I12XELic2+36YibyqlTJIQ V1SZEbIy26AbC2zH9WqaKyGyQnr/IPbTJ2Lv0dM3RaXoVf+CeK7gB2B+w1hZummD21c1Laua +VIMPCUQ+EM8W9EtX+0iJXxI+wsztLT6vltQcm+5Q7tY+HFUucizJkAOAz98YFucwKefbkTp eKvCfCwiM1bGatZEFFKIlvJ2QNMQNiUrqJBlW9nZp/k7pbG3oStOjvawD9ZbP9e0fnlWJIsj 6c7pX354Yi7kxIk/6gREidHLLqEb/otuwt1aoMPg97iUgDV5mlNef77lWE8vxmlY0FBWIXuZ yv0XYxf1WF6dRizwFFbxvUZzIJp3spAao7jLsQj1DbD2s5+S1BW09A0mI/1DjB6EhNN+4bDB SJCOv/ReK3tFJXuj/HbyDrOdoMt8aIFbe7YFLEExHpSk+HgN05Lg5TyTro8oW7TSMTk+8a5M kzaH4UGXTTBDP/g5cfL3RFPl79ubXwARAQABiQIfBBgBCAAJBQJXFRpKAhsMAAoJEKqx7BSn lIjvI/8P/jg0jl4Tbvg3B5kT6PxJOXHYu9OoyaHLcay6Cd+ZrOd1VQQCbOcgLFbf4Yr+rE9l mYsY67AUgq2QKmVVbn9pjvGsEaz8UmfDnz5epUhDxC6yRRvY4hreMXZhPZ1pbMa6A0a/WOSt AgFj5V6Z4dXGTM/lNManr0HjXxbUYv2WfbNt3/07Db9T+GZkpUotC6iknsTA4rJi6u2ls0W9 1UIvW4o01vb4nZRCj4rni0g6eWoQCGoVDk/xFfy7ZliR5B+3Z3EWRJcQskip/QAHjbLa3pml xAZ484fVxgeESOoaeC9TiBIp0NfH8akWOI0HpBCiBD5xaCTvR7ujUWMvhsX2n881r/hNlR9g fcE6q00qHSPAEgGr1bnFv74/1vbKtjeXLCcRKk3Ulw0bY1OoDxWQr86T2fZGJ/HIZuVVBf3+ gaYJF92GXFynHnea14nFFuFgOni0Mi1zDxYH/8yGGBXvo14KWd8JOW0NJPaCDFJkdS5hu0VY 7vJwKcyHJGxsCLU+Et0mryX8qZwqibJIzu7kUJQdQDljbRPDFd/xmGUFCQiQAncSilYOcxNU EMVCXPAQTteqkvA+gNqSaK1NM9tY0eQ4iJpo+aoX8HAcn4sZzt2pfUB9vQMTBJ2d4+m/qO6+ cFTAceXmIoFsN8+gFN3i8Is3u12u8xGudcBPvpoy4OoG Subject: Re: [PATCH v2] Add scripts/move-symbol-to-libc.py Message-ID: Date: Wed, 5 Aug 2020 16:22:51 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <87imgx2tur.fsf@mid.deneb.enyo.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Adhemerval Zanella via Libc-alpha Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" On 15/05/2020 10:47, Florian Weimer wrote: > This helper script can be used to move symbols to libc.abilist across > all architectures, while preserving their symbol version. LGTM, thanks. Reviewed-by: Adhemerval Zanella > > --- > scripts/move-symbol-to-libc.py | 217 +++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 217 insertions(+) > > diff --git a/scripts/move-symbol-to-libc.py b/scripts/move-symbol-to-libc.py > new file mode 100644 > index 0000000000..4863c4dd8a > --- /dev/null > +++ b/scripts/move-symbol-to-libc.py > @@ -0,0 +1,217 @@ > +#!/usr/bin/python3 > +# Move symbols from other shared objects into libc.so. > +# Copyright (C) 2020 Free Software Foundation, Inc. > +# This file is part of the GNU C Library. > +# > +# The GNU C Library is free software; you can redistribute it and/or > +# modify it under the terms of the GNU Lesser General Public > +# License as published by the Free Software Foundation; either > +# version 2.1 of the License, or (at your option) any later version. > +# > +# The GNU C Library is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > +# Lesser General Public License for more details. > +# > +# You should have received a copy of the GNU Lesser General Public > +# License along with the GNU C Library; if not, see > +# . > + > +"""Move symbols from other shared objects into libc.so. > + > +This script moves ABI symbols from non-libc abilists in to > +libc.abilist. Symbol versions are preserved. The script must be > +called from the top of the glibc source tree. > + > +""" > + > +import argparse > +import os.path > +import sys > + > +def replace_file(path, new_contents): > + """Atomically replace PATH with lines from NEW_CONTENTS. > + > + NEW_CONTENTS must be a sequence of strings. > + > + """ > + temppath = path + 'T' > + with open(temppath, 'w') as out: > + for line in new_contents: > + out.write(line) > + os.rename(temppath, path) > + > +class VersionedSymbol: > + """A combination of a symbol and its version.""" > + > + def __init__(self, symbol, version): > + """Construct a new versioned symbol.""" > + assert symbol > + assert version > + self.symbol = symbol > + self.version = version > + > + def __str__(self): > + return self.symbol + '@' + self.version > + > + def __eq__(self, other): > + return self.symbol == other.symbol and self.version == other.version > + > + def __hash__(self): > + return hash(self.symbol) ^ hash(self.version) Ok. > + > +def read_abilist(path): > + """Read the abilist file at PATH. > + > + Return a dictionary from VersionedSymbols to their flags (as > + strings). > + > + """ > + result = {} > + with open(path) as inp: > + for line in inp: > + version, symbol, flags = line.strip().split(' ', 2) > + result[VersionedSymbol(symbol, version)] = flags > + return result Ok. > + > +def abilist_lines(symbols): > + """Build the abilist file contents (as a list of lines). > + > + SYMBOLS is a dictionary from VersionedSymbols to their flags. > + > + """ > + result = [] > + for versym, flags in symbols.items(): > + result.append('{} {} {}\n'.format( > + versym.version, versym.symbol, flags)) > + result.sort() > + return result > + > +def add_to_libc_path(path, new_symbols): > + """Add SYMBOLS to the abilist file PATH. > + > + NEW_SYMBOLS is a dictionary from VersionedSymbols to their flags. > + > + """ > + original_symbols = read_abilist(path) > + updated_symbols = original_symbols.copy() > + updated_symbols.update(new_symbols) > + if updated_symbols != original_symbols: > + sys.stdout.write('updating libc abilist {}\n'.format(path)) > + replace_file(path, abilist_lines(updated_symbols)) Ok. > + > +# The name of the libc.so abilist file. > +libc_abilist = 'libc.abilist' > + > +def add_to_libc_fallback(directory, subdirs, symbol_lines): > + """Add SYMBOL_LINES to the libc.abilist files in SUBDIRS in DIRECTORY. > + > + All subdirectories must exist. If they do, return True. If not, > + skip processing and return False. > + > + """ > + abilists = [os.path.join(directory, subdir, libc_abilist) > + for subdir in subdirs] > + for abilist in abilists: > + if not os.path.exists(abilist): > + return False > + for abilist in abilists: > + add_to_libc_path(abilist, symbol_lines) > + return True Ok. > + > +def add_to_libc(directory, symbol_lines): > + > + """Add SYMBOL_LINES (a list of strings) to libc.abilist in DIRECTORY. > + > + Try specific subdirectories as well if libc.abilist is not found > + in DIRECTORY. > + > + """ > + libc_path = os.path.join(directory, libc_abilist) > + if os.path.exists(libc_path): > + add_to_libc_path(libc_path, symbol_lines) > + return > + > + # Special case for powerpc32 and mips32 variants. > + if add_to_libc_fallback(directory, ('fpu', 'nofpu'), symbol_lines): > + return > + > + # Special case for mips64. > + if add_to_libc_fallback(directory, ('n32', 'n64'), symbol_lines): > + return > + > + raise IOError('No libc.abilist found for: {}'.format(directory)) Ok. > + > +def move_symbols_1(path, to_move, moved_symbols): > + """Move SYMBOLS from the abilist file PATH to MOVED_SYMBOLS. > + > + TO_MOVE must be a set of strings. MOVED_SYMBOLS is a dictionary. > + > + """ > + suffix = '.abilist' > + assert path.endswith('.abilist') > + library = os.path.basename(path)[:-len(suffix)] > + placeholder = '__{}_version_placeholder'.format(library) > + > + new_lines = [] > + changed = False > + > + old_symbols = read_abilist(path) > + old_versions = set(versym.version for versym in old_symbols.keys()) > + matching_symbols = dict(e for e in old_symbols.items() > + if e[0].symbol in to_move) > + if matching_symbols: > + sys.stdout.write('updating {} abilist {}\n'.format(library, path)) > + new_symbols = dict(e for e in old_symbols.items() > + if e[0].symbol not in to_move) > + > + # Add placeholder symbols to prevent symbol versions from > + # going away completely. > + new_versions = set(versym.version for versym in new_symbols.keys()) > + for missing_version in old_versions - new_versions: > + new_symbols[VersionedSymbol(placeholder, missing_version)] = 'F' > + > + replace_file(path, abilist_lines(new_symbols)) > + > + moved_symbols.update(matching_symbols) Ok. > + > +def move_symbols(directory, files, symbols): > + """Move SYMBOLS from FILES (a list of abilist file names) in DIRECTORY. > + > + SYMBOLS must be a set of strings. > + > + """ > + moved_symbols = {} > + for filename in files: > + move_symbols_1(os.path.join(directory, filename), symbols, > + moved_symbols) > + if moved_symbols: > + add_to_libc(directory, moved_symbols) Ok. > + > +def get_parser(): > + """Return an argument parser for this module.""" > + parser = argparse.ArgumentParser(description=__doc__) > + parser.add_argument('--only-linux', action='store_true', > + help='Restrict the operation to Linux abilists') > + parser.add_argument('symbols', help='name of the symbol to move', > + nargs='+') > + return parser > + > +def main(argv): > + """The main entry point.""" > + parser = get_parser() > + opts = parser.parse_args(argv) > + if opts.only_linux: > + sysdeps = 'sysdeps/unix/sysv/linux' > + else: > + sysdeps = 'sysdeps' > + > + symbols = frozenset(opts.symbols) > + > + for directory, dirs, files in os.walk(sysdeps): > + move_symbols(directory, [name for name in files > + if name != 'libc.abilist' > + and name.endswith('.abilist')], symbols) > + > +if __name__ == '__main__': > + main(sys.argv[1:]) > Ok.