From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id BFC571F8C6 for ; Thu, 29 Jul 2021 05:43:20 +0000 (UTC) Received: from localhost ([::1]:45696 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m8ypD-0006az-J3 for normalperson@yhbt.net; Thu, 29 Jul 2021 01:43:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54558) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8yp4-0006aV-Gy for bug-gnulib@gnu.org; Thu, 29 Jul 2021 01:43:13 -0400 Received: from mail-wr1-f50.google.com ([209.85.221.50]:45571) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m8yp1-0007K1-BT for bug-gnulib@gnu.org; Thu, 29 Jul 2021 01:43:10 -0400 Received: by mail-wr1-f50.google.com with SMTP id m12so471991wru.12 for ; Wed, 28 Jul 2021 22:43:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7fGAWxlUOu2wGkZ0l3aPappmZN/yYfQZeYAtJ/Oj6o4=; b=okThyYRXUAzziao9lRX8cr1WB17+0V5g6buYJ78aQD+ojNVhcTF1utTPDuK2PYIvrn dYf3l+zuHeG24AVTB3A1YvXO8TblxRW7UZKZToa8//VOU3vfFOL+ATO/wrWovw8Eae24 rr1mxHkXCN3EkRF+Y2F68Uezigm7h/cxFmNiAyycbt9mHDjZn0micHvv9Ar0CTHFuIde PRMW45B5l1H7FxvvasOF9ChBlrzdc97AV/4POeIOcvMhxAJsz8dCAFL6KzqbJ1WVrvmB N+KDQo3GUYlYZu0mK3sCzYYLugUOkzNwteYz0r502l58fjp8I0DAOY07Z3dyiw99Iw2R C6EQ== X-Gm-Message-State: AOAM531uMzsRV3aZ+pkyEP+XOAMiFgqvGVtraUlt+gb2k1klhm2722sq eE+VkrQKUR6NQjIkwjuEwVFlecBVNckpBM4ostA= X-Google-Smtp-Source: ABdhPJwmd9zZFa89hIWgsJAt6FURhRih4awuqgabDsEHtN6qxBEsaA5KXSpht7TYSK58YiQqmngkvRxKOnLgcWTeZ5g= X-Received: by 2002:adf:f20d:: with SMTP id p13mr2730396wro.287.1627537385960; Wed, 28 Jul 2021 22:43:05 -0700 (PDT) MIME-Version: 1.0 References: <87r1fig8sj.fsf@latte.josefsson.org> In-Reply-To: <87r1fig8sj.fsf@latte.josefsson.org> From: Jim Meyering Date: Wed, 28 Jul 2021 22:42:53 -0700 Message-ID: Subject: Re: fts in gnulib behave different than glibc To: Simon Josefsson Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=209.85.221.50; envelope-from=meyering@gmail.com; helo=mail-wr1-f50.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "bug-gnulib@gnu.org List" Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" On Wed, Jul 28, 2021 at 1:08 AM Simon Josefsson via Gnulib discussion list wrote: > Hi. I replaced GNU InetUtils' old custom fts implementation with the > one from gnulib, but self-tests started failing. Looking at the code, > it seems gnulib's fts implementation has diverged compared to glibc, and > has some optimizations that (I think) change the API (wrt stat and > chdir). Also, gnulib's fts module is always enabled, even on modern > glibc systems. InetUtils's usage of fts works fine with modern glibc, > but it didn't work with gnulib's version (it needed a FTS_NOCHDIR). The > gnulib manual for the fts replacement module isn't terribly clear about > this. Is there a reason for this behaviour? > > I would prefer if there were two fts modules in gnulib: > > 1) One module 'fts' based on glibc's code, that is only enabled in on > systems that doesn't have fts, or where fts is known to be buggy. > > 2) One 'fts-faster' that is the current code, which can be used when you > always wants to pull in the optimized implementation. > > Then InetUtils would use system fts on glibc platforms, and not always > have to pull in a replacement copy. > > What do you think? > > I could live with a new module 'fts-optional' that only pulls in the > current 'fts' module when the system is lacking it. That doesn't fix > the API confusion, but is probably sufficient for InetUtils. There are fundamental flaws in the ABI of glibc's fts that make it unsuitable for use in any tool I care about. Those flaws make it easy to hit silly limits or to provoke inordinate resource usage/DoS. Is it ok for InetUtil's fts to be unable to do these things? (each of which afflicts glibc fts, from what I recall) - process files in a tree more than 2^16 levels deep - detect certain cycles efficiently - delete (in reasonable time) a hierarchy with too many entries in a single directory.