From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id A70801F8C6 for ; Thu, 29 Jul 2021 10:22:21 +0000 (UTC) Received: from localhost ([::1]:53120 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m93BE-0007AS-C4 for normalperson@yhbt.net; Thu, 29 Jul 2021 06:22:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m93B9-0007A6-87 for bug-gnulib@gnu.org; Thu, 29 Jul 2021 06:22:15 -0400 Received: from uggla.sjd.se ([2001:9b1:8633::107]:51526) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m93B6-0004W5-SU for bug-gnulib@gnu.org; Thu, 29 Jul 2021 06:22:14 -0400 DKIM-Signature: v=1; a=ed25519-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=ed2101; h=Content-Type:MIME-Version:Message-ID:In-Reply-To :Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding :Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=nNAwuXxlMasM7DisUOy7LaLTGWXPNxW0XsaVMprngHY=; t=1627554129; x=1628763729; b=C0+idUyjvz2ZefXdjEb3JWl/9y7R7mgtmwk58C1rzoo6hzYuW8xieVgjwfvxEOjxE8VJAD0YnM xWO7VqsNCkBg==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=rsa2101; h=Content-Type:MIME-Version:Message-ID: In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=nNAwuXxlMasM7DisUOy7LaLTGWXPNxW0XsaVMprngHY=; t=1627554129; x=1628763729; b=GSFROP9d+nkaLdyfq0N0uM/aCLeFNl27mV3MaHq52KsSWsSDZ12oHm0dYGeNhQZX0dWmUlm6jz tQgeKpAYGjP/iXkGBfDIwAVr+mA7M1XKyulru+tR9JxD/MwfNk/l5mORoY97LE3AKVxKT4KbGmPDS 3vJn53J7Pl0DWsHzowP6mH7zUXtaNc0PMoxDqmCm6NRL854/ihLJrliTyPVXpXZU8IeDmD/uDLNg5 HtDrXcazsWhcuMrQZnHoS9VfWpVyRhi4rBNDZ6mSlT/q4o192RvcB+GcuUBYdAL/RAjgD2kpWPfLA l7iO/I5CTb5qzBNnrxLgoqpZal1/Qw1JwSLocW01lO+mAHy1vi861ynebGp6JXnhkk5P4sNcewN8x Qpo2XLI5OiVLFE2Q1Koll9/Hakz2KUUuCyMBghMHo+jabnEBPgCIjgrbBcxwLO4iJE3CxpTEm2 ; Received: from [2001:9b1:41ac:ff00:b714:bfd1:35cb:d9bb] (port=42226 helo=latte) by uggla.sjd.se with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m93Az-0004Al-NC; Thu, 29 Jul 2021 10:22:05 +0000 To: Jim Meyering Cc: "bug-gnulib@gnu.org List" Subject: Re: fts in gnulib behave different than glibc References: <87r1fig8sj.fsf@latte.josefsson.org> OpenPGP: id=B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE; url=https://josefsson.org/key-20190320.txt X-Hashcash: 1:22:210729:jim@meyering.net::tLnNQQobZjuadPM+:6N9v X-Hashcash: 1:22:210729:bug-gnulib@gnu.org::zwjJLu9zgbDegqMj:9b1O Date: Thu, 29 Jul 2021 12:22:04 +0200 In-Reply-To: (Jim Meyering's message of "Wed, 28 Jul 2021 22:42:53 -0700") Message-ID: <87lf5p4dyr.fsf@latte.josefsson.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=2001:9b1:8633::107; envelope-from=simon@josefsson.org; helo=uggla.sjd.se X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" Reply-to: Simon Josefsson From: Simon Josefsson via Gnulib discussion list --=-=-= Content-Type: text/plain Jim Meyering writes: > On Wed, Jul 28, 2021 at 1:08 AM Simon Josefsson via Gnulib discussion > list wrote: >> Hi. I replaced GNU InetUtils' old custom fts implementation with the >> one from gnulib, but self-tests started failing. Looking at the code, >> it seems gnulib's fts implementation has diverged compared to glibc, and >> has some optimizations that (I think) change the API (wrt stat and >> chdir). Also, gnulib's fts module is always enabled, even on modern >> glibc systems. InetUtils's usage of fts works fine with modern glibc, >> but it didn't work with gnulib's version (it needed a FTS_NOCHDIR). The >> gnulib manual for the fts replacement module isn't terribly clear about >> this. Is there a reason for this behaviour? >> >> I would prefer if there were two fts modules in gnulib: >> >> 1) One module 'fts' based on glibc's code, that is only enabled in on >> systems that doesn't have fts, or where fts is known to be buggy. >> >> 2) One 'fts-faster' that is the current code, which can be used when you >> always wants to pull in the optimized implementation. >> >> Then InetUtils would use system fts on glibc platforms, and not always >> have to pull in a replacement copy. >> >> What do you think? >> >> I could live with a new module 'fts-optional' that only pulls in the >> current 'fts' module when the system is lacking it. That doesn't fix >> the API confusion, but is probably sufficient for InetUtils. > > There are fundamental flaws in the ABI of glibc's fts that make it > unsuitable for use in any tool I care about. Ouch -- is there disagreement from the glibc people on fixing glibc fts? Maybe the reaction will be different now. Are the problems inherent with the glibc ABI, or can they be fixed? If it isn't possible to fix, maybe the entire API should be declared deprecated and eventually removed. The glibc manual doesn't seem to document fts?! > Those flaws make it easy to hit silly limits or to provoke inordinate > resource usage/DoS. It would be nice to document these problems in more detail in the gnulib manual. Is there any known behavioural difference between glibc fts and gnulib fts? Documenting that too would be useful. InetUtils' required a FTS_NOCHDIR flag in order to continue behave as before (a simple command like 'ls foo' where foo is a directory failed). I don't see any self-tests in gnulib without that flag, so maybe this suggests there is some API/ABI difference. > Is it ok for InetUtil's fts to be unable to do these things? (each of > which afflicts glibc fts, from what I recall) > - process files in a tree more than 2^16 levels deep > - detect certain cycles efficiently > - delete (in reasonable time) a hierarchy with too many entries in a > single directory. InetUtils only uses fts for "DIR" in ftpd, when it emulate /bin/ls internally (based on some old BSD implementation of /bin/ls that uses fts). The first two applies, but not the third, I think, but it sounds like corner-cases. /Simon --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCYQKBTRQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFooX8AQC9tQRQgKW51uPKR54y6YbFI5T7XcWS rSRsndI8T2E27QEAgVlUbNw4dPgUOSV10gFBazfP7oxEiIkcjhoY75IzJwM= =3Y0d -----END PGP SIGNATURE----- --=-=-=--