From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id E80F71F461 for ; Fri, 5 Jul 2019 02:37:27 +0000 (UTC) Received: from localhost ([::1]:49568 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hjE6H-0005DN-HQ for normalperson@yhbt.net; Thu, 04 Jul 2019 22:37:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54142) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hjE69-0005DA-F3 for bug-gnulib@gnu.org; Thu, 04 Jul 2019 22:37:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hjE67-0001yC-70 for bug-gnulib@gnu.org; Thu, 04 Jul 2019 22:37:17 -0400 Received: from mo6-p00-ob.smtp.rzone.de ([2a01:238:20a:202:5300::8]:36870) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hjE64-0001vI-S4; Thu, 04 Jul 2019 22:37:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1562294225; s=strato-dkim-0002; d=clisp.org; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: X-RZG-CLASS-ID:X-RZG-AUTH:From:Subject:Sender; bh=ojnZ7cItioMGmN6JHmanxD6dqPUS8MYj23JT3quGwy8=; b=Rukwi1lAMQzANqLWFYUpJNvfZEZAz33px/U7ewTWmYrGW8fNsVrSq5QtXVgrGyk6TK 01W23tOuHvGwwsvSQO383tzT3Fv9TetzhpA4uT1Ej3QkOU+LG5Xu2FAoFsjlEPHnlKPJ qWM2Q5bOrAAw9YX38LMTwChhdTCsQSLbKlOKFjg/9XOBf3bokekZwh15CDiIZVL+tp+i ycwW/1FxqEXndrLA0ynGqAMrtFc5TpfxWa6vRMgL+1UJQcugXIatEeXO0253r0AnvhjM Rp42DH9ak23lkiW7imvO3ZyFn+0tdh2KCzMcRVEB9JSoDiI468rbUS0DzPtFGZZ5DVGK 0IlQ== X-RZG-AUTH: ":Ln4Re0+Ic/6oZXR1YgKryK8brlshOcZlIWs+iCP5vnk6shH+AHjwLuWOGaf0zJZW" X-RZG-CLASS-ID: mo00 Received: from bruno.haible.de by smtp.strato.de (RZmta 44.24 DYNA|AUTH) with ESMTPSA id v018bcv652awP7K (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Fri, 5 Jul 2019 04:36:58 +0200 (CEST) From: Bruno Haible To: bug-gnulib@gnu.org Subject: Re: [PATCH] stat: don't explicitly request file size for filenames Date: Fri, 05 Jul 2019 04:36:57 +0200 Message-ID: <3609540.Sq8Tfar0L4@omega> User-Agent: KMail/5.1.3 (Linux/4.4.0-151-generic; KDE/5.18.0; x86_64; ; ) In-Reply-To: References: <7AC7AA2A-EDEA-4A80-9A5A-02FAA2D823EF@whamcloud.com> <5a1b87fe-e265-6b0f-66eb-20ad1ba9c6e5@draigBrady.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart5855753.bPjDizB7X0" Content-Transfer-Encoding: 7Bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a01:238:20a:202:5300::8 X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jeff Layton , Andreas Dilger , "coreutils@gnu.org" Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" This is a multi-part message in MIME format. --nextPart5855753.bPjDizB7X0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Andreas Dilger wrote: > I was worried if > "ls" or something is saving all of the symlink targets in memory that using a > too-large buffer size would cause excess memory to be allocated for a long time. Good point. careadlinkat.c has a "Shrink BUF before returning it." logic, but not all similar functions in gnulib do. As the attached test program shows, some platforms (glibc, Cygwin) have a special optimization for a shrinking-realloc of the last malloc()ed block. Other platforms don't have this and a shrinking-realloc moves the data if there is a significant gain in space by doing so. Therefore I think it would be good to add a "Shrink BUF before returning it." logic also to areadlink-with-size.c areadlinkat-with-size.c etc. Something like this: diff --git a/lib/areadlink-with-size.c b/lib/areadlink-with-size.c index eacad3f..364cc08 100644 --- a/lib/areadlink-with-size.c +++ b/lib/areadlink-with-size.c @@ -87,6 +87,13 @@ areadlink_with_size (char const *file, size_t size) if (link_length < buf_size) { buffer[link_length] = 0; + /* Shrink BUFFER before returning it. */ + if (link_length + 1 < buf_size) + { + char *shrinked_buffer = realloc (buffer, link_length + 1); + if (shrinked_buffer != NULL) + buffer = shrinked_buffer; + } return buffer; } --nextPart5855753.bPjDizB7X0 Content-Disposition: attachment; filename="foo.c" Content-Transfer-Encoding: 7Bit Content-Type: text/x-csrc; charset="UTF-8"; name="foo.c" #include #include int main () { size_t sizes[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 15, 16, 17, 23, 31, 32, 33, 45, 63, 64, 65, 91, 127, 128, 129, 181, 255, 256, 257, 362, 511, 512, 513, 724, 1023, 1024, 1025, 1448, 2047, 2048, 2049, 2896, 4095, 4096, 4097, 5793, 8191, 8192, 8193, 11585, 16383, 16384, 16385, 23170, 32767, 32768, 32769, 46341, 65535, 65536, 65537 }; int i; for (i = 0; i < sizeof (sizes)/sizeof(sizes[0]); i++) { size_t size = sizes[i]; int j; for (j = size; j > 0; j--) { char *block = malloc (size); char *new_block = realloc (block, j); if (new_block == NULL) abort (); free (new_block); if (new_block != block) break; } printf ("initial size = %u: moved in memory for size <= %u\n", (unsigned int) size, (unsigned int) j); } } /* glibc 2.23, Cygwin: initial size = 1: moved in memory for size <= 0 initial size = 2: moved in memory for size <= 0 initial size = 3: moved in memory for size <= 0 initial size = 4: moved in memory for size <= 0 initial size = 5: moved in memory for size <= 0 initial size = 6: moved in memory for size <= 0 initial size = 7: moved in memory for size <= 0 initial size = 8: moved in memory for size <= 0 initial size = 9: moved in memory for size <= 0 initial size = 11: moved in memory for size <= 0 initial size = 15: moved in memory for size <= 0 initial size = 16: moved in memory for size <= 0 initial size = 17: moved in memory for size <= 0 initial size = 23: moved in memory for size <= 0 initial size = 31: moved in memory for size <= 0 initial size = 32: moved in memory for size <= 0 initial size = 33: moved in memory for size <= 0 initial size = 45: moved in memory for size <= 0 initial size = 63: moved in memory for size <= 0 initial size = 64: moved in memory for size <= 0 initial size = 65: moved in memory for size <= 0 initial size = 91: moved in memory for size <= 0 initial size = 127: moved in memory for size <= 0 initial size = 128: moved in memory for size <= 0 initial size = 129: moved in memory for size <= 0 initial size = 181: moved in memory for size <= 0 initial size = 255: moved in memory for size <= 0 initial size = 256: moved in memory for size <= 0 initial size = 257: moved in memory for size <= 0 initial size = 362: moved in memory for size <= 0 initial size = 511: moved in memory for size <= 0 initial size = 512: moved in memory for size <= 0 initial size = 513: moved in memory for size <= 0 initial size = 724: moved in memory for size <= 0 initial size = 1023: moved in memory for size <= 0 initial size = 1024: moved in memory for size <= 0 initial size = 1025: moved in memory for size <= 0 initial size = 1448: moved in memory for size <= 0 initial size = 2047: moved in memory for size <= 0 initial size = 2048: moved in memory for size <= 0 initial size = 2049: moved in memory for size <= 0 initial size = 2896: moved in memory for size <= 0 initial size = 4095: moved in memory for size <= 0 initial size = 4096: moved in memory for size <= 0 initial size = 4097: moved in memory for size <= 0 initial size = 5793: moved in memory for size <= 0 initial size = 8191: moved in memory for size <= 0 initial size = 8192: moved in memory for size <= 0 initial size = 8193: moved in memory for size <= 0 initial size = 11585: moved in memory for size <= 0 initial size = 16383: moved in memory for size <= 0 initial size = 16384: moved in memory for size <= 0 initial size = 16385: moved in memory for size <= 0 initial size = 23170: moved in memory for size <= 0 initial size = 32767: moved in memory for size <= 0 initial size = 32768: moved in memory for size <= 0 initial size = 32769: moved in memory for size <= 0 initial size = 46341: moved in memory for size <= 0 initial size = 65535: moved in memory for size <= 0 initial size = 65536: moved in memory for size <= 0 initial size = 65537: moved in memory for size <= 0 FreeBSD 12: initial size = 1: moved in memory for size <= 0 initial size = 2: moved in memory for size <= 0 initial size = 3: moved in memory for size <= 0 initial size = 4: moved in memory for size <= 0 initial size = 5: moved in memory for size <= 0 initial size = 6: moved in memory for size <= 0 initial size = 7: moved in memory for size <= 0 initial size = 8: moved in memory for size <= 0 initial size = 9: moved in memory for size <= 8 initial size = 11: moved in memory for size <= 8 initial size = 15: moved in memory for size <= 8 initial size = 16: moved in memory for size <= 8 initial size = 17: moved in memory for size <= 16 initial size = 23: moved in memory for size <= 16 initial size = 31: moved in memory for size <= 16 initial size = 32: moved in memory for size <= 16 initial size = 33: moved in memory for size <= 32 initial size = 45: moved in memory for size <= 32 initial size = 63: moved in memory for size <= 48 initial size = 64: moved in memory for size <= 48 initial size = 65: moved in memory for size <= 64 initial size = 91: moved in memory for size <= 80 initial size = 127: moved in memory for size <= 112 initial size = 128: moved in memory for size <= 112 initial size = 129: moved in memory for size <= 128 initial size = 181: moved in memory for size <= 160 initial size = 255: moved in memory for size <= 224 initial size = 256: moved in memory for size <= 224 initial size = 257: moved in memory for size <= 256 initial size = 362: moved in memory for size <= 320 initial size = 511: moved in memory for size <= 448 initial size = 512: moved in memory for size <= 448 initial size = 513: moved in memory for size <= 512 initial size = 724: moved in memory for size <= 640 initial size = 1023: moved in memory for size <= 896 initial size = 1024: moved in memory for size <= 896 initial size = 1025: moved in memory for size <= 1024 initial size = 1448: moved in memory for size <= 1280 initial size = 2047: moved in memory for size <= 1792 initial size = 2048: moved in memory for size <= 1792 initial size = 2049: moved in memory for size <= 2048 initial size = 2896: moved in memory for size <= 2560 initial size = 4095: moved in memory for size <= 3584 initial size = 4096: moved in memory for size <= 3584 initial size = 4097: moved in memory for size <= 4096 initial size = 5793: moved in memory for size <= 5120 initial size = 8191: moved in memory for size <= 7168 initial size = 8192: moved in memory for size <= 7168 initial size = 8193: moved in memory for size <= 8192 initial size = 11585: moved in memory for size <= 10240 initial size = 16383: moved in memory for size <= 14336 initial size = 16384: moved in memory for size <= 14336 initial size = 16385: moved in memory for size <= 14336 initial size = 23170: moved in memory for size <= 14336 initial size = 32767: moved in memory for size <= 14336 initial size = 32768: moved in memory for size <= 14336 initial size = 32769: moved in memory for size <= 14336 initial size = 46341: moved in memory for size <= 14336 initial size = 65535: moved in memory for size <= 14336 initial size = 65536: moved in memory for size <= 14336 initial size = 65537: moved in memory for size <= 14336 Solaris 10: initial size = 1: moved in memory for size <= 0 initial size = 2: moved in memory for size <= 0 initial size = 3: moved in memory for size <= 0 initial size = 4: moved in memory for size <= 0 initial size = 5: moved in memory for size <= 0 initial size = 6: moved in memory for size <= 0 initial size = 7: moved in memory for size <= 0 initial size = 8: moved in memory for size <= 0 initial size = 9: moved in memory for size <= 8 initial size = 11: moved in memory for size <= 8 initial size = 15: moved in memory for size <= 8 initial size = 16: moved in memory for size <= 8 initial size = 17: moved in memory for size <= 16 initial size = 23: moved in memory for size <= 16 initial size = 31: moved in memory for size <= 24 initial size = 32: moved in memory for size <= 24 initial size = 33: moved in memory for size <= 32 initial size = 45: moved in memory for size <= 32 initial size = 63: moved in memory for size <= 32 initial size = 64: moved in memory for size <= 32 initial size = 65: moved in memory for size <= 32 initial size = 91: moved in memory for size <= 32 initial size = 127: moved in memory for size <= 32 initial size = 128: moved in memory for size <= 32 initial size = 129: moved in memory for size <= 32 initial size = 181: moved in memory for size <= 32 initial size = 255: moved in memory for size <= 32 initial size = 256: moved in memory for size <= 32 initial size = 257: moved in memory for size <= 32 initial size = 362: moved in memory for size <= 32 initial size = 511: moved in memory for size <= 32 initial size = 512: moved in memory for size <= 32 initial size = 513: moved in memory for size <= 32 initial size = 724: moved in memory for size <= 32 initial size = 1023: moved in memory for size <= 32 initial size = 1024: moved in memory for size <= 32 initial size = 1025: moved in memory for size <= 32 initial size = 1448: moved in memory for size <= 32 initial size = 2047: moved in memory for size <= 32 initial size = 2048: moved in memory for size <= 32 initial size = 2049: moved in memory for size <= 32 initial size = 2896: moved in memory for size <= 32 initial size = 4095: moved in memory for size <= 32 initial size = 4096: moved in memory for size <= 32 initial size = 4097: moved in memory for size <= 32 initial size = 5793: moved in memory for size <= 32 initial size = 8191: moved in memory for size <= 32 initial size = 8192: moved in memory for size <= 32 initial size = 8193: moved in memory for size <= 32 initial size = 11585: moved in memory for size <= 32 initial size = 16383: moved in memory for size <= 32 initial size = 16384: moved in memory for size <= 32 initial size = 16385: moved in memory for size <= 32 initial size = 23170: moved in memory for size <= 32 initial size = 32767: moved in memory for size <= 32 initial size = 32768: moved in memory for size <= 32 initial size = 32769: moved in memory for size <= 32 initial size = 46341: moved in memory for size <= 32 initial size = 65535: moved in memory for size <= 32 initial size = 65536: moved in memory for size <= 32 initial size = 65537: moved in memory for size <= 32 MSVC: initial size = 1: moved in memory for size <= 0 initial size = 2: moved in memory for size <= 0 initial size = 3: moved in memory for size <= 0 initial size = 4: moved in memory for size <= 2 initial size = 5: moved in memory for size <= 5 initial size = 6: moved in memory for size <= 6 initial size = 7: moved in memory for size <= 7 initial size = 8: moved in memory for size <= 8 initial size = 9: moved in memory for size <= 9 initial size = 11: moved in memory for size <= 11 initial size = 15: moved in memory for size <= 15 initial size = 16: moved in memory for size <= 16 initial size = 17: moved in memory for size <= 17 initial size = 23: moved in memory for size <= 23 initial size = 31: moved in memory for size <= 18 initial size = 32: moved in memory for size <= 32 initial size = 33: moved in memory for size <= 0 initial size = 45: moved in memory for size <= 33 initial size = 63: moved in memory for size <= 55 initial size = 64: moved in memory for size <= 64 initial size = 65: moved in memory for size <= 58 initial size = 91: moved in memory for size <= 76 initial size = 127: moved in memory for size <= 122 initial size = 128: moved in memory for size <= 128 initial size = 129: moved in memory for size <= 127 initial size = 181: moved in memory for size <= 178 initial size = 255: moved in memory for size <= 252 initial size = 256: moved in memory for size <= 256 initial size = 257: moved in memory for size <= 254 initial size = 362: moved in memory for size <= 359 initial size = 511: moved in memory for size <= 506 initial size = 512: moved in memory for size <= 512 initial size = 513: moved in memory for size <= 509 initial size = 724: moved in memory for size <= 721 initial size = 1023: moved in memory for size <= 1020 initial size = 1024: moved in memory for size <= 1024 initial size = 1025: moved in memory for size <= 1022 initial size = 1448: moved in memory for size <= 1445 initial size = 2047: moved in memory for size <= 2043 initial size = 2048: moved in memory for size <= 2048 initial size = 2049: moved in memory for size <= 2046 initial size = 2896: moved in memory for size <= 2893 initial size = 4095: moved in memory for size <= 4091 initial size = 4096: moved in memory for size <= 4096 initial size = 4097: moved in memory for size <= 4094 initial size = 5793: moved in memory for size <= 5790 initial size = 8191: moved in memory for size <= 8188 initial size = 8192: moved in memory for size <= 8192 initial size = 8193: moved in memory for size <= 8190 initial size = 11585: moved in memory for size <= 11582 initial size = 16383: moved in memory for size <= 16380 initial size = 16384: moved in memory for size <= 16384 initial size = 16385: moved in memory for size <= 0 initial size = 23170: moved in memory for size <= 0 initial size = 32767: moved in memory for size <= 0 initial size = 32768: moved in memory for size <= 0 initial size = 32769: moved in memory for size <= 0 initial size = 46341: moved in memory for size <= 0 initial size = 65535: moved in memory for size <= 0 initial size = 65536: moved in memory for size <= 0 initial size = 65537: moved in memory for size <= 0 */ --nextPart5855753.bPjDizB7X0--