From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on starla X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 078D61F44D for ; Fri, 19 Apr 2024 00:17:30 +0000 (UTC) Authentication-Results: dcvr.yhbt.net; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=W61h/IrT; dkim-atps=neutral Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D0111384AB58 for ; Fri, 19 Apr 2024 00:17:28 +0000 (GMT) Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) by sourceware.org (Postfix) with ESMTPS id 5DDCF3858D33 for ; Fri, 19 Apr 2024 00:16:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5DDCF3858D33 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5DDCF3858D33 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::136 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713485794; cv=none; b=WVPpZ9qeIFb+lNVQcpgr4EfZgRNI8UozGZMKVX6GP6AMo5c38NUD01hnOfGwhJUZ2dmMZz/MjGCsBdJh7L7t+Sc8uloSXL6Px6jakhNsXio4GySKlOj1n9No5SbTpJ05/s40wgQ3Izo5DFWFEw1vIoASF9W/77vuVUlqmhPIj24= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713485794; c=relaxed/simple; bh=bRX4xog3J9LrzJkRT75Cs9EE2lWgEj+BL91lBsyjbFU=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=goUc8XUBNhpHs4lmUOHheoteTsS/bKVKyqSmEdCqWiARI0y/Hx8Kirlf24bDxSkDnR3cHs/U+k7L13yrFeZ1PLk2oz4yj6OMwHR+AUzXdheJodOhLwk+yAmOdhtbkt6y0EmBJLNFwBzUPFlCEFYs+JRpekRoBvYE2ZtSMm/OXQ8= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-518a56cdbcfso2356046e87.2 for ; Thu, 18 Apr 2024 17:16:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713485780; x=1714090580; darn=sourceware.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=LqGaFZ8SFFLuj7cvOYPPxXOytoIV7ST1Nnbw+2vgRII=; b=W61h/IrTwA+fR2wWaOKsGHMLEqCTpXOGFWKkfEAU7zjOXiz2LRrO19Y07wTpDPJG95 nUQoIk0MAQxsOYEvhK8t4N0Qzn/z3DqjwGCx+Af3cAUDh09vDsIN343HQlKStYEpL/1h 77e37bJbZAQzkXrCpFJozLRlWwK+P0NLRiEDyqeowomadWP+1HZ9rOFA0s6uoMtrncYA XH42QBorSDBOUmQp60Mm3jknJz9oLqjn9dvFoblwJTPL7UlnL1DI98eXVGsyYqC3l1fA bbfcx0hHNRN57MOVrioFo5R7Fl8F/qb/iKR+qHw9ZhYUfpC6qyaudWOG6RuEgvzAw26v E7xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713485780; x=1714090580; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LqGaFZ8SFFLuj7cvOYPPxXOytoIV7ST1Nnbw+2vgRII=; b=moR5wJ97bNFH3xk/84jn5+RyBaKhkmK8KSTCDQt1oQBVTpIWPdQe3kx/uNlGScvISL 3zy937qRgNsxSNbcgxP3BCpBliQHjRoI24SyjN3V1Rd12Xs0J1ztuIet+FyYsMqRbqRp 1/35bkJw3IgQHjRLQsWWnUX8wIieBsY/F9VD/UEC4CEJ14JsQpAdF93r0MW+K8KxLZrF HxMIPXPFv1oB1x9hyrjdsIdnqvGZ9OIM9rz+7OwCrdbpQnpFpBxINzm0Gy5RFTLwXIhg Vpv/Hx/0oj6uOEJk+Zy3FHAvZkJgbKeLn0+AywYG1WfqkeIM/So4X2ZbTicoRGMvq5aw U7hw== X-Forwarded-Encrypted: i=1; AJvYcCXk0iNu7hqdE0RzxSAb7TQZK00Kek/HqxhJF+F2Rl+7T3XhoetdsTdy17rY3jCC0pMMrERHAJEj7A+yLcBoCJcE9saGKAPMiQ/b X-Gm-Message-State: AOJu0YxrgZRqNExoSkIeisxuzhIANqWz8PZF05d/s5OVdy7TpFMzC8xJ +UH8nrd6oOSEPcOq82xppr4twA+YxIWonyIVK8ur6Vxe1LYfbD5zrvpBnjLWg+AZoUIqAG2CrkJ /PGlfBv9tC5xXNm7NpywKsE7YTcg= X-Google-Smtp-Source: AGHT+IHkiYyqp8S/4Oqic2R6tmHRvbPDHMKyZE+3v1pxRssUa9MDhGgKAl1E/jx0dLApsW2g3+zovvuGt+XFZzBHX5c= X-Received: by 2002:ac2:4642:0:b0:516:d232:2516 with SMTP id s2-20020ac24642000000b00516d2322516mr395092lfo.6.1713485779568; Thu, 18 Apr 2024 17:16:19 -0700 (PDT) MIME-Version: 1.0 References: <9831afe6-958a-fbd3-9434-05dd0c9b602a@draigBrady.com> <317fe0e2-8cf9-d4ac-ed56-e6ebcc2baa55@cs.wisc.edu> <8c490a55-598a-adf6-67c2-eb2a6099620a@cs.wisc.edu> <8d26e5046cc8bf49490e4aa3f6c00b09@kylheku.com> In-Reply-To: From: Zachary Santer Date: Thu, 18 Apr 2024 20:16:07 -0400 Message-ID: Subject: Modify buffering of standard streams via environment variables (not LD_PRELOAD)? To: Carl Edquist Cc: Kaz Kylheku , libc-alpha@sourceware.org, coreutils@gnu.org, =?UTF-8?Q?P=C3=A1draig_Brady?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Was "RFE: enable buffering on null-terminated data" On Wed, Mar 20, 2024 at 4:54=E2=80=AFAM Carl Edquist = wrote: > > However, if stdbuf's magic env vars are exported in your shell (either by > doing a trick like 'export $(env -i stdbuf -oL env)', or else more simply > by first starting a new shell with 'stdbuf -oL bash'), then every command > in your pipelines will start with the new default line-buffered stdout. > That way your line-items from build.sh should get passed all the way > through the pipeline as they are produced. Finally had a chance to try to build with 'stdbuf --output=3DL --error=3DL --' in front of the build script, and it caused some crazy problems. I was building Ada, though, so pretty good chance that part of the build chain doesn't link against libc at all. I got a bunch of ERROR: ld.so: object '/usr/libexec/coreutils/libstdbuf.so' from LD_PRELOAD cannot be preloaded: ignored. And then it somehow caused compiler errors relating to the size of what would be pointer types. Cleared out all the build products and tried again without stdbuf and everything was fine. >From the original thread just within the coreutils email list, "stdbuf feature request - line buffering but for null-terminated data": On Tue, Mar 12, 2024 at 12:42=E2=80=AFPM Kaz Kylheku wrot= e: > > I would say that if it is implemented, the programs which require > it should all make provisions to set it up themselves. > > stdbuf is a hack/workaround for programs that ignore the > issue of buffering. Specifically, programs which send information > to one of the three standard streams, such that the information > is required in a timely way. Those streams become fully buffered > when not connected to a terminal. I think I've partially come around to this point of view. However, instead of expecting all sorts of individual programs to implement their own buffering mode command-line options, could this be handled with environment variables, but without LD_PRELOAD? I don't know if libc itself can check for those environment variables and adjust each program's buffering on its own, but if so, that would be a much simpler solution. You could compare this to the various locale environment variables, though I think a lot of commands whose behavior differ from locale to locale do have to implement their own handling of that internally, at least to some extent. This seems like somewhat less of a hack, and if no part of a program looks for those environment variables, it isn't going to find itself getting broken by the dynamic linker. It's just not going to change its buffering. Additionally, things that don't link against libc could still honor these environment variables, if the developers behind them care to put in the effort. Zack