Re: RFE: enable buffering on null-terminated data

[Zachary Santer <zsanter@gmail.com>]

On Thu, Mar 14, 2024 at 11:14 AM Carl Edquist <edquist@cs.wisc.edu> wrote:

> Where things get sloppy is if you add some stuff in a pipeline after your
> build script, which results in things getting block-buffered along the
> way:
>
>         $ ./build.sh | sed s/what/ever/ | tee build.log
>
> And there you will definitely see a difference.

Sadly, the man page for stdbuf specifically calls out tee as being
unaffected by stdbuf, because it adjusts the buffering of its standard
streams itself. The script I mentioned pipes everything through tee,
and I don't think I'm willing to refactor it not to. Ah well.

> Oh, I imagine "undefined operation" means something more like
> "unspecified" here.  stdbuf(1) uses setbuf(3), so the behavior you'll get
> should be whatever the setbuf(3) from the libc on your system does.
>
> I think all this means is that the C/POSIX standards are a bit loose about
> what is required of setbuf(3) when a buffer size is specified, and there
> is room in the standard for it to be interpreted as only a hint.

> Works for me (on glibc-2.23)

Thanks for setting me straight here.

> What may not be obvious is that the shell does not need to get involved
> with writing input for a coprocess or reading its output - the shell can
> start other (very fast) programs with input/output redirected to/from the
> coprocess pipes to do that processing.

Gosh, I'd like to see an example of that, too.

> My point though earlier was that a null-terminated record buffering mode,
> as useful as it sounds on the surface (for null-terminated paths), may
> actually be something _nobody_ has ever actually needed for an actual (not
> contrived) workflow.

I considered how it seemed like something people could need years ago
and only thought to email into email lists about it last weekend.
Maybe there are all sorts of people out there who have been using
'stdbuf --output=0' on null-terminated data for years and never
thought to raise the issue. I know that's not a very strong argument,
though.