Re: RFE: enable buffering on null-terminated data

[Carl Edquist <edquist@cs.wisc.edu>]

[-- Attachment #1: Type: text/plain, Size: 1499 bytes --]


On Tue, 19 Mar 2024, Zachary Santer wrote:

> On Tue, Mar 19, 2024 at 1:24 AM Kaz Kylheku <kaz@kylheku.com> wrote:
>>
>> But what tee does is set up _IONBF on its output streams,
>> including stdout.
>
> So it doesn't buffer at all. Awesome. Nevermind.

Yay!  :D

And since tee uses fwrite to copy whatever input is available, that will 
mean 'records' are output on the same boundaries as the input (whether 
that be newlines, nuls, or just block boundaries).  So putting tee in the 
middle of a pipeline shouldn't itself interfere with whatever else you're 
up to.  (AND it's still relatively efficient, compared to some tools like 
cut that putchar a byte at a time.)

My note about pipelines like this though:

 	$ ./build.sh | sed s/what/ever/ | tee build.log

is that with the default stdio buffering, while all the commands in 
build.sh will be implicitly self-flushing, the sed in the middle will end 
up batching its output into blocks, so tee will also repeat them in 
blocks.

However, if stdbuf's magic env vars are exported in your shell (either by 
doing a trick like 'export $(env -i stdbuf -oL env)', or else more simply 
by first starting a new shell with 'stdbuf -oL bash'), then every command 
in your pipelines will start with the new default line-buffered stdout. 
That way your line-items from build.sh should get passed all the way 
through the pipeline as they are produced.


(But, proof's in the pudding, so whatever works for you :D )


Happy putting all the way!

Carl