Re: [PATCH] mingw: handle writes to non-blocking pipe

["René Scharfe" <l.s.r@web.de>]

Am 11.08.2022 um 10:47 schrieb Jeff King:
> On Thu, Aug 11, 2022 at 12:34:46AM +0200, René Scharfe wrote:
>
>>> OK, so we call GetNamedPipeInfo() to find the size of the pipe buffer.
>>> It's unclear to me from Microsoft's docs if that is the _total_ size, or
>>> if it's the remaining available size. Hopefully the latter, since none
>>> of this works otherwise. ;)
>>>
>>> But two corner cases:
>>>
>>>   - If we fail to get the size, we guess that it's the maximum. Is this
>>>     dangerous? I'm not sure why the call would fail, but if for some
>>>     reason it did fail and we can't make forward progress, would we
>>>     enter an infinite recursion of mingw_write()? Would it be safer to
>>>     bail with EAGAIN in such a case (through granted, that probably just
>>>     puts us into an infinite loop in xwrite())?
>>
>> AFAIU it's the total size, not the available space.  I think I confused
>> it with PIPE_BUF, which we should use instead.
>
> Hmm. If it's giving us the total size, that seems like it may fail in a
> case like this:
>
>   - we write a smaller amount to the pipe, say 7168 bytes. That leaves
>     1024 bytes in the buffer. The reader reads nothing yet.
>
>   - now we try to write another 4096 bytes. That's too big, so we get
>     ENOSPC. But when we ask for the pipe size, it tells us 8192. So we
>     _don't_ try to do a partial write of the remaining size, and return
>     EAGAIN. But now we've made no forward progress (i.e., even though
>     poll() said we could write, we don't, and we end up in the xwrite
>     loop).
>
> So we really do want to try to get a partial write. Even a single byte
> means we are making forward progress.
>
>> Alternatively we could retry with ever smaller sizes, down to one byte,
>> to avoid EAGAIN as much as possible.  Sounds costly, though.
>
> It's definitely not optimal, but it may not be too bad. If we cut the
> size in half each time, then at worst we make log2(N) extra write
> attempts, and we end up with a partial write within 50% of the optimal
> size.

OK, but we can't just split anything up as we like: POSIX wants us to
keep writes up to PIPE_BUF atomic.  When I read that name I mistakenly
thought it was the size of the pipe buffer, but it's a different value.
The minimum value according to POSIX is 512 bytes; on Linux it's 4KB.

And Windows doesn't seem to define it.  Bash's ulimit -p returns 8,
which is in units of 512 bytes, so it's 4KB like on Linux.  But that's
apparently not respected by Windows' write.

I just realized that we can interprete the situation slightly
differently.  Windows has effectively PIPE_BUF = 2^32, which means all
writes are atomic.  I don't see POSIX specifying a maximum allowed
value, so this must be allowed.  Which means you cannot rely on write
performing partial writes.  Makes sense?

If we accept that, then we need a special write function for
non-blocking pipes that chops the data into small enough chunks.

Another oddity: t3701 with yesterday's patch finishes fine with -i, but
hangs without it (not just when running it via prove).  O_o

René