Re: Examples of concurrent coproc usage?

[Zachary Santer <zsanter@gmail.com>]

On Thu, Mar 14, 2024 at 6:57 AM Carl Edquist <edquist@cs.wisc.edu> wrote:

> (And in general, latter coproc shells will have stray copies of the user
> shell's r/w ends from all previous coprocs.)

I didn't know that without MULTIPLE_COPROCS=1, bash wouldn't even
attempt to keep the fds from earlier coprocs out of later coprocs.

> Unexpectedly, the new multi-coproc code seems to close the user shell's
> end of a coprocess's pipes, once the coprocess has terminated.  When
> compiled with MULTIPLE_COPROCS=1, this is true even if there is only a
> single coproc:

> This is a bug.  The shell should not automatically close its read pipe to
> a coprocess that has terminated -- it should stay open to read the final
> output, and the user should be responsible for closing the read end
> explicitly.

> It also invites trouble if the shell variable that holds the fds gets
> removed unexpectedly when the coprocess terminates.  (Suddenly the
> variable expands to an empty string.)  It seems to me that the proper time
> to clear the coproc variable (if at all) is after the user has explicitly
> closed both of the fds.  *Or* else add an option to the coproc keyword to
> explicitly close the coproc - which will close both fds and clear the
> variable.

I agree. This was the discussion in [1], where it sounds like this was
the intended behavior. The array that bash originally created to store
the coproc fds is removed immediately, but the fds are evidently
closed at some later, indeterminate point. So, if you store the coproc
fds in a different array than the one bash gave you, you might still
be able to read from the read fd for a little while. That sounds
suspiciously like a race condition, though. The behavior without
MULTIPLE_COPROCS=1 might have changed since that discussion.

> That's a nice trick with the shell backgrounding all the coprocesses
> before connecting the fifos.  But yeah, to make subsequent coprocesses you
> do still have to close the copy of the user shell's fds that the coprocess
> shell inherits.  It sounds like you are doing that (nice!), but in any
> case it requires some care, and as these stack up it is really handy to
> have something manage it all for you.

I absolutely learned more about what I was doing from that
conversation with Chet three years ago.

> (Perhaps this is where I ask if you are happy with your solution or if you
> would like to try out something wildly more flexible...)

Admittedly, I am very curious to see your bash coprocess management
library. I don't know how you could implement coprocesses outside of
bash's coproc keyword without using FIFOs somehow.

> Happy coprocessing! :)

Thanks for your detailed description of all this.

[1] https://lists.gnu.org/archive/html/help-bash/2021-04/msg00136.html