Examples of concurrent coproc usage?

[Zachary Santer <zsanter@gmail.com>]

Was "RFE: enable buffering on null-terminated data"

On Mon, Mar 11, 2024 at 7:54 AM Carl Edquist <edquist@cs.wisc.edu> wrote:
>
> On Sun, 10 Mar 2024, Zachary Santer wrote:
>
> > On Sun, Mar 10, 2024 at 4:36 PM Carl Edquist <edquist@cs.wisc.edu> wrote:
> >>
> >> Out of curiosity, do you have an example command line for your use case?
> >
> > My use for 'stdbuf --output=L' is to be able to run a command within a
> > bash coprocess.
>
> Oh, cool, now you're talking!  ;)
>
>
> > (Really, a background process communicating with the parent process
> > through FIFOs, since Bash prints a warning message if you try to run
> > more than one coprocess at a time. Shouldn't make a difference here.)
>
> (Kind of a side-note ... bash's limited coprocess handling was a long
> standing annoyance for me in the past, to the point that I wrote a bash
> coprocess management library to handle multiple active coprocess and give
> convenient methods for interaction.  Perhaps the trickiest bit about
> multiple coprocesses open at once (which I suspect is the reason support
> was never added to bash) is that you don't want the second and subsequent
> coprocesses to inherit the pipe fds of prior open coprocesses.  This can
> result in deadlock if, for instance, you close your write end to coproc1,
> but coproc1 continues to wait for input because coproc2 also has a copy of
> a write end of the pipe to coproc1's input.  So you need to be smart about
> subsequent coprocesses first closing all fds associated with other
> coprocesses.

https://lists.gnu.org/archive/html/help-bash/2021-03/msg00296.html
https://lists.gnu.org/archive/html/help-bash/2021-04/msg00136.html

You're on the money, though there is a preprocessor directive you can
build bash with that will allow it to handle multiple concurrent
coprocesses without complaining: MULTIPLE_COPROCS=1. Chet Ramey's
sticking point was that he hadn't seen coprocesses used enough in the
wild to satisfactorily test that his implementation did in fact keep
the coproc file descriptors out of subshells. If you've got examples
you can direct him to, I'd really appreciate it.

> Word to the wise: you might encounter this issue (coproc2 prevents coproc1
> from seeing its end-of-input) even though you are rigging this up yourself
> with FIFOs rather than bash's coproc builtin.)

In my case, it's mostly a non-issue, because I fork the - now three -
background processes before exec'ing automatic fds redirecting to/from
their FIFO's in the parent process. All the automatic fds get put in
an array, and I do close them all at the beginning of a subsequent
process substitution.