Re: Examples of concurrent coproc usage?

[Chet Ramey <chet.ramey@case.edu>]

[-- Attachment #1.1: Type: text/plain, Size: 3386 bytes --]

On 4/9/24 11:58 AM, Carl Edquist wrote:
> On 4/4/24 7:23 PM, Martin D Kealey wrote:
> 
>> I'm somewhat uneasy about having coprocs inaccessible to each other. I 
>> can foresee reasonable cases where I'd want a coproc to utilize one or 
>> more other coprocs.
>>
>> In particular, I can see cases where a coproc is written to by one 
>> process, and read from by another.
>>
>> Can we at least have the auto-close behaviour be made optional, so that 
>> it can be turned off when we want to do something more sophisticated?
> 
> With support for multiple coprocs, auto-closing the fds to other coprocs 
> when creating new ones is important in order to avoid deadlocks.
> 
> But if you're willing to take on management of those coproc fds yourself, 
> you can expose them to new coprocs by making your own copies with exec 
> redirections.
> 
> But this only "kind of" works, because for some reason bash seems to close 
> all pipe fds for external commands in coprocs, even the ones that the user 
> explicitly copies with exec redirections.



> 
> (More on that in a bit.)
> 
> 
> On Mon, 8 Apr 2024, Chet Ramey wrote:
> 
>> On 4/4/24 7:23 PM, Martin D Kealey wrote:
>>> I'm somewhat uneasy about having coprocs inaccessible to each other. I 
>>> can foresee reasonable cases where I'd want a coproc to utilize one or 
>>> more other coprocs.

>>
>> That's not the intended purpose,

The original intent was to allow the shell to drive a long-running process
that ran more-or-less in parallel with it. Look at examples/scripts/bcalc
for an example of that kind of use.

> 
> For what it's worth, my experience is that coprocesses in bash (rigged up 
> by means other than the coproc keyword) become very fun and interesting 
> when you allow for the possibility of communication between coprocesses. 
> (Most of my use cases for coprocesses fall under this category, actually.)

Sure, as long as you're willing to take on file descriptor management
yourself. I just don't want to make it a new requirement, since it's never
been one before.

> Now, if you built bash with multiple coproc support, I would have expected 
> you could still rig this up, by doing the redirection work explicitly 
> yourself.  Something like this:
> 
>      coproc UP   { stdbuf -oL tr a-z A-Z; }
>      coproc DOWN { stdbuf -oL tr A-Z a-z; }
> 
>      # make user-managed backup copies of coproc fds
>      exec {up_r}<&${UP[0]} {up_w}>&${UP[1]}
>      exec {down_r}<&${DOWN[0]} {down_w}>&${DOWN[1]}
> 
>      coproc THREEWAY { tee /dev/fd/$up_w  /dev/fd/$down_w; }
> 
> 
> But the above doesn't actually work, as it seems that the coproc shell 
> (THREEWAY) closes specifically all the pipe fds (beyond 0,1,2), even the 
> user-managed ones explicitly copied with exec.

File descriptors the user saves with exec redirections beyond [0-2]
are set to close-on-exec. POSIX makes that behavior unspecified, but
bash has always done it.

Shells don't offer any standard way to modify the state of that flag,
but there is the `fdflags' loadable builtin you can experiment with
to change close-on-exec.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]