[ruby-core:101250] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[ruby-core:101250] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[samuel]

Issue #17369 has been reported by ioquatix (Samuel Williams).

----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3852

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:101252] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[eregontp]

Issue #17369 has been updated by Eregon (Benoit Daloze).


Does such code still work, with a scheduler?
```ruby
`echo foo`
p $? # => #<Process::Status: pid 43525 exit 0>
```

If not, it seems a significant problem, as existing would break with a scheduler.

Given the implementation in the test scheduler:
```ruby
  def process_wait(pid, flags)
    # This is a very simple way to implement a non-blocking wait:
    Thread.new do
      Process::Status.wait(pid, flags)
    end.join
  end
```
It sounds like you would need a way to set `$?` on the current Thread.
So that `$?` can be set for the caller.
I think that's fine to add.

I think `$?` should be Fiber-local, probably it's thread-local only for historic reasons.
Otherwise, just switching between Fibers (e.g., on IO) would expose the `$?` of other Fibers, which will lead to bugs.
I expect that change to cause extremely few compatibility issues (`$~`, etc are already fiber-local + frame-local).

----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-88929

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3852

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:101253] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[samuel]

Issue #17369 has been updated by ioquatix (Samuel Williams).


> Does such code still work, with a scheduler?

Yes.

> It sounds like you would need a way to set $? on the current Thread.

Nope, it's handled by `Process.wait` and so on.

> Otherwise, just switching between Fibers (e.g., on IO) would expose the $? of other Fibers, which will lead to bugs.

Agree, but we can't change this without potentially breaking existing code.

Also, is it okay that `Process.last_status` and `Process.last_status=` (hypothetical) are fiber local? Because Matz already said he was against class attributes that are actually fiber local (even if I agree in theory, excluding the fact that this is a breaking change).

> I expect that change to cause extremely few compatibility issues ($~, etc are already fiber-local + frame-local).

Great, if Matz can approve the change, then we can implement it, but it's separate from this PR, since this PR just makes the existing interface non-blocking.

----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-88930

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3852

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:101254] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[eregontp]

Issue #17369 has been updated by Eregon (Benoit Daloze).


I clarified with @ioquatix, the code above should be `    end.value` so it returns the Process::Status and `system` still sets it.
Then the change sounds good to me.

----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-88931

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3852

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:101321] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[samuel]

Issue #17369 has been updated by ioquatix (Samuel Williams).


Non-blocking `Process.wait` has been merged.

----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-89015

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:101335] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[naruse]

Issue #17369 has been updated by naruse (Yui NARUSE).


Is this feature discussed with ko1 and nobu?
Also I suspect Matz's approval is required for this change.

----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-89032

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:101359] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[matz]

Issue #17369 has been updated by matz (Yukihiro Matsumoto).


I am OK with `Process::Status.wait`. As far as I've heard the code quality needs upgrade.

Matz.


----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-89066

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:101703] [Ruby master Bug#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[samuel]

Issue #17369 has been updated by ioquatix (Samuel Williams).


We introduced experimental feature and implemented non-blocking hook for Ruby 3.

More work is required here, but we didn't make it in time for Ruby 3.0 - so we marked it as experimental.

We also need to implement `rb_f_system` in terms of `rb_process_status_wait`. Can someone else help with this?

----------------------------------------
Bug #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-89529

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:104379] [Ruby master Feature#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[samuel]

Issue #17369 has been updated by ioquatix (Samuel Williams).


See https://github.com/ruby/ruby/pull/4595 which implements non-blocking `Kernel#system`.

----------------------------------------
Feature #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-92616

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:105435] [Ruby master Feature#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[ioquatix (Samuel Williams)]

Issue #17369 has been updated by ioquatix (Samuel Williams).


The implementation is completed.

However, some parts are still pretty messy, including leaking `$?` process status.

We need to make `$?` fiber local.

However this might cause some issues in existing code.

Should we consider to deprecate `$?`?

----------------------------------------
Feature #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-93882

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:105617] [Ruby master Feature#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[Eregon (Benoit Daloze)]

Issue #17369 has been updated by Eregon (Benoit Daloze).


I think making `$?` Fiber-local makes sense, and unlikely to break anything.
I don't see the need to deprecate `$?`, and it's certainly not worth the cost to migrate existing to some other way to the get the status.

----------------------------------------
Feature #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-94105

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/

[ruby-core:105717] [Ruby master Feature#17369] Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.

[nobu (Nobuyoshi Nakada)]

Issue #17369 has been updated by nobu (Nobuyoshi Nakada).


Please file a new issue for fiber-local `$?`.

----------------------------------------
Feature #17369: Introduce non-blocking `Process.wait`, `Kernel.system` and related methods.
https://bugs.ruby-lang.org/issues/17369#change-94216

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
https://github.com/ruby/ruby/pull/3853

This PR introduces optional hooks to the scheduler interface for handling `Process.wait`, `Kernel.system` and other related methods (`waitpid`, `wait2`, etc).

It funnels all methods through a new interface `Process::Status.wait` which is almost identical to `Process.wait` except for several key differences:

- The return value is a single instance of `Process::Status`.
- It does not set thread local `$?`.

This is necessary for keeping the scheduler interface simple (and side effects are generally bad anyway).



-- 
https://bugs.ruby-lang.org/