* [ruby-core:106707] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler.
@ 2021-12-16 7:58 ioquatix (Samuel Williams)
2021-12-16 8:00 ` [ruby-core:106708] " ioquatix (Samuel Williams)
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: ioquatix (Samuel Williams) @ 2021-12-16 7:58 UTC (permalink / raw)
To: ruby-core
Issue #18411 has been reported by ioquatix (Samuel Williams).
----------------------------------------
Bug #18411: Introduce `Fiber.blocking` for disabling scheduler.
https://bugs.ruby-lang.org/issues/18411
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
When implementing pure-ruby IO scheduler, we may need to invoke some Ruby IO operations without entering the scheduler.
```ruby
def io_write(fiber, io, buffer, length)
offset = 0
while length > 0
# From offset until the end:
chunk = buffer.to_str(offset, length)
case result = io.write_nonblock(chunk, exception: false)
when :wait_readable
self.io_wait(fiber, io, IO::READABLE)
when :wait_writable
self.io_wait(fiber, io, IO::WRITABLE)
else
offset += result
length -= result
end
end
return offset
end
```
There are some cases where even in this code `read_nonblock` can invoke fiber scheduler creating infinite recursion.
Therefore, I propose to introduce `Fiber.blocking{...}` which has almost identical implementation to `Fiber.new(blocking: true) {}.resume`.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ruby-core:106708] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler.
2021-12-16 7:58 [ruby-core:106707] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler ioquatix (Samuel Williams)
@ 2021-12-16 8:00 ` ioquatix (Samuel Williams)
2022-09-21 7:12 ` [ruby-core:109971] [Ruby master Feature#18411] " ko1 (Koichi Sasada)
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: ioquatix (Samuel Williams) @ 2021-12-16 8:00 UTC (permalink / raw)
To: ruby-core
Issue #18411 has been updated by ioquatix (Samuel Williams).
@matz this isn't strictly necessary but makes pure Ruby implementation of IO read/write hooks more efficient. I'm not sure if there is valid case for general user code. Can we consider it for Ruby 3.1?
----------------------------------------
Bug #18411: Introduce `Fiber.blocking` for disabling scheduler.
https://bugs.ruby-lang.org/issues/18411#change-95393
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
When implementing pure-ruby IO scheduler, we may need to invoke some Ruby IO operations without entering the scheduler.
```ruby
def io_write(fiber, io, buffer, length)
offset = 0
while length > 0
# From offset until the end:
chunk = buffer.to_str(offset, length)
case result = io.write_nonblock(chunk, exception: false)
when :wait_readable
self.io_wait(fiber, io, IO::READABLE)
when :wait_writable
self.io_wait(fiber, io, IO::WRITABLE)
else
offset += result
length -= result
end
end
return offset
end
```
There are some cases where even in this code `read_nonblock` can invoke fiber scheduler creating infinite recursion.
Therefore, I propose to introduce `Fiber.blocking{...}` which has almost identical implementation to `Fiber.new(blocking: true) {}.resume`.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ruby-core:109971] [Ruby master Feature#18411] Introduce `Fiber.blocking` for disabling scheduler.
2021-12-16 7:58 [ruby-core:106707] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler ioquatix (Samuel Williams)
2021-12-16 8:00 ` [ruby-core:106708] " ioquatix (Samuel Williams)
@ 2022-09-21 7:12 ` ko1 (Koichi Sasada)
2022-09-21 10:51 ` [ruby-core:109975] " ioquatix (Samuel Williams)
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: ko1 (Koichi Sasada) @ 2022-09-21 7:12 UTC (permalink / raw)
To: ruby-core
Issue #18411 has been updated by ko1 (Koichi Sasada).
Let me confirm that why `write_nonblock` calls a fiber scheduler?
Or does it solve if `write_nonblock` doesn't call a fiber scheduler?
----------------------------------------
Feature #18411: Introduce `Fiber.blocking` for disabling scheduler.
https://bugs.ruby-lang.org/issues/18411#change-99221
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
----------------------------------------
When implementing pure-ruby IO scheduler, we may need to invoke some Ruby IO operations without entering the scheduler.
```ruby
def io_write(fiber, io, buffer, length)
offset = 0
while length > 0
# From offset until the end:
chunk = buffer.to_str(offset, length)
case result = io.write_nonblock(chunk, exception: false)
when :wait_readable
self.io_wait(fiber, io, IO::READABLE)
when :wait_writable
self.io_wait(fiber, io, IO::WRITABLE)
else
offset += result
length -= result
end
end
return offset
end
```
There are some cases where even in this code `read_nonblock` can invoke fiber scheduler creating infinite recursion.
Therefore, I propose to introduce `Fiber.blocking{...}` which has almost identical implementation to `Fiber.new(blocking: true) {}.resume`.
In the above code, we change the line:
```
case result = io.write_nonblock(chunk, exception: false)
```
to
```
case result = Fiber.blocking{io.write_nonblock(chunk, exception: false)}
```
This ensures that `write_nonblock` can never enter the scheduler again.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ruby-core:109975] [Ruby master Feature#18411] Introduce `Fiber.blocking` for disabling scheduler.
2021-12-16 7:58 [ruby-core:106707] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler ioquatix (Samuel Williams)
2021-12-16 8:00 ` [ruby-core:106708] " ioquatix (Samuel Williams)
2022-09-21 7:12 ` [ruby-core:109971] [Ruby master Feature#18411] " ko1 (Koichi Sasada)
@ 2022-09-21 10:51 ` ioquatix (Samuel Williams)
2022-10-06 8:08 ` [ruby-core:110204] " ioquatix (Samuel Williams)
2022-10-08 7:34 ` [ruby-core:110235] " ioquatix (Samuel Williams)
4 siblings, 0 replies; 6+ messages in thread
From: ioquatix (Samuel Williams) @ 2022-09-21 10:51 UTC (permalink / raw)
To: ruby-core
Issue #18411 has been updated by ioquatix (Samuel Williams).
`io_uring` handles non-blocking read and write system calls using the ring buffer. So even `read_nonblock` and `write_nonblock` system calls can go via the fiber scheduler. The system call itself is asynchronous even if the IO is non-blocking. I have not investigated the performance much since `io_uring` has not been in a completely usable state yet. But recently it was much better, so I may begin this evaluation soon.
----------------------------------------
Feature #18411: Introduce `Fiber.blocking` for disabling scheduler.
https://bugs.ruby-lang.org/issues/18411#change-99225
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
----------------------------------------
When implementing pure-ruby IO scheduler, we may need to invoke some Ruby IO operations without entering the scheduler.
```ruby
def io_write(fiber, io, buffer, length)
offset = 0
while length > 0
# From offset until the end:
chunk = buffer.to_str(offset, length)
case result = io.write_nonblock(chunk, exception: false)
when :wait_readable
self.io_wait(fiber, io, IO::READABLE)
when :wait_writable
self.io_wait(fiber, io, IO::WRITABLE)
else
offset += result
length -= result
end
end
return offset
end
```
There are some cases where even in this code `read_nonblock` can invoke fiber scheduler creating infinite recursion.
Therefore, I propose to introduce `Fiber.blocking{...}` which has almost identical implementation to `Fiber.new(blocking: true) {}.resume`.
In the above code, we change the line:
```
case result = io.write_nonblock(chunk, exception: false)
```
to
```
case result = Fiber.blocking{io.write_nonblock(chunk, exception: false)}
```
This ensures that `write_nonblock` can never enter the scheduler again.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ruby-core:110204] [Ruby master Feature#18411] Introduce `Fiber.blocking` for disabling scheduler.
2021-12-16 7:58 [ruby-core:106707] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler ioquatix (Samuel Williams)
` (2 preceding siblings ...)
2022-09-21 10:51 ` [ruby-core:109975] " ioquatix (Samuel Williams)
@ 2022-10-06 8:08 ` ioquatix (Samuel Williams)
2022-10-08 7:34 ` [ruby-core:110235] " ioquatix (Samuel Williams)
4 siblings, 0 replies; 6+ messages in thread
From: ioquatix (Samuel Williams) @ 2022-10-06 8:08 UTC (permalink / raw)
To: ruby-core
Issue #18411 has been updated by ioquatix (Samuel Williams).
It was accepted.
PR: https://github.com/ruby/ruby/pull/6498
----------------------------------------
Feature #18411: Introduce `Fiber.blocking` for disabling scheduler.
https://bugs.ruby-lang.org/issues/18411#change-99486
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
----------------------------------------
When implementing pure-ruby IO scheduler, we may need to invoke some Ruby IO operations without entering the scheduler.
```ruby
def io_write(fiber, io, buffer, length)
offset = 0
while length > 0
# From offset until the end:
chunk = buffer.to_str(offset, length)
case result = io.write_nonblock(chunk, exception: false)
when :wait_readable
self.io_wait(fiber, io, IO::READABLE)
when :wait_writable
self.io_wait(fiber, io, IO::WRITABLE)
else
offset += result
length -= result
end
end
return offset
end
```
There are some cases where even in this code `read_nonblock` can invoke fiber scheduler creating infinite recursion.
Therefore, I propose to introduce `Fiber.blocking{...}` which has almost identical implementation to `Fiber.new(blocking: true) {}.resume`.
In the above code, we change the line:
```
case result = io.write_nonblock(chunk, exception: false)
```
to
```
case result = Fiber.blocking{io.write_nonblock(chunk, exception: false)}
```
This ensures that `write_nonblock` can never enter the scheduler again.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 6+ messages in thread
* [ruby-core:110235] [Ruby master Feature#18411] Introduce `Fiber.blocking` for disabling scheduler.
2021-12-16 7:58 [ruby-core:106707] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler ioquatix (Samuel Williams)
` (3 preceding siblings ...)
2022-10-06 8:08 ` [ruby-core:110204] " ioquatix (Samuel Williams)
@ 2022-10-08 7:34 ` ioquatix (Samuel Williams)
4 siblings, 0 replies; 6+ messages in thread
From: ioquatix (Samuel Williams) @ 2022-10-08 7:34 UTC (permalink / raw)
To: ruby-core
Issue #18411 has been updated by ioquatix (Samuel Williams).
Status changed from Open to Closed
Assignee set to ioquatix (Samuel Williams)
It was merged.
----------------------------------------
Feature #18411: Introduce `Fiber.blocking` for disabling scheduler.
https://bugs.ruby-lang.org/issues/18411#change-99519
* Author: ioquatix (Samuel Williams)
* Status: Closed
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
When implementing pure-ruby IO scheduler, we may need to invoke some Ruby IO operations without entering the scheduler.
```ruby
def io_write(fiber, io, buffer, length)
offset = 0
while length > 0
# From offset until the end:
chunk = buffer.to_str(offset, length)
case result = io.write_nonblock(chunk, exception: false)
when :wait_readable
self.io_wait(fiber, io, IO::READABLE)
when :wait_writable
self.io_wait(fiber, io, IO::WRITABLE)
else
offset += result
length -= result
end
end
return offset
end
```
There are some cases where even in this code `read_nonblock` can invoke fiber scheduler creating infinite recursion.
Therefore, I propose to introduce `Fiber.blocking{...}` which has almost identical implementation to `Fiber.new(blocking: true) {}.resume`.
In the above code, we change the line:
```
case result = io.write_nonblock(chunk, exception: false)
```
to
```
case result = Fiber.blocking{io.write_nonblock(chunk, exception: false)}
```
This ensures that `write_nonblock` can never enter the scheduler again.
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-10-08 7:34 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-16 7:58 [ruby-core:106707] [Ruby master Bug#18411] Introduce `Fiber.blocking` for disabling scheduler ioquatix (Samuel Williams)
2021-12-16 8:00 ` [ruby-core:106708] " ioquatix (Samuel Williams)
2022-09-21 7:12 ` [ruby-core:109971] [Ruby master Feature#18411] " ko1 (Koichi Sasada)
2022-09-21 10:51 ` [ruby-core:109975] " ioquatix (Samuel Williams)
2022-10-06 8:08 ` [ruby-core:110204] " ioquatix (Samuel Williams)
2022-10-08 7:34 ` [ruby-core:110235] " ioquatix (Samuel Williams)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).