* [ruby-core:90226] [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed
[not found] <redmine.issue-15367.20181202164852@ruby-lang.org>
@ 2018-12-02 16:48 ` melentievm
2018-12-02 17:09 ` [ruby-core:90228] " Eric Wong
2018-12-02 18:35 ` [ruby-core:90229] " shevegen
` (2 subsequent siblings)
3 siblings, 1 reply; 7+ messages in thread
From: melentievm @ 2018-12-02 16:48 UTC (permalink / raw)
To: ruby-core
Issue #15367 has been reported by printercu (Max Melentiev).
----------------------------------------
Bug #15367: IO.select is not resumed when io-object gets closed
https://bugs.ruby-lang.org/issues/15367
* Author: printercu (Max Melentiev)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v:
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Here is sample code:
~~~ ruby
rp, wp = IO.pipe
t2 = Thread.new { IO.select([rp]) }
# This also does not work:
# t2 = Thread.new { IO.select([rp], nil, [rp]) }
sleep 0.01
rp.close
t2
# => #<Thread:0x00000000089b6ce0@(pry):51 sleep>
~~~
It happens only on linux, tested with 2.5.1, 2.6.0-preview2. On macOS it gives error, as expected:
~~~
#<Thread:0x00007fab3aebce58@(pry):5 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
1: from (pry):5:in `block in <main>'
(pry):5:in `select': Bad file descriptor (Errno::EBADF)
> t2
=> #<Thread:0x00007fab3aebce58@(pry):5 dead>
~~~
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [ruby-core:90229] [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed
[not found] <redmine.issue-15367.20181202164852@ruby-lang.org>
2018-12-02 16:48 ` [ruby-core:90226] [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed melentievm
@ 2018-12-02 18:35 ` shevegen
2018-12-02 20:08 ` [ruby-core:90231] " Eric Wong
2018-12-02 18:59 ` [ruby-core:90230] " Greg.mpls
2018-12-03 9:42 ` [ruby-core:90255] " melentievm
3 siblings, 1 reply; 7+ messages in thread
From: shevegen @ 2018-12-02 18:35 UTC (permalink / raw)
To: ruby-core
Issue #15367 has been updated by shevegen (Robert A. Heiler).
If I may inquire, how significant would the overhead be?
While I think Akira's comment is perfectly fine on its own, I
feel that if other people notice a different behaviour between
different operating systems then this may surprise them. (I
myself use almost exclusively Linux.)
----------------------------------------
Bug #15367: IO.select is not resumed when io-object gets closed
https://bugs.ruby-lang.org/issues/15367#change-75347
* Author: printercu (Max Melentiev)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v:
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Here is sample code:
~~~ ruby
rp, wp = IO.pipe
t2 = Thread.new { IO.select([rp]) }
# This also does not work:
# t2 = Thread.new { IO.select([rp], nil, [rp]) }
sleep 0.01
rp.close
t2
# => #<Thread:0x00000000089b6ce0@(pry):51 sleep>
~~~
It happens only on linux, tested with 2.5.1, 2.6.0-preview2. On macOS it gives error, as expected:
~~~
#<Thread:0x00007fab3aebce58@(pry):5 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
1: from (pry):5:in `block in <main>'
(pry):5:in `select': Bad file descriptor (Errno::EBADF)
> t2
=> #<Thread:0x00007fab3aebce58@(pry):5 dead>
~~~
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [ruby-core:90231] Re: [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed
2018-12-02 18:35 ` [ruby-core:90229] " shevegen
@ 2018-12-02 20:08 ` Eric Wong
0 siblings, 0 replies; 7+ messages in thread
From: Eric Wong @ 2018-12-02 20:08 UTC (permalink / raw)
To: ruby-core
shevegen@gmail.com wrote:
> If I may inquire, how significant would the overhead be?
I'd have to implement it to know for sure...
Thread::Light [Bug #13618] has a process-wide FD map anyways
(similar to the kernel fdtable) to deal with multiple
threads waiting on different operations on the same FD,
so we could take advantage of that.
IO.select is a heavy operation, already
Right now, the IO#close notification isn't so bad if few
threads are operating on FDs, but it's O(n) where `n' is
the number of threads calling rb_io_blocking_region in parallel,
regardless of FD.
Back to the Thread::Light FD map, the `n' would be reduced
to the number of threads for a certain FD, because there's
a per-FD linked-list (and getting to that linked-list is
just an array lookup, so O(1).
^ permalink raw reply [flat|nested] 7+ messages in thread
* [ruby-core:90230] [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed
[not found] <redmine.issue-15367.20181202164852@ruby-lang.org>
2018-12-02 16:48 ` [ruby-core:90226] [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed melentievm
2018-12-02 18:35 ` [ruby-core:90229] " shevegen
@ 2018-12-02 18:59 ` Greg.mpls
2018-12-03 9:42 ` [ruby-core:90255] " melentievm
3 siblings, 0 replies; 7+ messages in thread
From: Greg.mpls @ 2018-12-02 18:59 UTC (permalink / raw)
To: ruby-core
Issue #15367 has been updated by MSP-Greg (Greg L).
On Windows (MinGW), the thread is also sleeping...
----------------------------------------
Bug #15367: IO.select is not resumed when io-object gets closed
https://bugs.ruby-lang.org/issues/15367#change-75348
* Author: printercu (Max Melentiev)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v:
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Here is sample code:
~~~ ruby
rp, wp = IO.pipe
t2 = Thread.new { IO.select([rp]) }
# This also does not work:
# t2 = Thread.new { IO.select([rp], nil, [rp]) }
sleep 0.01
rp.close
t2
# => #<Thread:0x00000000089b6ce0@(pry):51 sleep>
~~~
It happens only on linux, tested with 2.5.1, 2.6.0-preview2. On macOS it gives error, as expected:
~~~
#<Thread:0x00007fab3aebce58@(pry):5 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
1: from (pry):5:in `block in <main>'
(pry):5:in `select': Bad file descriptor (Errno::EBADF)
> t2
=> #<Thread:0x00007fab3aebce58@(pry):5 dead>
~~~
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [ruby-core:90255] [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed
[not found] <redmine.issue-15367.20181202164852@ruby-lang.org>
` (2 preceding siblings ...)
2018-12-02 18:59 ` [ruby-core:90230] " Greg.mpls
@ 2018-12-03 9:42 ` melentievm
2018-12-03 10:18 ` [ruby-core:90257] " Eric Wong
3 siblings, 1 reply; 7+ messages in thread
From: melentievm @ 2018-12-03 9:42 UTC (permalink / raw)
To: ruby-core
Issue #15367 has been updated by printercu (Max Melentiev).
I'm using `IO.select` with ssl socket as it's suggested in docs https://ruby-doc.org/core-2.5.3/IO.html#method-c-select :
~~~ruby
def read_from_socket
socket.read_nonblock(read_buffer_size)
rescue IO::WaitReadable
IO.select([socket.to_io])
retry
rescue IO::WaitWritable
IO.select(nil, [socket.to_io])
retry
end
~~~
Other code is like this (simplified):
~~~ruby
def run
loop do
data = read_from_socket
process(data)
end
rescue Errno::EBADF, IOError
raise unless @stopped
end
def stop
@stopped = true
socket.close
end
Signal.trap('TERM') { Thread.new { stop } } # thread is just workaround for trap contex
run
# exit
~~~
I can't use just `Thread#kill` to not stop thread while it runs `#process`.
It works fine on macOS because `socket.close` makes read_from_socket fail with Errno::EBADF which is rescued later.
However this does not work on linux and `#run` hangs forever.
Is there a right way for this task to not face multi-thread limitations of `IO.select`?
For now I use workaround:
~~~ruby
SELECT_TIMEOUT = 10
def read_from_socket
socket.read_nonblock(read_buffer_size)
rescue IO::WaitReadable
nil until IO.select([socket.to_io], nil, nil, SELECT_TIMEOUT)
retry
rescue IO::WaitWritable
nil until IO.select(nil, [socket.to_io], nil, SELECT_TIMEOUT)
retry
end
~~~
----------------------------------------
Bug #15367: IO.select is not resumed when io-object gets closed
https://bugs.ruby-lang.org/issues/15367#change-75372
* Author: printercu (Max Melentiev)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v:
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Here is sample code:
~~~ ruby
rp, wp = IO.pipe
t2 = Thread.new { IO.select([rp]) }
# This also does not work:
# t2 = Thread.new { IO.select([rp], nil, [rp]) }
sleep 0.01
rp.close
t2
# => #<Thread:0x00000000089b6ce0@(pry):51 sleep>
~~~
It happens only on linux, tested with 2.5.1, 2.6.0-preview2. On macOS it gives error, as expected:
~~~
#<Thread:0x00007fab3aebce58@(pry):5 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
1: from (pry):5:in `block in <main>'
(pry):5:in `select': Bad file descriptor (Errno::EBADF)
> t2
=> #<Thread:0x00007fab3aebce58@(pry):5 dead>
~~~
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 7+ messages in thread
* [ruby-core:90257] Re: [Ruby trunk Bug#15367] IO.select is not resumed when io-object gets closed
2018-12-03 9:42 ` [ruby-core:90255] " melentievm
@ 2018-12-03 10:18 ` Eric Wong
0 siblings, 0 replies; 7+ messages in thread
From: Eric Wong @ 2018-12-03 10:18 UTC (permalink / raw)
To: ruby-core
melentievm@gmail.com wrote:
> I can't use just `Thread#kill` to not stop thread while it runs `#process`.
> It works fine on macOS because `socket.close` makes read_from_socket fail with Errno::EBADF which is rescued later.
> However this does not work on linux and `#run` hangs forever.
> Is there a right way for this task to not face multi-thread
> limitations of `IO.select`?
Several ways (there may be more, but I'm tired)
a) You can use Thread#kill with Thread.handle_interrupt around
#process, I think...
b) IO.select allows waiting on multiple FDs, so you could
wait on one process-wide pipe which every IO.select
call checks on:
int_pipe = IO.pipe
trap(:TERM) { int_pipe[1].write('.') }
IO.select([@socket, int_pipe[0]])
IO.select([int_pipe[0]], [@socket])
(Btw, you don't need #to_io when calling IO.select,
it already calls #to_io).
c) if you're dealing with Internet traffic, it's unreliable
and there's malicious clients trying to DoS you.
So you're going to need a SELECT_TIMEOUT anyways, I think...
^ permalink raw reply [flat|nested] 7+ messages in thread