[ruby-core:81581] [Ruby trunk Bug#13632] Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

[ruby-core:81581] [Ruby trunk Bug#13632] Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

[sir.nickolas]

Issue #13632 has been reported by nvashchenko (Nikolay Vashchenko).

----------------------------------------
Bug #13632: Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.
https://bugs.ruby-lang.org/issues/13632

* Author: nvashchenko (Nikolay Vashchenko)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.2.x, 2.3.x, 2.4.x, 2.5.x
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
In the bugfix for https://bugs.ruby-lang.org/issues/13076 has been introduced another bug, caused by a busy waiting in rb_notify_fd_close method, while the FD is being released. During this waiting, it pumps huge amounts of the ruby_error_stream_closed errors into thread's interrupt queue, which almost all stay there unprocessed. It can be up for several hundred of exceptions in the queue, depending on circumstances.
~~~ ruby
  a = []
  t = []
  10.times do
    r,w = IO.pipe
    a << [r,w]
    t << Thread.new do
      while r.gets
      end rescue IOError
      # Interrupt queue is full and all IO is broken
     Thread.current.pending_interrupt? # Expected to be false, because it's already rescued
      IO.sysopen ("/dev/tty")                   # Expected not to throw an error. On each such call, it dequeues the next item from interrupt queue until there's none
    end
  end
  a.each do |r,w|
    w.puts "test"
    w.close
    r.close
  end
  t.each do |th|
    th.join
  end
~~~
Output:
~~~ text
Traceback (most recent call last):
	1: from test2.rb:9:in `block (2 levels) in <main>'
test2.rb:9:in `sysopen': stream closed in another thread (IOError)
~~~
 







-- 
https://bugs.ruby-lang.org/

[ruby-core:81584] Re: [Ruby trunk Bug#13632] Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

[Eric Wong]

sir.nickolas@gmail.com wrote:
> Bug #13632: Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.
> https://bugs.ruby-lang.org/issues/13632

Investigating.  I've been looking at IO close notification for
[Feature #13618] (auto-Fiber) anyways.

[ruby-core:81585] Re: [Ruby trunk Bug#13632] Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

[Eric Wong]

sir.nickolas@gmail.com wrote:
> https://bugs.ruby-lang.org/issues/13632

r59020 should fix it trivially in trunk.

Backporting to <= 2.4 is only a little different due to the
data structure change:

	s/wfd->fd = -1/th->waiting_fd = -1/
	https://80x24.org/spew/20170606001646.22889-1-e@80x24.org/raw

Thanks for the report!

[ruby-core:81613] Re: [Ruby trunk Bug#13632] Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> sir.nickolas@gmail.com wrote:
> > https://bugs.ruby-lang.org/issues/13632
> 
> r59020 should fix it trivially in trunk.

Make that r59028 :x  r59020 interacted badly with r57422

> Backporting to <= 2.4 is only a little different due to the
> data structure change:

r59028 backporting is more difficult.

Below are links to backported patches from trunk to fix [Bug #13632] for
Ruby 2.4 maintenance branches and earlier.   I mainly wanted to
backport r59028, but that depends on r58812 (originally intended
as a pure performance optimization).

I'd rather not change r59028 to something unrecognizable from
what is in trunk for backporting, as that might negatively
impact future backports.

r58812: speed up IO#close with many threads
   https://80x24.org/spew/20170607195901.18958-2-e@80x24.org/raw
r59028: IO#close: do not enqueue redundant interrupts (take #2)
   https://80x24.org/spew/20170607195901.18958-3-e@80x24.org/raw

 test/ruby/test_io.rb | 22 ++++++++++++++++++++++
 thread.c             | 33 ++++++++++++++++++++++++---------
 vm.c                 |  1 -
 vm_core.h            |  4 ++--
 4 files changed, 48 insertions(+), 12 deletions(-)

[ruby-core:81966] [Ruby trunk Bug#13632] Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

[nagachika00]

Issue #13632 has been updated by nagachika (Tomoyuki Chikanaga).

Backport changed from 2.2: UNKNOWN, 2.3: DONE, 2.4: REQUIRED to 2.2: UNKNOWN, 2.3: DONE, 2.4: DONE

ruby_2_4 r59286 merged revision(s) 58284,58812,59028.

----------------------------------------
Bug #13632: Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.
https://bugs.ruby-lang.org/issues/13632#change-65689

* Author: nvashchenko (Nikolay Vashchenko)
* Status: Closed
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.2.x, 2.3.x, 2.4.x, 2.5.x
* Backport: 2.2: UNKNOWN, 2.3: DONE, 2.4: DONE
----------------------------------------
In the bugfix for https://bugs.ruby-lang.org/issues/13076 has been introduced another bug, caused by a busy waiting in rb_notify_fd_close method, while the FD is being released. During this waiting, it pumps huge amounts of the ruby_error_stream_closed errors into thread's interrupt queue, which almost all stay there unprocessed. It can be up for several hundred of exceptions in the queue, depending on circumstances.

```
  a = []
  t = []
  10.times do
    r,w = IO.pipe
    a << [r,w]
    t << Thread.new do
      while r.gets
      end rescue IOError
      # Interrupt queue is full and all IO is broken
     Thread.current.pending_interrupt? # Expected to be false, because it's already rescued
      IO.sysopen ("/dev/tty")                   # Expected not to throw an error. On each such call, it dequeues the next item from interrupt queue until there's none
    end
  end
  a.each do |r,w|
    w.puts "test"
    w.close
    r.close
  end
  t.each do |th|
    th.join
  end
```

Output:

``` text
Traceback (most recent call last):
	1: from test2.rb:9:in `block (2 levels) in <main>'
test2.rb:9:in `sysopen': stream closed in another thread (IOError)
```
 







-- 
https://bugs.ruby-lang.org/

[ruby-core:82085] [Ruby trunk Bug#13632] Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

[sir.nickolas]

Issue #13632 has been updated by nvashchenko (Nikolay Vashchenko).


I created a gem with a temporary workaround for versions 2.2.7, 2.3.4 and 2.4.1 to help folks until versions with backports are released:
https://rubygems.org/gems/stopgap_13632

----------------------------------------
Bug #13632: Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.
https://bugs.ruby-lang.org/issues/13632#change-65816

* Author: nvashchenko (Nikolay Vashchenko)
* Status: Closed
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.2.x, 2.3.x, 2.4.x, 2.5.x
* Backport: 2.2: UNKNOWN, 2.3: DONE, 2.4: DONE
----------------------------------------
In the bugfix for https://bugs.ruby-lang.org/issues/13076 has been introduced another bug, caused by a busy waiting in rb_notify_fd_close method, while the FD is being released. During this waiting, it pumps huge amounts of the ruby_error_stream_closed errors into thread's interrupt queue, which almost all stay there unprocessed. It can be up for several hundred of exceptions in the queue, depending on circumstances.

```
  a = []
  t = []
  10.times do
    r,w = IO.pipe
    a << [r,w]
    t << Thread.new do
      while r.gets
      end rescue IOError
      # Interrupt queue is full and all IO is broken
     Thread.current.pending_interrupt? # Expected to be false, because it's already rescued
      IO.sysopen ("/dev/tty")                   # Expected not to throw an error. On each such call, it dequeues the next item from interrupt queue until there's none
    end
  end
  a.each do |r,w|
    w.puts "test"
    w.close
    r.close
  end
  t.each do |th|
    th.join
  end
```

Output:

``` text
Traceback (most recent call last):
	1: from test2.rb:9:in `block (2 levels) in <main>'
test2.rb:9:in `sysopen': stream closed in another thread (IOError)
```
 







-- 
https://bugs.ruby-lang.org/