[ruby-core:69955] [Ruby trunk - Feature #11348] [Open] TracePoint API needs events for fiber's switching

[ruby-core:69955] [Ruby trunk - Feature #11348] [Open] TracePoint API needs events for fiber's switching

[os97673]

Issue #11348 has been reported by Oleg Sukhodolsky.

----------------------------------------
Feature #11348: TracePoint API needs events for fiber's switching
https://bugs.ruby-lang.org/issues/11348

* Author: Oleg Sukhodolsky
* Status: Open
* Priority: Normal
* Assignee: 
----------------------------------------
as discussed in https://github.com/deivid-rodriguez/byebug/issues/153 current implementation of byebug/debase has problem with stepping when Enumarator/Fiber is used.  The problem is that Fiber completely replaces stack (w/o any events) while the gems assume that any stack modification has corresponding call/return event.

It would be nice to receive events for Fiber's switching (at least) to be able to track stack for every fiber independently.
Also I think it would be nice to discuss the API before adding it because different set of event will allow to implement different debugger's capabilities.
E.g. plain :fiber-changed (event about every fiber switch) will only allow us to handle every fiber independently (like threads are handled) while debugger's user may expect that step into on enumerator.next will step into enumerator's code and step out from that code should go to the place where enumerator.next has been called.
So, perhaps we should have different events for Fiber#resume and Fiber#yield, but I'm not sure and would like to here other opinions.

Here are two tests which can be used to play with (one for Enumerator, another for Fiber)

~~~ruby
triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " "

5.times do
  print triangular_numbers.next, " "
end
~~~

~~~ruby
fiber = Fiber.new do
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    Fiber.yield number
  end
end

print fiber.resume, " "

5.times do
  print fiber.resume, " "
end
~~~



-- 
https://bugs.ruby-lang.org/

[ruby-core:70289] [Ruby trunk - Feature #11348] [Assigned] TracePoint API needs events for fiber's switching

[nagachika00]

Issue #11348 has been updated by Tomoyuki Chikanaga.

Status changed from Open to Assigned
Assignee set to Koichi Sasada

----------------------------------------
Feature #11348: TracePoint API needs events for fiber's switching
https://bugs.ruby-lang.org/issues/11348#change-53711

* Author: Oleg Sukhodolsky
* Status: Assigned
* Priority: Normal
* Assignee: Koichi Sasada
----------------------------------------
as discussed in https://github.com/deivid-rodriguez/byebug/issues/153 current implementation of byebug/debase has problem with stepping when Enumarator/Fiber is used.  The problem is that Fiber completely replaces stack (w/o any events) while the gems assume that any stack modification has corresponding call/return event.

It would be nice to receive events for Fiber's switching (at least) to be able to track stack for every fiber independently.
Also I think it would be nice to discuss the API before adding it because different set of event will allow to implement different debugger's capabilities.
E.g. plain :fiber-changed (event about every fiber switch) will only allow us to handle every fiber independently (like threads are handled) while debugger's user may expect that step into on enumerator.next will step into enumerator's code and step out from that code should go to the place where enumerator.next has been called.
So, perhaps we should have different events for Fiber#resume and Fiber#yield, but I'm not sure and would like to here other opinions.

Here are two tests which can be used to play with (one for Enumerator, another for Fiber)

~~~ruby
triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " "

5.times do
  print triangular_numbers.next, " "
end
~~~

~~~ruby
fiber = Fiber.new do
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    Fiber.yield number
  end
end

print fiber.resume, " "

5.times do
  print fiber.resume, " "
end
~~~



-- 
https://bugs.ruby-lang.org/

[ruby-core:70506] [Ruby trunk - Feature #11348] [Feedback] TracePoint API needs events for fiber's switching

[ko1]

Issue #11348 has been updated by Koichi Sasada.

Status changed from Closed to Feedback

I agree to add 'switching' events so I try to commit it.

Could you try it and could you tell me "what" is not enough?

(Actually, I don't increase the number of events)

For example, how about to provide a method to know resume chain like that?

```ruby
Fiber.new{ # f1
  Fiber.new{ # f2
    Fiber.nesting #=> [root_fiber, f1, f2]
  }
}
```

(not sure nesting is good name or not)


----------------------------------------
Feature #11348: TracePoint API needs events for fiber's switching
https://bugs.ruby-lang.org/issues/11348#change-53906

* Author: Oleg Sukhodolsky
* Status: Feedback
* Priority: Normal
* Assignee: Koichi Sasada
----------------------------------------
as discussed in https://github.com/deivid-rodriguez/byebug/issues/153 current implementation of byebug/debase has problem with stepping when Enumarator/Fiber is used.  The problem is that Fiber completely replaces stack (w/o any events) while the gems assume that any stack modification has corresponding call/return event.

It would be nice to receive events for Fiber's switching (at least) to be able to track stack for every fiber independently.
Also I think it would be nice to discuss the API before adding it because different set of event will allow to implement different debugger's capabilities.
E.g. plain :fiber-changed (event about every fiber switch) will only allow us to handle every fiber independently (like threads are handled) while debugger's user may expect that step into on enumerator.next will step into enumerator's code and step out from that code should go to the place where enumerator.next has been called.
So, perhaps we should have different events for Fiber#resume and Fiber#yield, but I'm not sure and would like to here other opinions.

Here are two tests which can be used to play with (one for Enumerator, another for Fiber)

~~~ruby
triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " "

5.times do
  print triangular_numbers.next, " "
end
~~~

~~~ruby
fiber = Fiber.new do
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    Fiber.yield number
  end
end

print fiber.resume, " "

5.times do
  print fiber.resume, " "
end
~~~



-- 
https://bugs.ruby-lang.org/

[ruby-core:70514] [Ruby trunk - Feature #11348] TracePoint API needs events for fiber's switching

[os97673]

Issue #11348 has been updated by Oleg Sukhodolsky.


Koichi Sasada wrote:
> I agree to add 'switching' events so I try to commit it.
> 
> Could you try it and could you tell me "what" is not enough?

I'm on vacation right now will play with ruby-head early next week.

----------------------------------------
Feature #11348: TracePoint API needs events for fiber's switching
https://bugs.ruby-lang.org/issues/11348#change-53916

* Author: Oleg Sukhodolsky
* Status: Feedback
* Priority: Normal
* Assignee: Koichi Sasada
----------------------------------------
as discussed in https://github.com/deivid-rodriguez/byebug/issues/153 current implementation of byebug/debase has problem with stepping when Enumarator/Fiber is used.  The problem is that Fiber completely replaces stack (w/o any events) while the gems assume that any stack modification has corresponding call/return event.

It would be nice to receive events for Fiber's switching (at least) to be able to track stack for every fiber independently.
Also I think it would be nice to discuss the API before adding it because different set of event will allow to implement different debugger's capabilities.
E.g. plain :fiber-changed (event about every fiber switch) will only allow us to handle every fiber independently (like threads are handled) while debugger's user may expect that step into on enumerator.next will step into enumerator's code and step out from that code should go to the place where enumerator.next has been called.
So, perhaps we should have different events for Fiber#resume and Fiber#yield, but I'm not sure and would like to here other opinions.

Here are two tests which can be used to play with (one for Enumerator, another for Fiber)

~~~ruby
triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " "

5.times do
  print triangular_numbers.next, " "
end
~~~

~~~ruby
fiber = Fiber.new do
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    Fiber.yield number
  end
end

print fiber.resume, " "

5.times do
  print fiber.resume, " "
end
~~~



-- 
https://bugs.ruby-lang.org/

[ruby-core:70591] [Ruby trunk - Feature #11348] TracePoint API needs events for fiber's switching

[os97673]

Issue #11348 has been updated by Oleg Sukhodolsky.


Oleg Sukhodolsky wrote:
> Koichi Sasada wrote:
> > I agree to add 'switching' events so I try to commit it.
> > 
> > Could you try it and could you tell me "what" is not enough?

I've played with the event a little bit and here is what I think.
Suppose we have the following program:

~~~ruby
f1 = Fiber.new do
  loop do
    Fiber.yield 1
  end
end
f1.resume # stopped at the line
f1.resume # want to stop here after step over
f1.resume
f1.resume
~~~

To implement expected behavior debugger (debase/byebug) store current stack size and stops at next line event with the same (or lower) stack size.
To handle fibers I would expect to use pair (fiber-stack-size, stack-size) and stop only if current fiber stack size is less than original or fiber stack sizes are equal and stack-size is equal or less than original (if you have better idea how to implement this feel free to share it with me).
So to implement this approach we need to detect fiber switch AND figure out if it was "fiber call" or "fiber return"  to be able to do this we either need to have two different events, or Fiber.nesting call you propose should help as at the moment we receive "fiber-switch" event, or (worse case for debugger) store the information about the switch and detect if it was call/return at next event we will process.  The latter is more complicated for debugger than 1 or 2.

Thus I'd prefer to have fiber-call/fiber-return, or be able to detect direction of the switch at the moment debugger is processing "fiber-switch".
Does this looks reasonable for you?  Is it doable on Ruby VM side? Do you have another ideas about implementing the debugger's functionality?

----------------------------------------
Feature #11348: TracePoint API needs events for fiber's switching
https://bugs.ruby-lang.org/issues/11348#change-53997

* Author: Oleg Sukhodolsky
* Status: Feedback
* Priority: Normal
* Assignee: Koichi Sasada
----------------------------------------
as discussed in https://github.com/deivid-rodriguez/byebug/issues/153 current implementation of byebug/debase has problem with stepping when Enumarator/Fiber is used.  The problem is that Fiber completely replaces stack (w/o any events) while the gems assume that any stack modification has corresponding call/return event.

It would be nice to receive events for Fiber's switching (at least) to be able to track stack for every fiber independently.
Also I think it would be nice to discuss the API before adding it because different set of event will allow to implement different debugger's capabilities.
E.g. plain :fiber-changed (event about every fiber switch) will only allow us to handle every fiber independently (like threads are handled) while debugger's user may expect that step into on enumerator.next will step into enumerator's code and step out from that code should go to the place where enumerator.next has been called.
So, perhaps we should have different events for Fiber#resume and Fiber#yield, but I'm not sure and would like to here other opinions.

Here are two tests which can be used to play with (one for Enumerator, another for Fiber)

~~~ruby
triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " "

5.times do
  print triangular_numbers.next, " "
end
~~~

~~~ruby
fiber = Fiber.new do
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    Fiber.yield number
  end
end

print fiber.resume, " "

5.times do
  print fiber.resume, " "
end
~~~



-- 
https://bugs.ruby-lang.org/

[ruby-core:70642] [Ruby trunk - Feature #11348] TracePoint API needs events for fiber's switching

[ko1]

Issue #11348 has been updated by Koichi Sasada.


> f1.resume # stopped at the line
> f1.resume # want to stop here after step over

Only for this purpose, how about to stop every fiber switching? It is true that the program enter newline!


----------------------------------------
Feature #11348: TracePoint API needs events for fiber's switching
https://bugs.ruby-lang.org/issues/11348#change-54043

* Author: Oleg Sukhodolsky
* Status: Feedback
* Priority: Normal
* Assignee: Koichi Sasada
----------------------------------------
as discussed in https://github.com/deivid-rodriguez/byebug/issues/153 current implementation of byebug/debase has problem with stepping when Enumarator/Fiber is used.  The problem is that Fiber completely replaces stack (w/o any events) while the gems assume that any stack modification has corresponding call/return event.

It would be nice to receive events for Fiber's switching (at least) to be able to track stack for every fiber independently.
Also I think it would be nice to discuss the API before adding it because different set of event will allow to implement different debugger's capabilities.
E.g. plain :fiber-changed (event about every fiber switch) will only allow us to handle every fiber independently (like threads are handled) while debugger's user may expect that step into on enumerator.next will step into enumerator's code and step out from that code should go to the place where enumerator.next has been called.
So, perhaps we should have different events for Fiber#resume and Fiber#yield, but I'm not sure and would like to here other opinions.

Here are two tests which can be used to play with (one for Enumerator, another for Fiber)

~~~ruby
triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " "

5.times do
  print triangular_numbers.next, " "
end
~~~

~~~ruby
fiber = Fiber.new do
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    Fiber.yield number
  end
end

print fiber.resume, " "

5.times do
  print fiber.resume, " "
end
~~~



-- 
https://bugs.ruby-lang.org/

[ruby-core:70645] [Ruby trunk - Feature #11348] TracePoint API needs events for fiber's switching

[os97673]

Issue #11348 has been updated by Oleg Sukhodolsky.


Koichi Sasada wrote:
> > f1.resume # stopped at the line
> > f1.resume # want to stop here after step over
> 
> Only for this purpose, how about to stop every fiber switching? It is true that the program enter newline!

I'm not sure I've completely understand you suggestion :(  Could you please clarify?
I've provided the example ask how to guarantee that we will not stop somewhere inside f1's implementation.

----------------------------------------
Feature #11348: TracePoint API needs events for fiber's switching
https://bugs.ruby-lang.org/issues/11348#change-54046

* Author: Oleg Sukhodolsky
* Status: Feedback
* Priority: Normal
* Assignee: Koichi Sasada
----------------------------------------
as discussed in https://github.com/deivid-rodriguez/byebug/issues/153 current implementation of byebug/debase has problem with stepping when Enumarator/Fiber is used.  The problem is that Fiber completely replaces stack (w/o any events) while the gems assume that any stack modification has corresponding call/return event.

It would be nice to receive events for Fiber's switching (at least) to be able to track stack for every fiber independently.
Also I think it would be nice to discuss the API before adding it because different set of event will allow to implement different debugger's capabilities.
E.g. plain :fiber-changed (event about every fiber switch) will only allow us to handle every fiber independently (like threads are handled) while debugger's user may expect that step into on enumerator.next will step into enumerator's code and step out from that code should go to the place where enumerator.next has been called.
So, perhaps we should have different events for Fiber#resume and Fiber#yield, but I'm not sure and would like to here other opinions.

Here are two tests which can be used to play with (one for Enumerator, another for Fiber)

~~~ruby
triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " "

5.times do
  print triangular_numbers.next, " "
end
~~~

~~~ruby
fiber = Fiber.new do
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    Fiber.yield number
  end
end

print fiber.resume, " "

5.times do
  print fiber.resume, " "
end
~~~



-- 
https://bugs.ruby-lang.org/

[ruby-core:70648] Re: [Ruby trunk - Feature #11348] TracePoint API needs events for fiber's switching

[SASADA Koichi]

On 2015/09/01 18:12, os97673@gmail.com wrote:
> I've provided the example ask how to guarantee that we will not stop somewhere inside f1's implementation.

Why you don't want to stop in a fiber?

-- 
// SASADA Koichi at atdot dot net