[ruby-core:97959] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[ruby-core:97959] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[samuel]

Issue #8215 has been updated by ioquatix (Samuel Williams).


@matz do you mind checking this when you have time? Especially `Fiber#backtrace` is super important for debugging issues with fibers.

----------------------------------------
Feature #8215: Support accessing Fiber-locals and backtraces for a Fiber
https://bugs.ruby-lang.org/issues/8215#change-85188

* Author: halorgium (Tim Carey-Smith)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
=begin
As part of debugging celluloid, I have been wanting to diagnose where the Fibers are running and their various locals.

I would expect the following to work.

  Thread.current[:key] = "outside"
  fiber = Fiber.new do
    Thread.current[:key] = "inside"
    Fiber.yield
  end
  fiber.resume
  fiber[:key] == "inside" # true
  fiber.backtrace # ...

I also wonder whether (({Fiber#[]})) should be implemented, so (({Fiber.current[:key]})) is possible.

For reference, here is the issue on the rubinius issue tracker: ((<"github/rubinius/rubinius/2200"|URL:https://github.com/rubinius/rubinius/issues/2200>))
=end


---Files--------------------------------
0001-cont.c-fiber-local-accessors.patch (2.94 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:97987] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[samuel]

Issue #8215 has been updated by ioquatix (Samuel Williams).


Here is a quick hack I used to add `Fiber.backtrace` to aid in debugging. I'd say, don't use it in production, but I did :p

```ruby
module Fiber::Backtrace
	def yield
		Fiber.current.backtrace = caller
		super
	end
end

class Fiber
	attr_accessor :backtrace
	
	class << self
		prepend Fiber::Backtrace
	end
end
```

----------------------------------------
Feature #8215: Support accessing Fiber-locals and backtraces for a Fiber
https://bugs.ruby-lang.org/issues/8215#change-85216

* Author: halorgium (Tim Carey-Smith)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
=begin
As part of debugging celluloid, I have been wanting to diagnose where the Fibers are running and their various locals.

I would expect the following to work.

  Thread.current[:key] = "outside"
  fiber = Fiber.new do
    Thread.current[:key] = "inside"
    Fiber.yield
  end
  fiber.resume
  fiber[:key] == "inside" # true
  fiber.backtrace # ...

I also wonder whether (({Fiber#[]})) should be implemented, so (({Fiber.current[:key]})) is possible.

For reference, here is the issue on the rubinius issue tracker: ((<"github/rubinius/rubinius/2200"|URL:https://github.com/rubinius/rubinius/issues/2200>))
=end


---Files--------------------------------
0001-cont.c-fiber-local-accessors.patch (2.94 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:98048] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[eregontp]

Issue #8215 has been updated by Eregon (Benoit Daloze).


@ioquatix Please create a separate feature ticket for `Fiber#backtrace`, as already asked by nobu in https://bugs.ruby-lang.org/issues/8215#note-5
I'd like to keep the discussion here about Fiber locals, not mix it with an unrelated method.
A separate ticket is better so e.g. the discussion is clear and focused, that ticket can be closed when implemented, the reference from NEWS will be much less confusing, etc.

Back to the fiber locals discussion:

> Thread safety is not my concern, the function can be marked as thread unsafe, and maybe we can add detection of this and warn against it.

It's of course unacceptable if a Ruby implementation behaves unsafely (e.g., crashes or loses fiber local variable) for this, so it is a relevant concern for this issue.

Another one is this needs `Fiber.current`, which is only available after `require "fiber"`.
I think that could be confusing, so I'd suggest making `Fiber.current` always available if we go that way.

I think `Fiber[:foo]` and `Fiber[:foo] = ...` is a better API because it avoids this problem entirely, and gives the possibility to have clean Thread and Fiber locals APIs.
We could also add a check in `Fiber#[]` but that would read the current Fiber a second time, and move it to a runtime error instead of simply not being possible with `Fiber[:foo]`.
Maybe the check is not so important since I guess `Thread#[]` will still need to work for a long while, but for new APIs I think a good safe design is worth considering.

If you think we need the ability to access another Fiber's locals, could you show some examples?

----------------------------------------
Feature #8215: Support accessing Fiber-locals and backtraces for a Fiber
https://bugs.ruby-lang.org/issues/8215#change-85277

* Author: halorgium (Tim Carey-Smith)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
=begin
As part of debugging celluloid, I have been wanting to diagnose where the Fibers are running and their various locals.

I would expect the following to work.

  Thread.current[:key] = "outside"
  fiber = Fiber.new do
    Thread.current[:key] = "inside"
    Fiber.yield
  end
  fiber.resume
  fiber[:key] == "inside" # true
  fiber.backtrace # ...

I also wonder whether (({Fiber#[]})) should be implemented, so (({Fiber.current[:key]})) is possible.

For reference, here is the issue on the rubinius issue tracker: ((<"github/rubinius/rubinius/2200"|URL:https://github.com/rubinius/rubinius/issues/2200>))
=end


---Files--------------------------------
0001-cont.c-fiber-local-accessors.patch (2.94 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:98052] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[eregontp]

Issue #8215 has been updated by Eregon (Benoit Daloze).


To make it clearer about "clean API": if we have Fiber#[] and Thread#[] then Thread#[] is just deceptive and doesn't do anything different.
OTOH, with Fiber[:foo] and Thread[:foo] we can actually have the intuitive semantics.

----------------------------------------
Feature #8215: Support accessing Fiber-locals and backtraces for a Fiber
https://bugs.ruby-lang.org/issues/8215#change-85280

* Author: halorgium (Tim Carey-Smith)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
=begin
As part of debugging celluloid, I have been wanting to diagnose where the Fibers are running and their various locals.

I would expect the following to work.

  Thread.current[:key] = "outside"
  fiber = Fiber.new do
    Thread.current[:key] = "inside"
    Fiber.yield
  end
  fiber.resume
  fiber[:key] == "inside" # true
  fiber.backtrace # ...

I also wonder whether (({Fiber#[]})) should be implemented, so (({Fiber.current[:key]})) is possible.

For reference, here is the issue on the rubinius issue tracker: ((<"github/rubinius/rubinius/2200"|URL:https://github.com/rubinius/rubinius/issues/2200>))
=end


---Files--------------------------------
0001-cont.c-fiber-local-accessors.patch (2.94 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:98055] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[samuel]

Issue #8215 has been updated by ioquatix (Samuel Williams).


> It's of course unacceptable if a Ruby implementation behaves unsafely (e.g., crashes or loses fiber local variable) for this, so it is a relevant concern for this issue.

We already have situations in JRuby and TruffleRuby where unsynchronized access to shared mutable state can cause a crash or data corruption. So why is this different?

If you believe it's important, then I defer to your judgement. However, from my point of view:

- This method should not be invoked across threads.
- This method should not need to be thread safe.
- The GVL might make it safe but it shouldn't be guaranteed.
- Maybe we need annotations or other documentation to explain this.

> If you think we need the ability to access another Fiber's locals, could you show some examples?

The reason to access other fiber locals is mostly for debugging purposes. In Async, we make a list of tasks:

```
 4874m43s     info: Falcon::Controller::Proxy [oid=0x2b28b0f0d558] [pid=324886] [2020-04-24 09:55:55 +0000]
                  | #<Async::Reactor:0x2b28b10316c8 1 children running>
                  |         #<Async::Task:0x2b28b1031498 (running)>
                  |                 #<Async::Task:0x2b28b10312e0 waiting for signal USR1 (running)>
                  |                 #<Async::Task:0x2b28b1030930 accepting secure connection #<Addrinfo: [::ffff:34.228.115.75]:26564 TCP> (running)>
                  |                         #<Async::Task:0x2b28b1079b1c connected to #<Addrinfo: /srv/http/www.oriontransfer.co.nz/application.ipc SOCK_STREAM> [fd=17] (complete)>
                  |                                 #<Async::Task:0x2b28b109a7b8 h2 reading data for Async::HTTP::Protocol::HTTP2::Client. (running)>
                  |                         #<Async::Task:0x2b28b10869e8 Reading h2 requests for Async::HTTP::Protocol::HTTP2::Server. (complete)>
                  |                                 #<Async::Task:0x2b28b13a2dc0 connected to #<Addrinfo: /srv/http/www.codeotaku.com/application.ipc SOCK_STREAM> [fd=16] (complete)>
                  |                                         #<Async::Task:0x2b28b12fb8f4 h2 reading data for Async::HTTP::Protocol::HTTP2::Client. (running)>
                  |                         #<Async::Task:0x2b28b0fefe30 Reading h2 requests for Async::HTTP::Protocol::HTTP2::Server. (running)>
                  |                                 #<Async::Task:0x2b28b1004b64 h2 reading data for Async::HTTP::Protocol::HTTP2::Server. (running)>
```

When something is going wrong, the ability to dig into the fiber state is very useful. My plan is to make debugging tools to expose this more easily but I can't do it if there are no methods to access the state.


----------------------------------------
Feature #8215: Support accessing Fiber-locals and backtraces for a Fiber
https://bugs.ruby-lang.org/issues/8215#change-85283

* Author: halorgium (Tim Carey-Smith)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
=begin
As part of debugging celluloid, I have been wanting to diagnose where the Fibers are running and their various locals.

I would expect the following to work.

  Thread.current[:key] = "outside"
  fiber = Fiber.new do
    Thread.current[:key] = "inside"
    Fiber.yield
  end
  fiber.resume
  fiber[:key] == "inside" # true
  fiber.backtrace # ...

I also wonder whether (({Fiber#[]})) should be implemented, so (({Fiber.current[:key]})) is possible.

For reference, here is the issue on the rubinius issue tracker: ((<"github/rubinius/rubinius/2200"|URL:https://github.com/rubinius/rubinius/issues/2200>))
=end


---Files--------------------------------
0001-cont.c-fiber-local-accessors.patch (2.94 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:98057] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[eregontp]

Issue #8215 has been updated by Eregon (Benoit Daloze).


> We already have situations in JRuby and TruffleRuby where unsynchronized access to shared mutable state can cause a crash or data corruption. So why is this different?

Let's not introduce more bugs (and IMHO those bugs should be fixed), especially if these methods are supposed to access *local* state, which means state you may not access from any other Thread.

It's probably fine to access locals of another Fiber of the same Thread though, that's not racy since Fibers of a Thread don't run simultaneously.
So I'd be OK with that, if we raise when trying to access locals of a Fiber belonging to a different Thread.

Is there a reason you want to store this state in Fiber locals and not e.g., in instance variables of the Fiber object?

----------------------------------------
Feature #8215: Support accessing Fiber-locals and backtraces for a Fiber
https://bugs.ruby-lang.org/issues/8215#change-85285

* Author: halorgium (Tim Carey-Smith)
* Status: Assigned
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
=begin
As part of debugging celluloid, I have been wanting to diagnose where the Fibers are running and their various locals.

I would expect the following to work.

  Thread.current[:key] = "outside"
  fiber = Fiber.new do
    Thread.current[:key] = "inside"
    Fiber.yield
  end
  fiber.resume
  fiber[:key] == "inside" # true
  fiber.backtrace # ...

I also wonder whether (({Fiber#[]})) should be implemented, so (({Fiber.current[:key]})) is possible.

For reference, here is the issue on the rubinius issue tracker: ((<"github/rubinius/rubinius/2200"|URL:https://github.com/rubinius/rubinius/issues/2200>))
=end


---Files--------------------------------
0001-cont.c-fiber-local-accessors.patch (2.94 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99520] [Ruby master Feature#8215] Support accessing Fiber-locals and backtraces for a Fiber

[samuel]

Issue #8215 has been updated by ioquatix (Samuel Williams).

Status changed from Assigned to Closed

It looks like we can implement `Fiber#backtrace`:

https://bugs.ruby-lang.org/issues/16815

and in addition we should continue the discussion about Fiber locals:

https://bugs.ruby-lang.org/issues/13893

----------------------------------------
Feature #8215: Support accessing Fiber-locals and backtraces for a Fiber
https://bugs.ruby-lang.org/issues/8215#change-86981

* Author: halorgium (Tim Carey-Smith)
* Status: Closed
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
=begin
As part of debugging celluloid, I have been wanting to diagnose where the Fibers are running and their various locals.

I would expect the following to work.

  Thread.current[:key] = "outside"
  fiber = Fiber.new do
    Thread.current[:key] = "inside"
    Fiber.yield
  end
  fiber.resume
  fiber[:key] == "inside" # true
  fiber.backtrace # ...

I also wonder whether (({Fiber#[]})) should be implemented, so (({Fiber.current[:key]})) is possible.

For reference, here is the issue on the rubinius issue tracker: ((<"github/rubinius/rubinius/2200"|URL:https://github.com/rubinius/rubinius/issues/2200>))
=end


---Files--------------------------------
0001-cont.c-fiber-local-accessors.patch (2.94 KB)


-- 
https://bugs.ruby-lang.org/