* [ruby-core:98931] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
@ 2020-06-24 12:57 kuwabara
2020-08-28 21:35 ` [ruby-core:99759] " merch-redmine
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: kuwabara @ 2020-06-24 12:57 UTC (permalink / raw
To: ruby-core
Issue #16983 has been reported by pocke (Masataka Kuwabara).
----------------------------------------
Bug #16983: RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
https://bugs.ruby-lang.org/issues/16983
* Author: pocke (Masataka Kuwabara)
* Status: Open
* Priority: Normal
* ruby -v: ruby 2.8.0dev (2020-06-23T13:58:26Z master dc351ff984) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
# Problem
`RubyVM::AST.of(method)` returns a meaningless node if the method is defined in eval.
For example:
```ruby
p 'blah'
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
method = method(:foo)
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
I expect the node of `foo` method, or `nil`. But it returns a `STR` node.
It becomes a big problem when `AST.of` receives arbitrary methods.
Because we can't distinguish a method is defined in `eval` or not.
It means we can't believe the returned value of `AST.of` if the method may receive a method defined in `eval`.
For example:
```ruby
def do_something_for_each_method_ast(klass)
klass.instance_methods(false).each do |m|
ast = RubyVM::AbstractSyntaxTree.of(klass.instance_method(m))
next unless ast
do_something ast
end
end
class A
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
end
do_something_for_each_method_ast A
```
In the example, I expect the `do_something` method receives only node for a method definition,
but it may pass a wrong node if any method is defined in `eval`.
# Cause (I guess)
I guess the cause is misleading node number.
In and out of an `eval` block uses different sequences of node number.
So if I specify `__FILE__` to `eval`, the actual file and code in `eval` may have the same node number.
For example
```ruby
p 'blah' # Node number for 'blah' is 1, file name is "test.rb"
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo # Node number for `def` is also 1, file name is also "test.rb"
end
RUBY
method = method(:foo)
# It finds a node from node number 1 by reading "test.rb", so it get the str node.
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* [ruby-core:99759] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
2020-06-24 12:57 [ruby-core:98931] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval kuwabara
@ 2020-08-28 21:35 ` merch-redmine
2021-04-17 7:21 ` [ruby-core:103492] " mame
2021-05-21 19:11 ` [ruby-core:103957] " merch-redmine
2 siblings, 0 replies; 4+ messages in thread
From: merch-redmine @ 2020-08-28 21:35 UTC (permalink / raw
To: ruby-core
Issue #16983 has been updated by jeremyevans0 (Jeremy Evans).
I'm not sure if this is a bug, but it does seem like a fundamental and significant limitation with the design of RubyVM::AbstractSyntaxTree.of. RubyVM::AbstractSyntaxTree.of reparses the file the method is defined in and cannot handle any cases where `eval` or similar are used. You'll get a node completely different from what you would expect. Here's another example:
```ruby
eval DATA.read, binding, __FILE__, 14
method = method(:foo)
pp RubyVM::AbstractSyntaxTree.of(method)
__END__
def foo
end
```
Output:
```
(VCALL@1:16-1:23 :binding)
```
Because it reparses the file, you'll also get the wrong result if the file is modified:
```ruby
def bar
end
File.write(__FILE__, File.read(__FILE__).gsub('def bar', "def foo\nbar"))
method = method(:bar)
pp RubyVM::AbstractSyntaxTree.of(method)
```
Output:
```
(SCOPE@1:0-3:3
tbl: []
args:
(ARGS@1:7-1:7
pre_num: 0
pre_init: nil
opt: nil
first_post: nil
post_num: 0
post_init: nil
rest: nil
kw: nil
kwrest: nil
block: nil)
body: (VCALL@2:0-2:3 :bar))
```
And if the interpreter can no longer access the file (chroot, file deletion, permission change, or other file system access limiting), you get an error.
I can't think of a way to fix this without all iseq methods holding a reference to the string used to parse them, and having RubyVM::AbstractSyntaxTree.of work off that string. I'm not sure how much extra memory use that would cause, or if such an approach is considered acceptable.
----------------------------------------
Bug #16983: RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
https://bugs.ruby-lang.org/issues/16983#change-87263
* Author: pocke (Masataka Kuwabara)
* Status: Open
* Priority: Normal
* ruby -v: ruby 2.8.0dev (2020-06-23T13:58:26Z master dc351ff984) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
# Problem
`RubyVM::AST.of(method)` returns a meaningless node if the method is defined in eval.
For example:
```ruby
p 'blah'
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
method = method(:foo)
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
I expect the node of `foo` method, or `nil`. But it returns a `STR` node.
It becomes a big problem when `AST.of` receives arbitrary methods.
Because we can't distinguish a method is defined in `eval` or not.
It means we can't believe the returned value of `AST.of` if the method may receive a method defined in `eval`.
For example:
```ruby
def do_something_for_each_method_ast(klass)
klass.instance_methods(false).each do |m|
ast = RubyVM::AbstractSyntaxTree.of(klass.instance_method(m))
next unless ast
do_something ast
end
end
class A
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
end
do_something_for_each_method_ast A
```
In the example, I expect the `do_something` method receives only node for a method definition,
but it may pass a wrong node if any method is defined in `eval`.
# Cause (I guess)
I guess the cause is misleading node number.
In and out of an `eval` block uses different sequences of node number.
So if I specify `__FILE__` to `eval`, the actual file and code in `eval` may have the same node number.
For example
```ruby
p 'blah' # Node number for 'blah' is 1, file name is "test.rb"
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo # Node number for `def` is also 1, file name is also "test.rb"
end
RUBY
method = method(:foo)
# It finds a node from node number 1 by reading "test.rb", so it get the str node.
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* [ruby-core:103492] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
2020-06-24 12:57 [ruby-core:98931] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval kuwabara
2020-08-28 21:35 ` [ruby-core:99759] " merch-redmine
@ 2021-04-17 7:21 ` mame
2021-05-21 19:11 ` [ruby-core:103957] " merch-redmine
2 siblings, 0 replies; 4+ messages in thread
From: mame @ 2021-04-17 7:21 UTC (permalink / raw
To: ruby-core
Issue #16983 has been updated by mame (Yusuke Endoh).
Assignee set to ko1 (Koichi Sasada)
Status changed from Open to Assigned
This ticket was discussed on dev-meeting. A method (or proc) created in an `eval` context should be marked and `AST.of` should raise an exception against a marked method. @ko1 said that he will do.
----------------------------------------
Bug #16983: RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
https://bugs.ruby-lang.org/issues/16983#change-91588
* Author: pocke (Masataka Kuwabara)
* Status: Assigned
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 2.8.0dev (2020-06-23T13:58:26Z master dc351ff984) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
# Problem
`RubyVM::AST.of(method)` returns a meaningless node if the method is defined in eval.
For example:
```ruby
p 'blah'
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
method = method(:foo)
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
I expect the node of `foo` method, or `nil`. But it returns a `STR` node.
It becomes a big problem when `AST.of` receives arbitrary methods.
Because we can't distinguish a method is defined in `eval` or not.
It means we can't believe the returned value of `AST.of` if the method may receive a method defined in `eval`.
For example:
```ruby
def do_something_for_each_method_ast(klass)
klass.instance_methods(false).each do |m|
ast = RubyVM::AbstractSyntaxTree.of(klass.instance_method(m))
next unless ast
do_something ast
end
end
class A
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
end
do_something_for_each_method_ast A
```
In the example, I expect the `do_something` method receives only node for a method definition,
but it may pass a wrong node if any method is defined in `eval`.
# Cause (I guess)
I guess the cause is misleading node number.
In and out of an `eval` block uses different sequences of node number.
So if I specify `__FILE__` to `eval`, the actual file and code in `eval` may have the same node number.
For example
```ruby
p 'blah' # Node number for 'blah' is 1, file name is "test.rb"
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo # Node number for `def` is also 1, file name is also "test.rb"
end
RUBY
method = method(:foo)
# It finds a node from node number 1 by reading "test.rb", so it get the str node.
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
* [ruby-core:103957] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
2020-06-24 12:57 [ruby-core:98931] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval kuwabara
2020-08-28 21:35 ` [ruby-core:99759] " merch-redmine
2021-04-17 7:21 ` [ruby-core:103492] " mame
@ 2021-05-21 19:11 ` merch-redmine
2 siblings, 0 replies; 4+ messages in thread
From: merch-redmine @ 2021-05-21 19:11 UTC (permalink / raw
To: ruby-core
Issue #16983 has been updated by jeremyevans0 (Jeremy Evans).
Because I know how busy @ko1 is, I thought I'd save him some work and tried to implement this myself: https://github.com/ruby/ruby/pull/4519
----------------------------------------
Bug #16983: RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval
https://bugs.ruby-lang.org/issues/16983#change-92089
* Author: pocke (Masataka Kuwabara)
* Status: Assigned
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 2.8.0dev (2020-06-23T13:58:26Z master dc351ff984) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
# Problem
`RubyVM::AST.of(method)` returns a meaningless node if the method is defined in eval.
For example:
```ruby
p 'blah'
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
method = method(:foo)
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
I expect the node of `foo` method, or `nil`. But it returns a `STR` node.
It becomes a big problem when `AST.of` receives arbitrary methods.
Because we can't distinguish a method is defined in `eval` or not.
It means we can't believe the returned value of `AST.of` if the method may receive a method defined in `eval`.
For example:
```ruby
def do_something_for_each_method_ast(klass)
klass.instance_methods(false).each do |m|
ast = RubyVM::AbstractSyntaxTree.of(klass.instance_method(m))
next unless ast
do_something ast
end
end
class A
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo
end
RUBY
end
do_something_for_each_method_ast A
```
In the example, I expect the `do_something` method receives only node for a method definition,
but it may pass a wrong node if any method is defined in `eval`.
# Cause (I guess)
I guess the cause is misleading node number.
In and out of an `eval` block uses different sequences of node number.
So if I specify `__FILE__` to `eval`, the actual file and code in `eval` may have the same node number.
For example
```ruby
p 'blah' # Node number for 'blah' is 1, file name is "test.rb"
eval <<~RUBY, binding, __FILE__, __LINE__ + 1
def foo # Node number for `def` is also 1, file name is also "test.rb"
end
RUBY
method = method(:foo)
# It finds a node from node number 1 by reading "test.rb", so it get the str node.
pp RubyVM::AbstractSyntaxTree.of(method)
# => (STR@3:5-3:12 "def foo\n" + "end\n")
```
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-05-21 19:11 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-06-24 12:57 [ruby-core:98931] [Ruby master Bug#16983] RubyVM::AbstractSyntaxTree.of(method) returns meaningless node if the method is defined in eval kuwabara
2020-08-28 21:35 ` [ruby-core:99759] " merch-redmine
2021-04-17 7:21 ` [ruby-core:103492] " mame
2021-05-21 19:11 ` [ruby-core:103957] " merch-redmine
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).