[ruby-core:92701] [Ruby trunk Feature#14844] Future of RubyVM::AST?

[mame@ruby-lang.org]

Issue #14844 has been updated by mame (Yusuke Endoh).


Eregon (Benoit Daloze) wrote:
> Could you or @yui-knk propose a description to include in the documentation, summarizing what was said?

I think that it is stated very clearly in the doc of RubyVM module.

```
    /*
     * Document-class: RubyVM
     *
     * The RubyVM module provides some access to Ruby internals.
     * This module is for very limited purposes, such as debugging,
     * prototyping, and research.  Normal users must not use it.
     */
```


> Could you also give your opinion on accessing Node members by name (https://bugs.ruby-lang.org/issues/14844#note-13) ?

I have no strong opinion, but am a bit negative.  I'm afraid if making it too useful may wrongly attract casual users.
And, if the feature requires an additional code for the node definition, it may be cumbersome to tweak the node definition.


> > Ripper does not reproduce the details including parser-level optimization.
> 
> What kind of details? Could you give an example?
> Things like OPCALL instead of CALL? Is that useful for any tool?

For example, `foo(**{})` is currently removed at the parser.  Ripper does not.

```
$ ruby -rripper -e 'pp RubyVM::AbstractSyntaxTree.parse("foo()")'
(SCOPE@1:0-1:5 tbl: [] args: nil body: (FCALL@1:0-1:5 :foo nil))

$ ruby -rripper -e 'pp RubyVM::AbstractSyntaxTree.parse("foo(**{})")'
(SCOPE@1:0-1:9 tbl: [] args: nil body: (FCALL@1:0-1:9 :foo nil))

$ ruby -rripper -e 'pp Ripper.sexp("foo()")'
[:program,
 [[:method_add_arg, [:fcall, [:@ident, "foo", [1, 0]]], [:arg_paren, nil]]]]

$ ruby -rripper -e 'pp Ripper.sexp("foo(**{})")'
[:program,
 [[:method_add_arg,
   [:fcall, [:@ident, "foo", [1, 0]]],
   [:arg_paren,
    [:args_add_block,
     [[:bare_assoc_hash, [[:assoc_splat, [:hash, nil]]]]],
     false]]]]]
```

I don't think this is a bug of Ripper.  Rather, this reflects the difference between their purposes.  I think Ripper provides an AST that is closer to the original code.  RubyVM::AST provides what the interpreter sees.  (It is arguable whether the omission of `**{}` is good or bad.  Anyway, it is another story.)

Another example is "for" statement.  The actual parser has 20+ lines for the syntax, but Ripper code has only one line.
https://github.com/ruby/ruby/blob/ea3e7e268546599883b25d9a33d26e042461ac25/parse.y#L2796

I'm unsure if this makes an important difference for any tool, but I think it is important for a purpose to research and debug MRI.

----------------------------------------
Feature #14844: Future of RubyVM::AST? 
https://bugs.ruby-lang.org/issues/14844#change-78059

* Author: rmosolgo (Robert Mosolgo)
* Status: Open
* Priority: Normal
* Assignee: yui-knk (Kaneko Yuichiro)
* Target version: 
----------------------------------------
Hi! Thanks for all your great work on the Ruby language. 

I saw the new RubyVM::AST module in 2.6.0-preview2 and I quickly went to try it out. 

I'd love to have a well-documented, user-friendly way to parse and manipulate Ruby code using the Ruby standard library, so I'm pretty excited to try it out. (I've been trying to learn Ripper recently, too: https://ripper-preview.herokuapp.com/, https://rmosolgo.github.io/ripper_events/ .)

Based on my exploration, I opened a small PR on GitHub with some documentation: https://github.com/ruby/ruby/pull/1888

I'm curious though, are there future plans for this module? For example, we might: 

- Add more details about each node (for example, we could expose the names of identifiers and operators through the node classes)
- Document each node type 

I see there is a lot more information in the C structures that we could expose, and I'm interested to help out if it's valuable. What do you think? 



-- 
https://bugs.ruby-lang.org/