[ruby-core:88509] [Ruby trunk Feature#14844] Future of RubyVM::AST?

[mame@ruby-lang.org]

Issue #14844 has been updated by mame (Yusuke Endoh).


bozhidar (Bozhidar Batsov) wrote:
> I'm really curious what's the purpose of this module and why wasn't in developed in collaboration with the maintainers of libraries like https://github.com/whitequark/parser and https://github.com/whitequark/ast, and the maintainers of prominent AST-based tools (e.g. https://github.com/rubocop-hq/rubocop)? 

First of all, thank you for developing the parser gem and related tools including Rubocop.

As far as I understand, RubyVM module is completely different than other builtin modules.  It exposes an access to Ruby internal for very limited purpose, such as debugging the internal, prototyping a new feature, implementing a MRI-bundled feature, etc.  No compatibility is guaranteed at all; its API will arbitrarily change along with internal change.  It is never intended for normal users to use it.  Usefulness is not a priority for the modules under RubyVM.  This fact can also be seen from `RubyVM::InstructionSequence`.  (Its has a very long name, which also represents non-casual use, so it may be better to rename `RubyVM::AST` with `AbstractSyntaxTree`.)

In fact, Yuichiro Kaneko developped `RubyVM::AST` for testing a parser-related new feature, column number of each node, that he introduced in Ruby 2.5.  It was originally a hidden external library.  Some people (including me) requested to expose it as an internal-use module, and matz approved it.  Then it is introduced into trunk as an experimental feature.  Kaneko-san wanted `RubyVM::AST` for some study purpose (for example, finding all callsites of some function), and my motivation for the exposure is that it would be needed to import Ruby3's type system.  (There are some proposals for type-checking Ruby.  It is not decided which would be imported, or it is even uncertain if a AST module is really needed, but I just wanted to remove a blocker candidate in advance.)

`RubyVM::AST` does not decrease the value of the parser gem.  `RubyVM::AST` is useful to investigate how MRI looks a Ruby program, but is not useful as a general Ruby program parser; it may optimize the AST by omitting some non-significant letters and restructuring the tree structure, so the result might not correspond to the original source code literally.  I think that this property is not useful for Rubocop, for example.

In short, I think that RubyVM::AST is not what people expected.  Users may use it just for research purpose, but must not use it in production.

----------------------------------------
Feature #14844: Future of RubyVM::AST? 
https://bugs.ruby-lang.org/issues/14844#change-73568

* Author: rmosolgo (Robert Mosolgo)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
Hi! Thanks for all your great work on the Ruby language. 

I saw the new RubyVM::AST module in 2.6.0-preview2 and I quickly went to try it out. 

I'd love to have a well-documented, user-friendly way to parse and manipulate Ruby code using the Ruby standard library, so I'm pretty excited to try it out. (I've been trying to learn Ripper recently, too: https://ripper-preview.herokuapp.com/, https://rmosolgo.github.io/ripper_events/ .)

Based on my exploration, I opened a small PR on GitHub with some documentation: https://github.com/ruby/ruby/pull/1888

I'm curious though, are there future plans for this module? For example, we might: 

- Add more details about each node (for example, we could expose the names of identifiers and operators through the node classes)
- Document each node type 

I see there is a lot more information in the C structures that we could expose, and I'm interested to help out if it's valuable. What do you think? 



-- 
https://bugs.ruby-lang.org/