[ruby-core:102036] [Ruby master Bug#17530] irb handles << incorrectly with variable as identifier

[ruby-core:102036] [Ruby master Bug#17530] irb handles << incorrectly with variable as identifier

[dfranklin]

Issue #17530 has been reported by danfranklin (Dan Franklin).

----------------------------------------
Bug #17530: irb handles << incorrectly with variable as identifier
https://bugs.ruby-lang.org/issues/17530

* Author: danfranklin (Dan Franklin)
* Status: Open
* Priority: Normal
* ruby -v: 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
If you attempt to start a here-document with an identifier that is also a variable, it is interpreted as an append, which makes sense.
That is,

> s1 = 'testing'
> s2 = 'this'
> s2 <<s1
> adding text here does not work
> s1

Gets an expected error:

> testcase.rb:4: syntax error, unexpected local variable or method, expecting '('
> adding text here does not work

But irb doesn't know this, and gathers input as though it's a here-document:

```
irb(main):001:0' s1 = 'testing'
=> "testing"
irb(main):002:0' s2 = 'this'
=> "this"
irb(main):003:0" s2 <<s1
irb(main):004:0" adding text here does not work
irb(main):005:0" s1
Traceback (most recent call last):
3: from /home/centos/.rubies/ruby-3.0.0/bin/irb:23:in `<main>'
2: from /home/centos/.rubies/ruby-3.0.0/bin/irb:23:in `load'
1: from /home/centos/.rubies/ruby-3.0.0/lib/ruby/gems/3.0.0/gems/irb-1.3.0/exe/irb:11:in `<top (required)>'
SyntaxError ((irb):4: syntax error, unexpected local variable or method, expecting '(')
adding text here does not work
^~~~
irb(main):006:0> 

```
Notice the prompt changes on lines 3-5 as though line 3 started a double-quoted here document.

irb should realize that line 3 is appending to s2, not starting a here-document.

This bug is present in the latest irb:

ruby --version
ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]

irb --version
irb 1.3.0 (2020-12-25)

It's an odd edge case, but it confused me when I tried to understand the double meaning of <<, so it seemed worth noting.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103926] [Ruby master Bug#17530] irb handles << incorrectly with variable as identifier

[merch-redmine]

Issue #17530 has been updated by jeremyevans0 (Jeremy Evans).


This issue comes from Ripper, and is due to the fact that in irb, ripper is being called line-by-line and not all at once.  When ripper parses the `s2 <<s1` line, it doesn't know that `s2` is a local variable, so it treats `<<s1` as a heredoc (`on_heredoc_beg`) instead of `<<` and `s1` as separate tokens.

There are a few ways to fix this:

1. Modify ripper to somehow take a list of local variables to consider already in scope, and use that.
2. Modify irb to prepend the local variables in scope when using Ripper to evaluate each line.
3. Modify irb to check on `on_heredoc_beg` for preceding `on_sp` and `on_ident` tokens, and if the `on_ident` token is a local variable in scope, with similar changes for `on_heredoc_end`.

I thought approach 2 was the simplest to implement, so I've submitted a pull request for it: https://github.com/ruby/irb/pull/242

FWIW, the behavior that switches from heredoc to append is not that heredoc identifier (`s1` in the example) is a local variable, but that the receiver of `<<` (`s2` in the example) is a local variable.  If `s2` is a method, the code is treated as a heredoc:

```ruby
s1 = 'testing'
def s2(*a); p('this' + a.inspect) end
s2 <<s1
adding text here does not work
s1
# "this[\"adding text here does not work\\n\"]"
```

If `s1` is a method, the code is treated as an append:

```ruby
def s1(*a); p('testing' + a.inspect) end
s2 = 'this'
s2 <<s1
adding text here does not work
s1
# -:4: syntax error, unexpected local variable or method, expecting '('
# adding text here does not work
```

----------------------------------------
Bug #17530: irb handles << incorrectly with variable as identifier
https://bugs.ruby-lang.org/issues/17530#change-92056

* Author: danfranklin (Dan Franklin)
* Status: Open
* Priority: Normal
* ruby -v: 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
If you attempt to start a here-document with an identifier that is also a variable, it is interpreted as an append, which makes sense.
That is,

> s1 = 'testing'
> s2 = 'this'
> s2 <<s1
> adding text here does not work
> s1

Gets an expected error:

> testcase.rb:4: syntax error, unexpected local variable or method, expecting '('
> adding text here does not work

But irb doesn't know this, and gathers input as though it's a here-document:

```
irb(main):001:0' s1 = 'testing'
=> "testing"
irb(main):002:0' s2 = 'this'
=> "this"
irb(main):003:0" s2 <<s1
irb(main):004:0" adding text here does not work
irb(main):005:0" s1
Traceback (most recent call last):
3: from /home/centos/.rubies/ruby-3.0.0/bin/irb:23:in `<main>'
2: from /home/centos/.rubies/ruby-3.0.0/bin/irb:23:in `load'
1: from /home/centos/.rubies/ruby-3.0.0/lib/ruby/gems/3.0.0/gems/irb-1.3.0/exe/irb:11:in `<top (required)>'
SyntaxError ((irb):4: syntax error, unexpected local variable or method, expecting '(')
adding text here does not work
^~~~
irb(main):006:0> 

```
Notice the prompt changes on lines 3-5 as though line 3 started a double-quoted here document.

irb should realize that line 3 is appending to s2, not starting a here-document.

This bug is present in the latest irb:

ruby --version
ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]

irb --version
irb 1.3.0 (2020-12-25)

It's an odd edge case, but it confused me when I tried to understand the double meaning of <<, so it seemed worth noting.




-- 
https://bugs.ruby-lang.org/