ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:100214] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
       [not found] <redmine.issue-15993.20190710194105.25466@ruby-lang.org>
@ 2020-09-29 14:04 ` tschoening
  2021-06-16 17:53 ` [ruby-core:104317] " merch-redmine
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 4+ messages in thread
From: tschoening @ 2020-09-29 14:04 UTC (permalink / raw
  To: ruby-core

Issue #15993 has been updated by tschoening (Thorsten Schöning).


I think I have a similar problem originally reported at GitHub already: 

https://github.com/rubygems/rubygems/issues/3853

I have a Ruby-based shell application which needs to require a library during startup. I'm using the following command line:

``` shell
"..\ruby\bin\ruby.exe" "-I../runtime/lib" "../visualizer/bin/ksv" "--require=de/[...]/par_opp_dispatcher.rb" "--opaque-types=true" "../files_to_show/recs_clt.bin" "de/[...]/par_recs_clt.rb"
```

This results in the following error, while the first line describes the current directory I'm in. It contains some German umlaut `ü`. Using an ASCII-only path, things work as expected.

``` shell
C:\[...]\Müller electronic\[...]\ks_ruby_visualizer>show.cmd
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- C:/[...]/Müller electronic/[...]/rubygems.rb (LoadError)
```

The problem seems to be that at some point Ruby really seems to forward UTF-8 encoded bytes into the file system and such a path simply doesn't exist. The interesting thing is that many times before the path is forwarded correctly according to the following ProcMon-log:

``` shell
18:57:48,7938985	ruby.exe	15296	CreateFile	C:\[...]\Müller electronic\[...]\rubygems.rb	SUCCESS	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
18:57:48,7940217	ruby.exe	15296	QueryBasicInformationFile	C:\[...]\Müller electronic\[...]\rubygems.rb	SUCCESS	CreationTime: 24.07.2020 14:48:44, LastAccessTime: 24.07.2020 14:48:44, LastWriteTime: 01.10.2019 23:01:05, ChangeTime: 04.02.2020 22:30:28, FileAttributes: A 0x80000
18:57:48,7940500	ruby.exe	15296	CloseFile	C:\[...]\Müller electronic\[...]\rubygems.rb	SUCCESS	
18:57:48,7942644	ruby.exe	15296	CreateFile	C:\[...]\Müller electronic\[...]\rubygems.rb	SUCCESS	Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a, OpenResult: Opened
18:57:48,7943188	ruby.exe	15296	CloseFile	C:\[...]\Müller electronic\[...]\rubygems.rb	SUCCESS	
18:57:48,7945545	ruby.exe	15296	CreateFile	C:\[...]\Müller electronic\[...]\rubygems.rb	PATH NOT FOUND	Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
```

![Clipboard01](https://user-images.githubusercontent.com/6223655/88416199-13655d80-cde0-11ea-9b0d-2319b6c3af2b.png)

Here are my current environment details:

``` shell
$ gem env version
3.0.3
```

* Windows 10 1909 x86-64
* default codepages Windows-1252 and CP-850
* Ruby 2.6.5


----------------------------------------
Bug #15993: 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
https://bugs.ruby-lang.org/issues/15993#change-87804

* Author: inversion (Yura Babak)
* Status: Open
* Priority: Normal
* ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
I’m trying to build a cross-platform portable application with Ruby onboard and there is a problem on Windows. 
A user usually installs it to the Roaming folder which sits inside a user folder which can often have not a Latin name or contain spaces). 
When there is a Cyrillic character (maybe just not Latin) in the path — require of any gem doesn’t work:

```
D:\users\киї\Ruby\2.6\bin>ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]

D:\users\киї\Ruby\2.6\bin>ruby -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>ruby --disable=rubyopt -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>gem list
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

```

We can see such encoding transformations in the output:

```
киї (utf-8) == РєРёС— (win1251)
```

I have an old Ruby installation that works fine:

```
D:\users\киї\Ruby\2.0\bin>ruby -e "require 'logger'"

D:\users\киї\Ruby\2.0\bin>ruby -v
ruby 2.0.0p451 (2014-02-24) [i386-mingw32]
```
The same is for `ruby 2.0.0p643 (2015-02-25) [i386-mingw32]` .

I also checked that require fails in the same case for
`ruby 2.1.9p490 (2016-03-30 revision 54437) [i386-mingw32]`



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [ruby-core:104317] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
       [not found] <redmine.issue-15993.20190710194105.25466@ruby-lang.org>
  2020-09-29 14:04 ` [ruby-core:100214] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir tschoening
@ 2021-06-16 17:53 ` merch-redmine
  2021-07-09 11:12 ` [ruby-core:104567] " yura.des
  2021-07-09 15:52 ` [ruby-core:104569] " merch-redmine
  3 siblings, 0 replies; 4+ messages in thread
From: merch-redmine @ 2021-06-16 17:53 UTC (permalink / raw
  To: ruby-core

Issue #15993 has been updated by jeremyevans0 (Jeremy Evans).

Status changed from Open to Closed

This appears to be fixed starting in Ruby 2.7 (also works in 3.0):

```
D:\Евгений>C:\Ruby26-x64\bin\ruby -I D:\Евгений -e "require 'logger'"
Traceback (most recent call last):
        2: from -e:1:in `<main>'
        1: from C:/Ruby26-x64/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
C:/Ruby26-x64/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': No such file or directory -- D:/Евгений/logger.rb (LoadError)

D:\Евгений>C:\Ruby27-x64\bin\ruby -I D:\Евгений -e "require 'logger'"

D:\Евгений>C:\Ruby30-x64\bin\ruby -I D:\Евгений -e "require 'logger'"
```

As Ruby 2.6 is in security maintenance mode, the change will not be backported.

----------------------------------------
Bug #15993: 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
https://bugs.ruby-lang.org/issues/15993#change-92540

* Author: inversion (Yura Babak)
* Status: Closed
* Priority: Normal
* ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
I’m trying to build a cross-platform portable application with Ruby onboard and there is a problem on Windows. 
A user usually installs it to the Roaming folder which sits inside a user folder which can often have not a Latin name or contain spaces). 
When there is a Cyrillic character (maybe just not Latin) in the path — require of any gem doesn’t work:

```
D:\users\киї\Ruby\2.6\bin>ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]

D:\users\киї\Ruby\2.6\bin>ruby -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>ruby --disable=rubyopt -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>gem list
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

```

We can see such encoding transformations in the output:

```
киї (utf-8) == РєРёС— (win1251)
```

I have an old Ruby installation that works fine:

```
D:\users\киї\Ruby\2.0\bin>ruby -e "require 'logger'"

D:\users\киї\Ruby\2.0\bin>ruby -v
ruby 2.0.0p451 (2014-02-24) [i386-mingw32]
```
The same is for `ruby 2.0.0p643 (2015-02-25) [i386-mingw32]` .

I also checked that require fails in the same case for
`ruby 2.1.9p490 (2016-03-30 revision 54437) [i386-mingw32]`



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [ruby-core:104567] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
       [not found] <redmine.issue-15993.20190710194105.25466@ruby-lang.org>
  2020-09-29 14:04 ` [ruby-core:100214] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir tschoening
  2021-06-16 17:53 ` [ruby-core:104317] " merch-redmine
@ 2021-07-09 11:12 ` yura.des
  2021-07-09 15:52 ` [ruby-core:104569] " merch-redmine
  3 siblings, 0 replies; 4+ messages in thread
From: yura.des @ 2021-07-09 11:12 UTC (permalink / raw
  To: ruby-core

Issue #15993 has been updated by inversion (Yura Babak).


jeremyevans0 (Jeremy Evans) wrote in #note-7:
> This appears to be fixed starting in Ruby 2.7 (also works in 3.0):
 
**Still, there is a problem.**
`require 'bundler/setup'` fails if `LOAD_PATH` or `Gem.dir` contain Cyrillic chars, the error is similar to:
```
incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
```

From the trace I have prepared the **minimum reproducible case** :
1. Put Ruby in a location where the path will contain Cyrillic chars, like `"D:\users\киї\Ruby"`
1. Prepare 2 files (saved in UTF-8 encoding) somewhere in a location where the path will contain Cyrillic chars (can be near that Ruby):
   https://gist.github.com/Inversion-des/75949795cc5be707c19d31901e79d1cf
1. Open cmd and ensure to do `chcp 1251` in the current console session.
1. run `"[this Ruby path]" f1.rb`

You will see that the same `__dir__` output is different between files (f2 is required). If you will try to run f2.rb — output will be the same as for f1. So, **require_relative somehow changes the encoding** here.

To emulate problems with the 'bundler/setup' there are next lines:
``` ruby
# fails: incompatible character encodings: Windows-1251 and UTF-8 (Encoding::CompatibilityError)
p start_with:$LOAD_PATH[0].start_with?(__dir__)
# fails: incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
p start_with:$LOAD_PATH[0].start_with?(Gem.dir)
```

To see the real problem you should comment these lines and also prepare next files (I'm not sure content is important by add at least one gem there)
* Gemfile
* Gemfile.lock

And to see both problems there should also be the `.bundle\config` file with a line like:
`BUNDLE_PATH: "../platform/Ruby_gems"`

In the `bundler\settings.rb` it will use `explicit_path` if the `BUNDLE_PATH` defined and `Bundler.rubygems.gem_dir` otherwise.


Workaround to overcome both errors you can find in the f1.rb in the related commented section:
``` ruby
Gem.dir.force_encoding 'UTF-8'
Gem.path.each {|path| path.force_encoding 'UTF-8' }
if $:[0].encoding.name == 'Windows-1251'
    $:.each {|path| path.encode! 'UTF-8' }
    $:.push '.'    # somehow it helps, looks like a modification of array is needed
end

```

My environment:
* Windows10 Pro
* Ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x64-mingw32]
* Bundler version 2.2.22
* RubyGems version 3.2.22

----------------------------------------
Bug #15993: 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
https://bugs.ruby-lang.org/issues/15993#change-92846

* Author: inversion (Yura Babak)
* Status: Closed
* Priority: Normal
* ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
I’m trying to build a cross-platform portable application with Ruby onboard and there is a problem on Windows. 
A user usually installs it to the Roaming folder which sits inside a user folder which can often have not a Latin name or contain spaces). 
When there is a Cyrillic character (maybe just not Latin) in the path — require of any gem doesn’t work:

```
D:\users\киї\Ruby\2.6\bin>ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]

D:\users\киї\Ruby\2.6\bin>ruby -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>ruby --disable=rubyopt -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>gem list
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

```

We can see such encoding transformations in the output:

```
киї (utf-8) == РєРёС— (win1251)
```

I have an old Ruby installation that works fine:

```
D:\users\киї\Ruby\2.0\bin>ruby -e "require 'logger'"

D:\users\киї\Ruby\2.0\bin>ruby -v
ruby 2.0.0p451 (2014-02-24) [i386-mingw32]
```
The same is for `ruby 2.0.0p643 (2015-02-25) [i386-mingw32]` .

I also checked that require fails in the same case for
`ruby 2.1.9p490 (2016-03-30 revision 54437) [i386-mingw32]`



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [ruby-core:104569] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
       [not found] <redmine.issue-15993.20190710194105.25466@ruby-lang.org>
                   ` (2 preceding siblings ...)
  2021-07-09 11:12 ` [ruby-core:104567] " yura.des
@ 2021-07-09 15:52 ` merch-redmine
  3 siblings, 0 replies; 4+ messages in thread
From: merch-redmine @ 2021-07-09 15:52 UTC (permalink / raw
  To: ruby-core

Issue #15993 has been updated by jeremyevans0 (Jeremy Evans).

Status changed from Closed to Open

inversion (Yura Babak) wrote in #note-8:
> jeremyevans0 (Jeremy Evans) wrote in #note-7:
> > This appears to be fixed starting in Ruby 2.7 (also works in 3.0):
>  
> **Still, there is a problem.**
> `require 'bundler/setup'` fails if `LOAD_PATH` or `Gem.dir` contain Cyrillic chars, the error is similar to:
> ```
> incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
> ```
> 
> From the trace I have prepared the **minimum reproducible case** :
> 1. Put Ruby in a location where the path will contain Cyrillic chars, like `"D:\users\киї\Ruby"`
> 1. Prepare 2 files (saved in UTF-8 encoding) somewhere in a location where the path will contain Cyrillic chars (can be near that Ruby):
>    https://gist.github.com/Inversion-des/75949795cc5be707c19d31901e79d1cf
> 1. Open cmd and ensure to do `chcp 1251` in the current console session.
> 1. run `"[this Ruby path]" f1.rb`

I was able to reproduce the issue, but only when I installed Ruby into a path not supported by the `Windows-1251` encoding:

```
d:\Евгений>d:\zz-können2\Ruby31-x64\bin\bundle install --local
d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:94:in `expand_path': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:94:in `expand_path'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:94:in `bundle_path'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:682:in `configure_gem_home'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:663:in `configure_gem_home_and_path'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:80:in `configure'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:193:in `definition'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli/install.rb:57:in `run'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:259:in `block in install'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/settings.rb:133:in `temporary'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:258:in `install'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:30:in `dispatch'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:24:in `start'
        from d:/zz-können2/Ruby31-x64/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:49:in `block in <top (required)>'
        from d:/zz-können2/Ruby31-x64/lib/ruby/3.1.0/bundler/friendly_errors.rb:130:in `with_friendly_errors'
        from d:/zz-können2/Ruby31-x64/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:37:in `<top (required)>'
        from d:/zz-k?nnen2/Ruby31-x64/bin/bundle:31:in `load'
        from d:/zz-k?nnen2/Ruby31-x64/bin/bundle:31:in `<main>'
```

Part of the underlying issue seems to be that `__FILE__` and `__dir__` are not UTF-8 encoded for the main script, unlike required files.  I'm not sure if changing that alone will fix the issue, though.

When I run the following script (`f3.rb`):

```
p ['__FILE__', __FILE__, __FILE__.encoding]
p ['__dir__', __dir__, __dir__.encoding]
p ['Gem.dir', Gem.dir, Gem.dir.encoding]
puts 'Gem.path'
Gem.path.each do |s|
  p [s, s.encoding]
end
puts '$:'
$:.each do |s|
  p [s, s.encoding]
end
```

I get the following when using Ruby installed in a non-ASCII path:

```
d:\Евгений>d:\zz-können2\Ruby31-x64\bin\ruby D:\Евгений\f3.rb
["__FILE__", "D:/\xC5\xE2\xE3\xE5\xED\xE8\xE9/f3.rb", #<Encoding:Windows-1251>]
["__dir__", "D:/\xC5\xE2\xE3\xE5\xED\xE8\xE9", #<Encoding:Windows-1251>]
["Gem.dir", "d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #<Encoding:ASCII-8BIT>]
Gem.path
["C:/Users/jeremye/.gem/ruby/3.1.0", #<Encoding:UTF-8>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #<Encoding:ASCII-8BIT>]
$:
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0/x64-ucrt", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0/x64-ucrt", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0/x64-mingw-ucrt", #<Encoding:ASCII-8BIT>]
```

and when installed into an ASCII path:

```
d:\Евгений>C:\Ruby30-x64\bin\ruby d:\Евгений\f3.rb
["__FILE__", "d:/\xC5\xE2\xE3\xE5\xED\xE8\xE9/f3.rb", #<Encoding:Windows-1251>]
["__dir__", "d:/\xC5\xE2\xE3\xE5\xED\xE8\xE9", #<Encoding:Windows-1251>]
["Gem.dir", "C:/Ruby30-x64/lib/ruby/gems/3.0.0", #<Encoding:ASCII-8BIT>]
Gem.path
["C:/Users/jeremye/.gem/ruby/3.0.0", #<Encoding:UTF-8>]
["C:/Ruby30-x64/lib/ruby/gems/3.0.0", #<Encoding:ASCII-8BIT>]
$:
["C:/Ruby30-x64/lib/ruby/site_ruby/3.0.0", #<Encoding:Windows-1251>]
["C:/Ruby30-x64/lib/ruby/site_ruby/3.0.0/x64-msvcrt", #<Encoding:Windows-1251>]
["C:/Ruby30-x64/lib/ruby/site_ruby", #<Encoding:Windows-1251>]
["C:/Ruby30-x64/lib/ruby/vendor_ruby/3.0.0", #<Encoding:Windows-1251>]
["C:/Ruby30-x64/lib/ruby/vendor_ruby/3.0.0/x64-msvcrt", #<Encoding:Windows-1251>]
["C:/Ruby30-x64/lib/ruby/vendor_ruby", #<Encoding:Windows-1251>]
["C:/Ruby30-x64/lib/ruby/3.0.0", #<Encoding:Windows-1251>]
["C:/Ruby30-x64/lib/ruby/3.0.0/x64-mingw32", #<Encoding:Windows-1251>]
```

It looks like the difference in the non-ASCII path case is that `ASCII-8BIT` encoding is used even if the path itself is valid UTF-8.  This is true even if you force a UTF-8 code page (though that does fix `__FILE__` and `__dir__`):

```
d:\Евгений>chcp 65001
Active code page: 65001

d:\Евгений>d:\zz-können2\Ruby31-x64\bin\ruby D:\Евгений\f3.rb
["__FILE__", "D:/Евгений/f3.rb", #<Encoding:UTF-8>]
["__dir__", "D:/Евгений", #<Encoding:UTF-8>]
["Gem.dir", "d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #<Encoding:ASCII-8BIT>]
Gem.path
["C:/Users/jeremye/.gem/ruby/3.1.0", #<Encoding:UTF-8>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #<Encoding:ASCII-8BIT>]
$:
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0/x64-ucrt", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0/x64-ucrt", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0", #<Encoding:ASCII-8BIT>]
["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0/x64-mingw-ucrt", #<Encoding:ASCII-8BIT>]
```

Since there does appear to be an issue, I'll reopen this.  Hopefully someone with more knowledge in this area can suggest a possible fix.

----------------------------------------
Bug #15993: 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
https://bugs.ruby-lang.org/issues/15993#change-92849

* Author: inversion (Yura Babak)
* Status: Open
* Priority: Normal
* ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
I’m trying to build a cross-platform portable application with Ruby onboard and there is a problem on Windows. 
A user usually installs it to the Roaming folder which sits inside a user folder which can often have not a Latin name or contain spaces). 
When there is a Cyrillic character (maybe just not Latin) in the path — require of any gem doesn’t work:

```
D:\users\киї\Ruby\2.6\bin>ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]

D:\users\киї\Ruby\2.6\bin>ruby -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>ruby --disable=rubyopt -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>gem list
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

```

We can see such encoding transformations in the output:

```
киї (utf-8) == РєРёС— (win1251)
```

I have an old Ruby installation that works fine:

```
D:\users\киї\Ruby\2.0\bin>ruby -e "require 'logger'"

D:\users\киї\Ruby\2.0\bin>ruby -v
ruby 2.0.0p451 (2014-02-24) [i386-mingw32]
```
The same is for `ruby 2.0.0p643 (2015-02-25) [i386-mingw32]` .

I also checked that require fails in the same case for
`ruby 2.1.9p490 (2016-03-30 revision 54437) [i386-mingw32]`



-- 
https://bugs.ruby-lang.org/

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-07-09 15:52 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <redmine.issue-15993.20190710194105.25466@ruby-lang.org>
2020-09-29 14:04 ` [ruby-core:100214] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir tschoening
2021-06-16 17:53 ` [ruby-core:104317] " merch-redmine
2021-07-09 11:12 ` [ruby-core:104567] " yura.des
2021-07-09 15:52 ` [ruby-core:104569] " merch-redmine

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).