[ruby-core:95196] [Ruby master Bug#16233] winruby UTF8 Fallback for no CodePage

[ruby-core:95196] [Ruby master Bug#16233] winruby UTF8 Fallback for no CodePage

[gabrielnagy]

Issue #16233 has been reported by gabuscus (Gabriel Nagy).

----------------------------------------
Bug #16233: winruby UTF8 Fallback for no CodePage
https://bugs.ruby-lang.org/issues/16233

* Author: gabuscus (Gabriel Nagy)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.3
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
This is to deal specifically with setting the Region to Arabic which
sets the codepage to 720. This codepage isn't recognised by Ruby which
causes it to fail during initialisation when setting up LOCALE.

This patch to Ruby is necessary to handle languages/regions on windows
where the codepage is not supported by Ruby such as Arabic which uses 
codepage 720.
If the codepage is not found, the Locale falls back to UTF8.

This is a well known Ruby/Ruby on Rails issue which is described at
https://stackoverflow.com/questions/22815542/rails4-unknown-encoding-name-cp720

---Files--------------------------------
windows_nocodepage_utf8_fallback_r2.5.patch (946 Bytes)


-- 
https://bugs.ruby-lang.org/

[ruby-core:95215] [Ruby master Bug#16233] winruby UTF8 Fallback for no CodePage

[duerst]

Issue #16233 has been updated by duerst (Martin Dürst).


If codepage 720 is not supported, then we should look at supporting it.

----------------------------------------
Bug #16233: winruby UTF8 Fallback for no CodePage
https://bugs.ruby-lang.org/issues/16233#change-81889

* Author: gabuscus (Gabriel Nagy)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.3
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
This is to deal specifically with setting the Region to Arabic which
sets the codepage to 720. This codepage isn't recognised by Ruby which
causes it to fail during initialisation when setting up LOCALE.

This patch to Ruby is necessary to handle languages/regions on windows
where the codepage is not supported by Ruby such as Arabic which uses 
codepage 720.
If the codepage is not found, the Locale falls back to UTF8.

This is a well known Ruby/Ruby on Rails issue which is described at
https://stackoverflow.com/questions/22815542/rails4-unknown-encoding-name-cp720

Opened PR: https://github.com/ruby/ruby/pull/2518

---Files--------------------------------
windows_nocodepage_utf8_fallback_r2.5.patch (946 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:95217] [Ruby master Bug#16233] winruby UTF8 Fallback for no CodePage

[gabrielnagy]

Issue #16233 has been updated by gabuscus (Gabriel Nagy).


duerst (Martin Dürst) wrote:
> If codepage 720 is not supported, then we should look at supporting it.

fair point, however investigation on this Puppet ticket: https://tickets.puppetlabs.com/browse/PA-2191 concluded that falling back to UTF-8 fixes the issue (for codepage 720 at least). we could also check the codepage when catching the ArgumentError so we fallback only in that case

----------------------------------------
Bug #16233: winruby UTF8 Fallback for no CodePage
https://bugs.ruby-lang.org/issues/16233#change-81890

* Author: gabuscus (Gabriel Nagy)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.6.3
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
This is to deal specifically with setting the Region to Arabic which
sets the codepage to 720. This codepage isn't recognised by Ruby which
causes it to fail during initialisation when setting up LOCALE.

This patch to Ruby is necessary to handle languages/regions on windows
where the codepage is not supported by Ruby such as Arabic which uses 
codepage 720.
If the codepage is not found, the Locale falls back to UTF8.

This is a well known Ruby/Ruby on Rails issue which is described at
https://stackoverflow.com/questions/22815542/rails4-unknown-encoding-name-cp720

Opened PR: https://github.com/ruby/ruby/pull/2518

---Files--------------------------------
windows_nocodepage_utf8_fallback_r2.5.patch (946 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:96843] [Ruby master Feature#16233] winruby UTF8 Fallback for no CodePage

[naruse]

Issue #16233 has been updated by naruse (Yui NARUSE).

Backport deleted (2.5: UNKNOWN, 2.6: UNKNOWN)
ruby -v deleted (2.6.3)
Target version set to 2.8
Assignee set to duerst (Martin Dürst)
Tracker changed from Bug to Feature

It looks Ruby should support Windows-1256 with alias CP720.
https://en.wikipedia.org/wiki/Windows-1256

@duerst Could you add support for it?

----------------------------------------
Feature #16233: winruby UTF8 Fallback for no CodePage
https://bugs.ruby-lang.org/issues/16233#change-83847

* Author: gabuscus (Gabriel Nagy)
* Status: Open
* Priority: Normal
* Assignee: duerst (Martin Dürst)
* Target version: 2.8
----------------------------------------
This is to deal specifically with setting the Region to Arabic which
sets the codepage to 720. This codepage isn't recognised by Ruby which
causes it to fail during initialisation when setting up LOCALE.

This patch to Ruby is necessary to handle languages/regions on windows
where the codepage is not supported by Ruby such as Arabic which uses 
codepage 720.
If the codepage is not found, the Locale falls back to UTF8.

This is a well known Ruby/Ruby on Rails issue which is described at
https://stackoverflow.com/questions/22815542/rails4-unknown-encoding-name-cp720

Opened PR: https://github.com/ruby/ruby/pull/2518

---Files--------------------------------
windows_nocodepage_utf8_fallback_r2.5.patch (946 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:96847] [Ruby master Feature#16233] winruby UTF8 Fallback for no CodePage

[duerst]

Issue #16233 has been updated by duerst (Martin Dürst).


Will do, but probably not very soon, because it's now the most busy time of the year at work, sorry.

Some notes: For the encoding (onigX), windows-1256 is a replica of iso-8859-6. But windows-1256 is much more complete, while iso-8859-6 has many unassigned codepoints, so it may be a good idea to fix this if we get around to it.

Just adding an alias 'CP720' for windows-1256 is much easier. 

----------------------------------------
Feature #16233: winruby UTF8 Fallback for no CodePage
https://bugs.ruby-lang.org/issues/16233#change-83851

* Author: gabuscus (Gabriel Nagy)
* Status: Open
* Priority: Normal
* Assignee: duerst (Martin Dürst)
* Target version: 2.8
----------------------------------------
This is to deal specifically with setting the Region to Arabic which
sets the codepage to 720. This codepage isn't recognised by Ruby which
causes it to fail during initialisation when setting up LOCALE.

This patch to Ruby is necessary to handle languages/regions on windows
where the codepage is not supported by Ruby such as Arabic which uses 
codepage 720.
If the codepage is not found, the Locale falls back to UTF8.

This is a well known Ruby/Ruby on Rails issue which is described at
https://stackoverflow.com/questions/22815542/rails4-unknown-encoding-name-cp720

Opened PR: https://github.com/ruby/ruby/pull/2518

---Files--------------------------------
windows_nocodepage_utf8_fallback_r2.5.patch (946 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>