[ruby-core:43557] [ruby-trunk - Bug #6190][Open] String#encode return string containing invalid chars but marked as valid

[ruby-core:43557] [ruby-trunk - Bug #6190][Open] String#encode return string containing invalid chars but marked as valid

[pplr (Pierre PLR)]

Issue #6190 has been reported by pplr (Pierre PLR).

----------------------------------------
Bug #6190: String#encode return string containing invalid chars but marked as valid
https://bugs.ruby-lang.org/issues/6190

Author: pplr (Pierre PLR)
Status: Open
Priority: Normal
Assignee: 
Category: 
Target version: 
ruby -v: ruby 1.9.2p290 (2011-07-09 revision 32553) [i686-linux]



 >> a = " \xE9 ".encode('UTF-8', 'UTF-8', :invalid => :replace, :replace => "?")
 >> a.valid_encoding?
 => true
 >> a
 => " \xE9 "
 >> a.squeeze
 ArgumentError: invalid byte sequence in UTF-8
 	from (irb):32:in `squeeze'
 	from (irb):32
 	from /usr/bin/irb:12:in `<main>'

The expected string is " ? ", as the documentation for the ":replace" options says :
If the value is :replace, encode replaces invalid byte sequences in str with the replacement character.



-- 
http://bugs.ruby-lang.org/

[ruby-core:43571] [ruby-trunk - Bug #6190] String#encode return string containing invalid chars but marked as valid

[duerst (Martin Dürst)]

Issue #6190 has been updated by duerst (Martin Dürst).

Description updated

pplr (Pierre PLR) wrote:
> >> a = " \xE9 ".encode('UTF-8', 'UTF-8', :invalid => :replace, :replace => "?")
>  >> a.valid_encoding?
>  => true

Nobu fixed this so it won't return true anymore, which would be a lie.


>  >> a
>  => " \xE9 "

> The expected string is " ? ", as the documentation for the ":replace" options says :
> If the value is :replace, encode replaces invalid byte sequences in str with the replacement character.

I added documentation to say that encoding from encoding A to the same encoding A is a no-op. Changing this would not be impossible, but would involve quite some work, and would make these operations slower.
----------------------------------------
Bug #6190: String#encode return string containing invalid chars but marked as valid
https://bugs.ruby-lang.org/issues/6190#change-25066

Author: pplr (Pierre PLR)
Status: Closed
Priority: Normal
Assignee: 
Category: 
Target version: 
ruby -v: ruby 1.9.2p290 (2011-07-09 revision 32553) [i686-linux]


 >> a = " \xE9 ".encode('UTF-8', 'UTF-8', :invalid => :replace, :replace => "?")
 >> a.valid_encoding?
 => true
 >> a
 => " \xE9 "
 >> a.squeeze
 ArgumentError: invalid byte sequence in UTF-8
 	from (irb):32:in `squeeze'
 	from (irb):32
 	from /usr/bin/irb:12:in `<main>'

The expected string is " ? ", as the documentation for the ":replace" options says :
If the value is :replace, encode replaces invalid byte sequences in str with the replacement character.



-- 
http://bugs.ruby-lang.org/

[ruby-core:52743] [Backport93 - Backport #6190][Assigned] String#encode return string containing invalid chars but marked as valid

[naruse (Yui NARUSE)]

Issue #6190 has been updated by naruse (Yui NARUSE).

Status changed from Closed to Assigned


----------------------------------------
Backport #6190: String#encode return string containing invalid chars but marked as valid
https://bugs.ruby-lang.org/issues/6190#change-36833

Author: pplr (Pierre PLR)
Status: Assigned
Priority: Normal
Assignee: 
Category: 
Target version: 


 >> a = " \xE9 ".encode('UTF-8', 'UTF-8', :invalid => :replace, :replace => "?")
 >> a.valid_encoding?
 => true
 >> a
 => " \xE9 "
 >> a.squeeze
 ArgumentError: invalid byte sequence in UTF-8
 	from (irb):32:in `squeeze'
 	from (irb):32
 	from /usr/bin/irb:12:in `<main>'

The expected string is " ? ", as the documentation for the ":replace" options says :
If the value is :replace, encode replaces invalid byte sequences in str with the replacement character.



-- 
http://bugs.ruby-lang.org/

[ruby-core:52747] [Backport93 - Backport #6190] String#encode return string containing invalid chars but marked as valid

[zzak (Zachary Scott)]

Issue #6190 has been updated by zzak (Zachary Scott).

Assignee set to naruse (Yui NARUSE)

naruse-san what do you want for this ticket?
----------------------------------------
Backport #6190: String#encode return string containing invalid chars but marked as valid
https://bugs.ruby-lang.org/issues/6190#change-36835

Author: pplr (Pierre PLR)
Status: Assigned
Priority: Normal
Assignee: naruse (Yui NARUSE)
Category: 
Target version: 


 >> a = " \xE9 ".encode('UTF-8', 'UTF-8', :invalid => :replace, :replace => "?")
 >> a.valid_encoding?
 => true
 >> a
 => " \xE9 "
 >> a.squeeze
 ArgumentError: invalid byte sequence in UTF-8
 	from (irb):32:in `squeeze'
 	from (irb):32
 	from /usr/bin/irb:12:in `<main>'

The expected string is " ? ", as the documentation for the ":replace" options says :
If the value is :replace, encode replaces invalid byte sequences in str with the replacement character.



-- 
http://bugs.ruby-lang.org/