* [ruby-core:105534] [Ruby master Feature#14975] String#append without changing receiver's encoding
[not found] <redmine.issue-14975.20180808031535.3344@ruby-lang.org>
@ 2021-10-04 0:38 ` nobu (Nobuyoshi Nakada)
2021-12-21 19:22 ` [ruby-core:106765] " ioquatix (Samuel Williams)
1 sibling, 0 replies; 2+ messages in thread
From: nobu (Nobuyoshi Nakada) @ 2021-10-04 0:38 UTC (permalink / raw)
To: ruby-core
Issue #14975 has been updated by nobu (Nobuyoshi Nakada).
Status changed from Open to Rejected
Closing since this seems not acceptable.
Feel free to reopen if any progress.
----------------------------------------
Feature #14975: String#append without changing receiver's encoding
https://bugs.ruby-lang.org/issues/14975#change-93991
* Author: ioquatix (Samuel Williams)
* Status: Rejected
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
I'm not sure where this fits in, but in order to avoid garbage and superfluous function calls, is it possible that `String#<<`, `String#concat` or the (proposed) `String#append` can avoid changing the encoding of the receiver?
Right now it's very tricky to do this in a way that doesn't require extra allocations. Here is what I do:
```ruby
class Buffer < String
BINARY = Encoding::BINARY
def initialize
super
force_encoding(BINARY)
end
def << string
if string.encoding == BINARY
super(string)
else
super(string.b) # Requires extra allocation.
end
return self
end
alias concat <<
end
```
When the receiver is binary, but contains byte sequences, appending UTF_8 can fail:
```
"Foobar".b << "Føøbar"
=> "FoobarFøøbar"
> "Føøbar".b << "Føøbar"
Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8
```
So, it's not possible to append data, generally, and then call `force_encoding(Encoding::BINARY)`. One must ensure the string is binary before appending it.
It would be nice if there was a solution which didn't require additional allocations/copies/linear scans for what should basically be a `memcpy`.
See also: https://bugs.ruby-lang.org/issues/14033 and https://bugs.ruby-lang.org/issues/13626#note-3
There are two options to fix this:
1/ Don't change receiver encoding in any case.
2/ Apply 1, but only when receiver is using `Encoding::BINARY`
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 2+ messages in thread
* [ruby-core:106765] [Ruby master Feature#14975] String#append without changing receiver's encoding
[not found] <redmine.issue-14975.20180808031535.3344@ruby-lang.org>
2021-10-04 0:38 ` [ruby-core:105534] [Ruby master Feature#14975] String#append without changing receiver's encoding nobu (Nobuyoshi Nakada)
@ 2021-12-21 19:22 ` ioquatix (Samuel Williams)
1 sibling, 0 replies; 2+ messages in thread
From: ioquatix (Samuel Williams) @ 2021-12-21 19:22 UTC (permalink / raw)
To: ruby-core
Issue #14975 has been updated by ioquatix (Samuel Williams).
`IO::Buffer` makes this less important.
----------------------------------------
Feature #14975: String#append without changing receiver's encoding
https://bugs.ruby-lang.org/issues/14975#change-95469
* Author: ioquatix (Samuel Williams)
* Status: Rejected
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
I'm not sure where this fits in, but in order to avoid garbage and superfluous function calls, is it possible that `String#<<`, `String#concat` or the (proposed) `String#append` can avoid changing the encoding of the receiver?
Right now it's very tricky to do this in a way that doesn't require extra allocations. Here is what I do:
```ruby
class Buffer < String
BINARY = Encoding::BINARY
def initialize
super
force_encoding(BINARY)
end
def << string
if string.encoding == BINARY
super(string)
else
super(string.b) # Requires extra allocation.
end
return self
end
alias concat <<
end
```
When the receiver is binary, but contains byte sequences, appending UTF_8 can fail:
```
"Foobar".b << "Føøbar"
=> "FoobarFøøbar"
> "Føøbar".b << "Føøbar"
Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8
```
So, it's not possible to append data, generally, and then call `force_encoding(Encoding::BINARY)`. One must ensure the string is binary before appending it.
It would be nice if there was a solution which didn't require additional allocations/copies/linear scans for what should basically be a `memcpy`.
See also: https://bugs.ruby-lang.org/issues/14033 and https://bugs.ruby-lang.org/issues/13626#note-3
There are two options to fix this:
1/ Don't change receiver encoding in any case.
2/ Apply 1, but only when receiver is using `Encoding::BINARY`
--
https://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-12-21 19:22 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <redmine.issue-14975.20180808031535.3344@ruby-lang.org>
2021-10-04 0:38 ` [ruby-core:105534] [Ruby master Feature#14975] String#append without changing receiver's encoding nobu (Nobuyoshi Nakada)
2021-12-21 19:22 ` [ruby-core:106765] " ioquatix (Samuel Williams)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).