From: "nobu (Nobuyoshi Nakada)" <noreply@ruby-lang.org>
To: ruby-core@ruby-lang.org
Subject: [ruby-core:105534] [Ruby master Feature#14975] String#append without changing receiver's encoding
Date: Mon, 04 Oct 2021 00:38:07 +0000 (UTC) [thread overview]
Message-ID: <redmine.journal-93991.20211004003801.3344@ruby-lang.org> (raw)
In-Reply-To: redmine.issue-14975.20180808031535.3344@ruby-lang.org
Issue #14975 has been updated by nobu (Nobuyoshi Nakada).
Status changed from Open to Rejected
Closing since this seems not acceptable.
Feel free to reopen if any progress.
----------------------------------------
Feature #14975: String#append without changing receiver's encoding
https://bugs.ruby-lang.org/issues/14975#change-93991
* Author: ioquatix (Samuel Williams)
* Status: Rejected
* Priority: Normal
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
I'm not sure where this fits in, but in order to avoid garbage and superfluous function calls, is it possible that `String#<<`, `String#concat` or the (proposed) `String#append` can avoid changing the encoding of the receiver?
Right now it's very tricky to do this in a way that doesn't require extra allocations. Here is what I do:
```ruby
class Buffer < String
BINARY = Encoding::BINARY
def initialize
super
force_encoding(BINARY)
end
def << string
if string.encoding == BINARY
super(string)
else
super(string.b) # Requires extra allocation.
end
return self
end
alias concat <<
end
```
When the receiver is binary, but contains byte sequences, appending UTF_8 can fail:
```
"Foobar".b << "Føøbar"
=> "FoobarFøøbar"
> "Føøbar".b << "Føøbar"
Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8
```
So, it's not possible to append data, generally, and then call `force_encoding(Encoding::BINARY)`. One must ensure the string is binary before appending it.
It would be nice if there was a solution which didn't require additional allocations/copies/linear scans for what should basically be a `memcpy`.
See also: https://bugs.ruby-lang.org/issues/14033 and https://bugs.ruby-lang.org/issues/13626#note-3
There are two options to fix this:
1/ Don't change receiver encoding in any case.
2/ Apply 1, but only when receiver is using `Encoding::BINARY`
--
https://bugs.ruby-lang.org/
next parent reply other threads:[~2021-10-04 0:38 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <redmine.issue-14975.20180808031535.3344@ruby-lang.org>
2021-10-04 0:38 ` nobu (Nobuyoshi Nakada) [this message]
2021-12-21 19:22 ` [ruby-core:106765] [Ruby master Feature#14975] String#append without changing receiver's encoding ioquatix (Samuel Williams)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.ruby-lang.org/en/community/mailing-lists/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=redmine.journal-93991.20211004003801.3344@ruby-lang.org \
--to=ruby-core@ruby-lang.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).