From: "Eregon (Benoit Daloze) via ruby-core" <ruby-core@ml.ruby-lang.org>
To: ruby-core@ml.ruby-lang.org
Cc: "Eregon (Benoit Daloze)" <noreply@ruby-lang.org>
Subject: [ruby-core:117441] [Ruby master Misc#20406] Question about Regexp encoding negotiation
Date: Thu, 04 Apr 2024 11:35:56 +0000 (UTC) [thread overview]
Message-ID: <redmine.journal-107821.20240404113556.13553@ruby-lang.org> (raw)
In-Reply-To: redmine.issue-20406.20240402171943.13553@ruby-lang.org
Issue #20406 has been updated by Eregon (Benoit Daloze).
Indeed, on a similar topic I wonder how much encoding negotiation at Regexp creation time matters.
Because there is another encoding negotiation between the regexp and the string being matched which happens when matching.
Maybe the Regexp encoding should e.g. always be US-ASCII if there are only 7-bit characters in the Regexp source,
or maybe always UTF-8 in that case since it's most likely a regexp will be matched against UTF-8 strings,
this illustrates the Regexp encoding doesn't really matter for the 7-bit source case.
Or maybe Regexp literals should just always use the source encoding, that would make things a lot simpler and closer to string literals.
And the `/nesu` flag would just override the source encoding (and maybe be eventually deprecated, but probably not worth it if their semantics are clear).
I'm not sure what's the point of `Regexp#fixed_encoding?` either, it seems regardless of it a Regexp can be matched with strings of different but compatible encodings (the docs about this in `ri Regexp` are incorrect).
----------------------------------------
Misc #20406: Question about Regexp encoding negotiation
https://bugs.ruby-lang.org/issues/20406#change-107821
* Author: andrykonchin (Andrew Konchin)
* Status: Open
----------------------------------------
I am wondering what are the rules to calculate Regexp literal encoding in case an encoding modifier is specified.
From the documentstion:
> By default, a regexp with only US-ASCII characters has US-ASCII encoding:
> ...
> A regular expression containing non-US-ASCII characters is assumed to use the source encoding. This can be overridden with one of the following modifiers.
> //n ...
> //u ...
> //e ...
> //s ...
Looking at the following examples I would assume that these rules are followed except one case:
```ruby
p /\xc2\xa1/e .encoding # EUC-JP
p /#{ }\xc2\xa1/e .encoding # EUC-JP
p /a/e .encoding # EUC-JP
p /a #{} a/e .encoding # EUC-JP
p /#{} a/e .encoding # US-ASCII
```
The last Regexp `/#{} a/e` is supposed to have `EUC-JP` encoding but has `US-ASCII`. So I am wondering what rule is applied in this case.
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/
prev parent reply other threads:[~2024-04-04 11:36 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-02 17:19 [ruby-core:117408] [Ruby master Misc#20406] Question about Regexp encoding negotiation andrykonchin (Andrew Konchin) via ruby-core
2024-04-03 6:10 ` [ruby-core:117421] " shyouhei (Shyouhei Urabe) via ruby-core
2024-04-03 9:57 ` [ruby-core:117426] " Eregon (Benoit Daloze) via ruby-core
2024-04-03 10:29 ` [ruby-core:117427] " Eregon (Benoit Daloze) via ruby-core
2024-04-03 10:33 ` [ruby-core:117428] " Eregon (Benoit Daloze) via ruby-core
2024-04-04 0:09 ` [ruby-core:117437] " duerst via ruby-core
2024-04-04 11:35 ` Eregon (Benoit Daloze) via ruby-core [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.ruby-lang.org/en/community/mailing-lists/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=redmine.journal-107821.20240404113556.13553@ruby-lang.org \
--to=ruby-core@ruby-lang.org \
--cc=noreply@ruby-lang.org \
--cc=ruby-core@ml.ruby-lang.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).