From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS4713 221.184.0.0/13 X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from neon.ruby-lang.org (neon.ruby-lang.org [221.186.184.75]) by dcvr.yhbt.net (Postfix) with ESMTP id 0F1C81F461 for ; Wed, 15 May 2019 05:13:51 +0000 (UTC) Received: from neon.ruby-lang.org (localhost [IPv6:::1]) by neon.ruby-lang.org (Postfix) with ESMTP id BABA912082F; Wed, 15 May 2019 14:13:43 +0900 (JST) Received: from o1678916x28.outbound-mail.sendgrid.net (o1678916x28.outbound-mail.sendgrid.net [167.89.16.28]) by neon.ruby-lang.org (Postfix) with ESMTPS id E772C120A10 for ; Wed, 15 May 2019 14:13:40 +0900 (JST) Received: by filter0094p3las1.sendgrid.net with SMTP id filter0094p3las1-2370-5CDBA005-1F 2019-05-15 05:13:41.722347364 +0000 UTC m=+126473.386738825 Received: from herokuapp.com (unknown [34.228.185.226]) by ismtpd0019p1iad2.sendgrid.net (SG) with ESMTP id Rr6wMAdTTI2ZhrPlr7iNGA for ; Wed, 15 May 2019 05:13:41.448 +0000 (UTC) Date: Wed, 15 May 2019 05:13:41 +0000 (UTC) From: merch-redmine@jeremyevans.net Message-ID: References: Mime-Version: 1.0 X-Redmine-MailingListIntegration-Message-Ids: 68133 X-Redmine-Project: ruby-trunk X-Redmine-Issue-Id: 15839 X-Redmine-Issue-Author: nobu X-Redmine-Sender: jeremyevans0 X-Mailer: Redmine X-Redmine-Host: bugs.ruby-lang.org X-Redmine-Site: Ruby Issue Tracking System X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-SG-EID: =?us-ascii?Q?RVE3t853K5scBhbmJHUzZTFFeVC=2FZSUmHZ0Dc+26wcEi2CTgsF1oz0wTSSxGGN?= =?us-ascii?Q?BI00hsQ56JJmHYvLMYl54AREXN8X2qpfWewSf5W?= =?us-ascii?Q?eqjKCawv5UvOTJ6lOdf23cHz243cdxuq4i8lkEd?= =?us-ascii?Q?duOtQ7OOG45SkitmG6BYSJmAMauhhnyiX3pJmxW?= =?us-ascii?Q?FhtEoex7bV3PVsgo1DPPihvH6tw=2F2iLVx1A=3D=3D?= To: ruby-core@ruby-lang.org X-ML-Name: ruby-core X-Mail-Count: 92650 Subject: [ruby-core:92650] [Ruby trunk Bug#15839] mixed encoding heredoc should be a syntax error regardless the order X-BeenThere: ruby-core@ruby-lang.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Ruby developers List-Id: Ruby developers List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ruby-core-bounces@ruby-lang.org Sender: "ruby-core" Issue #15839 has been updated by jeremyevans0 (Jeremy Evans). File mixed-encoding-heredoc-fix.patch added Heredocs are parsed line-by-line, and mixed encoding is already detected if it is on the same line: ```ruby #encoding: cp932 p <<-STR \xe9\x9d\u1234 STR # UTF-8 mixed within Windows-31J source # \xe9\x9d\u1234 # syntax error, unexpected end-of-input, expecting tSTRING_CONTENT or tSTRING_DBEG or tSTRING_DVAR or tSTRING_END ``` In order to handle mixed content on separate lines, we need to keep track of the temporary encoding of the string, which was previously done via a local variable in `tokadd_string`. The attached patch adds a second `rb_encoding **` argument to `tokadd_string` for keeping track of the temporary encoding, so that `here_document` can store the value between lines. ---------------------------------------- Bug #15839: mixed encoding heredoc should be a syntax error regardless the order https://bugs.ruby-lang.org/issues/15839#change-78012 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: * Backport: 2.4: REQUIRED, 2.5: REQUIRED, 2.6: REQUIRED ---------------------------------------- This heredoc isn't a syntax error, ```ruby #encoding: cp932 p <<-STR \xe9\x9d \u1234 STR ``` whereas this is. ```ruby #encoding: cp932 " \xe9\x9d \u1234 " ``` ---Files-------------------------------- mixed-encoding-heredoc-fix.patch (5.28 KB) -- https://bugs.ruby-lang.org/