From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id F06911F5AE for ; Wed, 8 Jul 2020 18:08:26 +0000 (UTC) Received: from localhost ([::1]:47216 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtEUb-00027v-Px for normalperson@yhbt.net; Wed, 08 Jul 2020 14:08:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42262) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtEUT-00027T-VF; Wed, 08 Jul 2020 14:08:18 -0400 Received: from mo4-p00-ob.smtp.rzone.de ([85.215.255.22]:27421) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtEUO-0001iN-AZ; Wed, 08 Jul 2020 14:08:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1594231688; s=strato-dkim-0002; d=clisp.org; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: X-RZG-CLASS-ID:X-RZG-AUTH:From:Subject:Sender; bh=HPQUflEXGEUd8/BSbCZqawPOp+HArHKRChjtbcwQiyM=; b=s0kY5PkJs9ezPma2o9GUTFge1Sc7F9Rh8MpaodnGNAVWapuMw2OlfLNxTFBBgPyoov +4/WEbl+u+FZUhicsuJI80pfCU0h2EK+RHQJM7VEbJ32ThqH9Pn61fLykpQxQcifOUcu oPgwMp/bZB0X9Hexj4eRPd8ibC2MB+IHL38lmAHD1Zkxp6ZVWSZHiqLQyBX9NMZ6xMM4 ctvqEyUh7ZkOYC53Dp+TH/4Y8/bbqkmLsbaSbyiGv77rw6b1eWzNJuk2fKbygPelhyYO YBe5xraVCxBN0t3zw5UnXSH6ataKQ/xo3M/54WiEfp6S5xkHogE6nFmIl4JuU6jPdBgV 8jMg== X-RZG-AUTH: ":Ln4Re0+Ic/6oZXR1YgKryK8brlshOcZlIWs+iCP5vnk6shH+AHjwLuWOH6fzxfs=" X-RZG-CLASS-ID: mo00 Received: from bruno.haible.de by smtp.strato.de (RZmta 46.10.5 DYNA|AUTH) with ESMTPSA id R03d1aw68I7wQJ8 (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (curve X9_62_prime256v1 with 256 ECDH bits, eq. 3072 bits RSA)) (Client did not present a certificate); Wed, 8 Jul 2020 20:07:58 +0200 (CEST) From: Bruno Haible To: bug-gnulib@gnu.org Subject: Re: portability issues with unicodeio Date: Wed, 08 Jul 2020 20:07:57 +0200 Message-ID: <5129694.dEMNA6ixTA@omega> User-Agent: KMail/5.1.3 (Linux/4.4.0-179-generic; KDE/5.18.0; x86_64; ; ) In-Reply-To: References: <13303583.938451.1593894838288.JavaMail.yahoo.ref@mail.yahoo.co.jp> <1053773524.1039079.1594067726913.JavaMail.yahoo@mail.yahoo.co.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart2137726.ZAYov7PDFD" Content-Transfer-Encoding: 7Bit Received-SPF: none client-ip=85.215.255.22; envelope-from=bruno@clisp.org; helo=mo4-p00-ob.smtp.rzone.de X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/08 14:08:08 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kiyoshi KANAZAWA , Akim Demaille , Bison Bugs Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" This is a multi-part message in MIME format. --nextPart2137726.ZAYov7PDFD Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Hi Akim, Kiyoshi-san, > Unfortunately on Kiyoshi's environment (SunOS hidden 5.11 11.3 i86pc i386 i86pc, > GCC 9.3.0) we get '?' instead of '.' in the C locale. We get a genuine ASCII > '?', it's not some fallback from the terminal which fails to display the > character. And we properly get the bullet with en_US.UTF-8. I am adding a unit test for the 'unicodeio' module (see below). It contains a test for the C locale, and this test works fine on Solaris 11.3. Therefore I have to ask Kiyoshi two questions: 1) What is the output of the program 'locale'? $ locale 2) What are the link dependencies of the coreutils printf program? $ ldd $coreutilsbin/printf Bruno 2020-07-08 Bruno Haible unicodeio: Add tests. * tests/test-unicodeio.c: New file. * tests/test-unicodeio1.sh: New file. * tests/test-unicodeio2.sh: New file. * tests/test-unicodeio3.sh: New file. * modules/unicodeio-tests: New file. 2020-07-08 Bruno Haible unicodeio: Document link requirements. * modules/unicodeio (Link): New section. --nextPart2137726.ZAYov7PDFD Content-Disposition: attachment; filename="0001-unicodeio-Document-link-requirements.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0001-unicodeio-Document-link-requirements.patch" >From 77157f813248fb7dfd4d854ab85d107a1fe026b6 Mon Sep 17 00:00:00 2001 From: Bruno Haible Date: Wed, 8 Jul 2020 20:00:00 +0200 Subject: [PATCH 1/2] unicodeio: Document link requirements. * modules/unicodeio (Link): New section. --- ChangeLog | 5 +++++ modules/unicodeio | 3 +++ 2 files changed, 8 insertions(+) diff --git a/ChangeLog b/ChangeLog index 9cac500..aa36674 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2020-07-08 Bruno Haible + + unicodeio: Document link requirements. + * modules/unicodeio (Link): New section. + 2020-07-07 Bruno Haible doc: Remove support for some very old platforms. diff --git a/modules/unicodeio b/modules/unicodeio index 8164ac4..51721db 100644 --- a/modules/unicodeio +++ b/modules/unicodeio @@ -24,6 +24,9 @@ lib_SOURCES += unicodeio.h unicodeio.c Include: "unicodeio.h" +Link: +$(LTLIBICONV) when linking with libtool, $(LIBICONV) otherwise + License: GPL -- 2.7.4 --nextPart2137726.ZAYov7PDFD Content-Disposition: attachment; filename="0002-unicodeio-Add-tests.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0002-unicodeio-Add-tests.patch" >From 7f89a25089f411dce4b22b55000de75c4bffe524 Mon Sep 17 00:00:00 2001 From: Bruno Haible Date: Wed, 8 Jul 2020 20:01:36 +0200 Subject: [PATCH 2/2] unicodeio: Add tests. * tests/test-unicodeio.c: New file. * tests/test-unicodeio1.sh: New file. * tests/test-unicodeio2.sh: New file. * tests/test-unicodeio3.sh: New file. * modules/unicodeio-tests: New file. --- ChangeLog | 9 ++++++ modules/unicodeio-tests | 21 +++++++++++++ tests/test-unicodeio.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/test-unicodeio1.sh | 5 +++ tests/test-unicodeio2.sh | 15 +++++++++ tests/test-unicodeio3.sh | 15 +++++++++ 6 files changed, 146 insertions(+) create mode 100644 modules/unicodeio-tests create mode 100644 tests/test-unicodeio.c create mode 100755 tests/test-unicodeio1.sh create mode 100755 tests/test-unicodeio2.sh create mode 100755 tests/test-unicodeio3.sh diff --git a/ChangeLog b/ChangeLog index aa36674..c2182f0 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,14 @@ 2020-07-08 Bruno Haible + unicodeio: Add tests. + * tests/test-unicodeio.c: New file. + * tests/test-unicodeio1.sh: New file. + * tests/test-unicodeio2.sh: New file. + * tests/test-unicodeio3.sh: New file. + * modules/unicodeio-tests: New file. + +2020-07-08 Bruno Haible + unicodeio: Document link requirements. * modules/unicodeio (Link): New section. diff --git a/modules/unicodeio-tests b/modules/unicodeio-tests new file mode 100644 index 0000000..d771ff8 --- /dev/null +++ b/modules/unicodeio-tests @@ -0,0 +1,21 @@ +Files: +tests/test-unicodeio1.sh +tests/test-unicodeio2.sh +tests/test-unicodeio3.sh +tests/test-unicodeio.c +tests/macros.h +m4/locale-fr.m4 +m4/locale-zh.m4 +m4/codeset.m4 + +Depends-on: +setlocale + +configure.ac: +gt_LOCALE_FR_UTF8 +gt_LOCALE_ZH_CN + +Makefile.am: +TESTS += test-unicodeio1.sh test-unicodeio2.sh test-unicodeio3.sh +check_PROGRAMS += test-unicodeio +test_unicodeio_LDADD = $(LDADD) $(LIB_SETLOCALE) $(LIBICONV) diff --git a/tests/test-unicodeio.c b/tests/test-unicodeio.c new file mode 100644 index 0000000..be91a91 --- /dev/null +++ b/tests/test-unicodeio.c @@ -0,0 +1,81 @@ +/* Tests for Unicode character output. + + Copyright (C) 2020 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +/* Written by Bruno Haible, 2020. */ + +#include + +/* Specification. */ +#include "unicodeio.h" + +#include +#include + +#include "macros.h" + +#define TEST_CODE 0x2022 +#define TEST_CODE_AS_UTF8 "\xe2\x80\xa2" +#define TEST_CODE_AS_GB18030 "\x81\x36\xa6\x31" + +static char result[64]; + +static long +success_callback (const char *buf, size_t buflen, void *callback_arg) +{ + memcpy (result, buf, buflen); + result[buflen] = '\0'; + return 42; +} + +static long +failure_callback (unsigned int code, const char *msg, void *callback_arg) +{ + ASSERT (code == TEST_CODE); + strcpy (result, "."); + return 55; +} + +int +main (int argc, char *argv[]) +{ + /* configure should already have checked that the locale is supported. */ + if (setlocale (LC_ALL, "") == NULL) + return 1; + + switch (unicode_to_mb (TEST_CODE, success_callback, failure_callback, NULL)) + { + case 42: + if (argc > 1) + switch (argv[1][0]) + { + case '2': + ASSERT (strcmp (result, TEST_CODE_AS_UTF8) == 0); + break; + case '3': + ASSERT (strcmp (result, TEST_CODE_AS_GB18030) == 0); + break; + } + break; + case 55: + ASSERT (strcmp (result, ".") == 0); + break; + default: + ASSERT (0); + } + + return 0; +} diff --git a/tests/test-unicodeio1.sh b/tests/test-unicodeio1.sh new file mode 100755 index 0000000..5f5f1bd --- /dev/null +++ b/tests/test-unicodeio1.sh @@ -0,0 +1,5 @@ +#!/bin/sh + +# Test in the C locale. +LC_ALL=C \ +${CHECKER} ./test-unicodeio${EXEEXT} 1 diff --git a/tests/test-unicodeio2.sh b/tests/test-unicodeio2.sh new file mode 100755 index 0000000..f0ec8fb --- /dev/null +++ b/tests/test-unicodeio2.sh @@ -0,0 +1,15 @@ +#!/bin/sh + +# Test whether a specific UTF-8 locale is installed. +: ${LOCALE_FR_UTF8=fr_FR.UTF-8} +if test $LOCALE_FR_UTF8 = none; then + if test -f /usr/bin/localedef; then + echo "Skipping test: no french Unicode locale is installed" + else + echo "Skipping test: no french Unicode locale is supported" + fi + exit 77 +fi + +LC_ALL=$LOCALE_FR_UTF8 \ +${CHECKER} ./test-unicodeio${EXEEXT} 2 diff --git a/tests/test-unicodeio3.sh b/tests/test-unicodeio3.sh new file mode 100755 index 0000000..1544c96 --- /dev/null +++ b/tests/test-unicodeio3.sh @@ -0,0 +1,15 @@ +#!/bin/sh + +# Test whether a specific GB18030 locale is installed. +: ${LOCALE_ZH_CN=zh_CN.GB18030} +if test $LOCALE_ZH_CN = none; then + if test -f /usr/bin/localedef; then + echo "Skipping test: no transitional chinese locale is installed" + else + echo "Skipping test: no transitional chinese locale is supported" + fi + exit 77 +fi + +LC_ALL=$LOCALE_ZH_CN \ +${CHECKER} ./test-unicodeio${EXEEXT} 3 -- 2.7.4 --nextPart2137726.ZAYov7PDFD--