iconv module should detect buggy glibc iconv

iconv module should detect buggy glibc iconv

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

See https://sourceware.org/bugzilla/show_bug.cgi?id=29913

This is a bug I just came across (in my capacity as maintainer of Recode)
in current glibc.

glibc iconv(3) can return EILSEQ when the input is merely untranslatable,
rather than invalid. This is directly contrary to the POSIX spec[1], which
says:

[EILSEQ]

Input conversion stopped due to an input byte that does not belong to the
input codeset.

And:

If iconv() encounters a character in the input buffer that is valid, but
> for which an identical character does not exist in the target codeset,
> iconv() shall perform an implementation-defined conversion on this
> character.
>

So, it would be nice if gnulib detected this problem. (Then I would of
course use GNU libiconv!)

[1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/iconv.html
-- 
https://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 2885 bytes --]

Re: iconv module should detect buggy glibc iconv

[Bruno Haible]

Reuben Thomas wrote:
> See https://sourceware.org/bugzilla/show_bug.cgi?id=29913
> 
> This is a bug I just came across (in my capacity as maintainer of Recode)
> in current glibc.
> 
> glibc iconv(3) can return EILSEQ when the input is merely untranslatable,
> rather than invalid. This is directly contrary to the POSIX spec

No, this is not contrary to POSIX. I explain it in the ticket you cited.

> So, it would be nice if gnulib detected this problem.

No, we won't do that. Gnulib is meant to make GNU and POSIX compatible
behaviour available to applications on all platforms. The requested
behaviour, which amounts to adding //TRANSLIT implicitly to *all*
iconv conversions, would be a degradation compared to what we have in
glibc and GNU libiconv.

> (Then I would of course use GNU libiconv!)

It won't help you, since it implements the same behaviour: strict
conversion checking when //TRANSLIT or //IGNORE is not specified, and
loose conversion when //TRANSLIT is specified.

Bruno

Re: iconv module should detect buggy glibc iconv

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

On Sun, 19 Feb 2023 at 00:58, Bruno Haible <bruno@clisp.org> wrote:

> Reuben Thomas wrote:
> > See https://sourceware.org/bugzilla/show_bug.cgi?id=29913
> >
> > This is a bug I just came across (in my capacity as maintainer of Recode)
> > in current glibc.
> >
> > glibc iconv(3) can return EILSEQ when the input is merely untranslatable,
> > rather than invalid. This is directly contrary to the POSIX spec
>
> No, this is not contrary to POSIX. I explain it in the ticket you cited.
>

Thanks, I've followed up on that ticket.

-- 
https://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 1258 bytes --]