Re: Issue in man page wcsncpy.3

Re: Issue in man page wcsncpy.3

[Alejandro Colomar via Libc-alpha]

[-- Attachment #1.1: Type: text/plain, Size: 2547 bytes --]

Hi Helge, glibc developers,

On 12/4/22 10:07, Helge Kreutzmann wrote:
> Without further ado, the following was found:
> 
> Issue:    Is the "L" in the bracket (for the NULL character) correct?
> 
> "The B<wcsncpy>()  function is the wide-character equivalent of the"
> "B<strncpy>(3)  function.  It copies at most I<n> wide characters from the"
> "wide-character string pointed to by I<src>, including the terminating null"
> "wide character (L\\(aq\\e0\\(aq), to the array pointed to by I<dest>."
> "Exactly I<n> wide characters are written at I<dest>.  If the length"
> "I<wcslen(src)> is smaller than I<n>, the remaining wide characters in the"
> "array pointed to by I<dest> are filled with null wide characters.  If the"
> "length I<wcslen(src)> is greater than or equal to I<n>, the string pointed"
> "to by I<dest> will not be terminated by a null wide character."

As an unrelated note.  I've had this running in my mind for some time... your 
various bug reports for strncpy(3) and similar wide character functions have 
triggered those thougts.

I'm going to mark strncpy(3) and similar functions as deprecated, even if no 
libc or standard has done so.  There's wide agreement (at least in some 
communities) that strncpy(3) _is evil_.  There's simply no use for it.

I propose that glibc also marks it as deprecated.

I've worked for a few months on improving string handling in various projects: 
shadow-utils <https://github.com/shadow-maint/shadow/pull/569>, and in nginx 
Unit.  I've come to the following guidelines for using strings:

-  strlcpy(3):  Copy from string to string, detecting truncation
-  strscpy(9):  Copy from untrusted string to string, detecting broken ones
-  ustr2str():  Copy from unterminated string to string
    -  Definition: 
<https://github.com/shadow-maint/shadow/pull/569/files#diff-a2b8f8568ad025c2aeb4c64d3510f42d874e9ef0a64a89c2dd750aa93aabcd97>

-  stpecpy():  Copy from string to string, with easy & safe concatenation, and 
reporting truncation at the end of the chain call.
    -  Definition: <https://software.codidact.com/posts/285946/287522#answer-287522>

-  ustr2stpe():  Combination of ustr2str() and stpecpy().
    -  Definition:  Not yet public


And also, memcpy(), mempcpy(), or memccpy() can be used for copying unterminated 
strings.

But I don't see any scenario where strncpy() is the right function to call.  And 
the name is certainly not telling that either.


Cheers,

Alex


-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Issue in man page wcsncpy.3

[Alejandro Colomar via Libc-alpha]

[-- Attachment #1.1: Type: text/plain, Size: 3102 bytes --]

Hi,

On 12/4/22 21:42, Alejandro Colomar wrote:
> As an unrelated note.  I've had this running in my mind for some time... your 
> various bug reports for strncpy(3) and similar wide character functions have 
> triggered those thougts.
> 
> I'm going to mark strncpy(3) and similar functions as deprecated, even if no 
> libc or standard has done so.  There's wide agreement (at least in some 
> communities) that strncpy(3) _is evil_.  There's simply no use for it.
> 
> I propose that glibc also marks it as deprecated.
> 
> I've worked for a few months on improving string handling in various projects: 
> shadow-utils <https://github.com/shadow-maint/shadow/pull/569>, and in nginx 
> Unit.  I've come to the following guidelines for using strings:
> 
> -  strlcpy(3):  Copy from string to string, detecting truncation
> -  strscpy(9):  Copy from untrusted string to string, detecting broken ones
> -  ustr2str():  Copy from unterminated string to string
>     -  Definition: 
> <https://github.com/shadow-maint/shadow/pull/569/files#diff-a2b8f8568ad025c2aeb4c64d3510f42d874e9ef0a64a89c2dd750aa93aabcd97>
> 
> -  stpecpy():  Copy from string to string, with easy & safe concatenation, and 
> reporting truncation at the end of the chain call.
>     -  Definition: 
> <https://software.codidact.com/posts/285946/287522#answer-287522>
> 
> -  ustr2stpe():  Combination of ustr2str() and stpecpy().
>     -  Definition:  Not yet public
> 
> 
> And also, memcpy(), mempcpy(), or memccpy() can be used for copying unterminated 
> strings.
> 
> But I don't see any scenario where strncpy() is the right function to call.  And 
> the name is certainly not telling that either.

I did some initial work, to split strcpy(3) and strncpy(3) manual pages, and 
clearly document the only purpose for which strncpy(3) seems to be useful (or at 
least, not plain wrong): copying a string into a fixed-width buffer, and 
ensuring that no garbage is leaked.

However, even for that niche use-case, it has issues: it can't report truncation.

Consider a trivial implementation of strncpy(3):

            char *
            strncpy(char *dest, const char *src, size_t n)
            {
                bzero(dest, n);
                memccpy(dest, src, '\0', n);

                return dest;
            }

As this implementation hints, this function is only useful when you do want to 
apply those two calls in that order.  There are few use cases for that, but not 
inexistent, okay.  However:

(1)  Having the 2 separate calls is probably better for self documentation than 
strncpy(3), especially since strncpy(3) has been misused extensively.  However, 
if a project correctly documents its use of strncpy(3), it might be fine, and 
less code.

(2)  strncpy(3) can't detect truncation.  strncpy(3) has been misdesigned, and 
this repurpose is not great.

(3)  Its name is not at all telling what it does.


Which is why I think we should deprecate it.

Cheers,

Alex

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Issue in man page wcsncpy.3

[Sam James via Libc-alpha]

[-- Attachment #1: Type: text/plain, Size: 1819 bytes --]



> On 4 Dec 2022, at 20:42, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
> 
> Hi Helge, glibc developers,
> 
> On 12/4/22 10:07, Helge Kreutzmann wrote:
>> Without further ado, the following was found:
>> Issue:    Is the "L" in the bracket (for the NULL character) correct?
>> "The B<wcsncpy>()  function is the wide-character equivalent of the"
>> "B<strncpy>(3)  function.  It copies at most I<n> wide characters from the"
>> "wide-character string pointed to by I<src>, including the terminating null"
>> "wide character (L\\(aq\\e0\\(aq), to the array pointed to by I<dest>."
>> "Exactly I<n> wide characters are written at I<dest>.  If the length"
>> "I<wcslen(src)> is smaller than I<n>, the remaining wide characters in the"
>> "array pointed to by I<dest> are filled with null wide characters.  If the"
>> "length I<wcslen(src)> is greater than or equal to I<n>, the string pointed"
>> "to by I<dest> will not be terminated by a null wide character."
> 
> As an unrelated note.  I've had this running in my mind for some time... your various bug reports for strncpy(3) and similar wide character functions have triggered those thougts.
> 
> I'm going to mark strncpy(3) and similar functions as deprecated, even if no libc or standard has done so.  There's wide agreement (at least in some communities) that strncpy(3) _is evil_.  There's simply no use for it.
> 

Please don't do this unilaterally. Apple did this unilaterally for sprintf which has caused problems, as well.

It's going to cause confusion as people will inevitably ask where/who deprecated it and there won't
be a solid answer. And if we can't get a libc to agree to deprecate it as well, then doing it in the man
pages is wrong. Even if I understand the spirit of the idea.

Best,
sam

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: Issue in man page wcsncpy.3

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]



> On 4 Dec 2022, at 23:06, Sam James via Libc-alpha <libc-alpha@sourceware.org> wrote:
> 
> 
> 
>> On 4 Dec 2022, at 20:42, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
>> 
>> Hi Helge, glibc developers,
>> 
>> On 12/4/22 10:07, Helge Kreutzmann wrote:
>>> Without further ado, the following was found:
>>> Issue:    Is the "L" in the bracket (for the NULL character) correct?
>>> "The B<wcsncpy>()  function is the wide-character equivalent of the"
>>> "B<strncpy>(3)  function.  It copies at most I<n> wide characters from the"
>>> "wide-character string pointed to by I<src>, including the terminating null"
>>> "wide character (L\\(aq\\e0\\(aq), to the array pointed to by I<dest>."
>>> "Exactly I<n> wide characters are written at I<dest>.  If the length"
>>> "I<wcslen(src)> is smaller than I<n>, the remaining wide characters in the"
>>> "array pointed to by I<dest> are filled with null wide characters.  If the"
>>> "length I<wcslen(src)> is greater than or equal to I<n>, the string pointed"
>>> "to by I<dest> will not be terminated by a null wide character."
>> 
>> As an unrelated note.  I've had this running in my mind for some time... your various bug reports for strncpy(3) and similar wide character functions have triggered those thougts.
>> 
>> I'm going to mark strncpy(3) and similar functions as deprecated, even if no libc or standard has done so.  There's wide agreement (at least in some communities) that strncpy(3) _is evil_.  There's simply no use for it.
>> 
> 
> Please don't do this unilaterally. Apple did this unilaterally for sprintf which has caused problems, as well.

snprintf, that is

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: Issue in man page wcsncpy.3

[Alejandro Colomar via Libc-alpha]

[-- Attachment #1.1: Type: text/plain, Size: 6094 bytes --]

Hi Sam!

On 12/5/22 00:06, Sam James wrote:
> 
> 
>> On 4 Dec 2022, at 20:42, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
>>
>> Hi Helge, glibc developers,
>>
>> On 12/4/22 10:07, Helge Kreutzmann wrote:
>>> Without further ado, the following was found:
>>> Issue:    Is the "L" in the bracket (for the NULL character) correct?
>>> "The B<wcsncpy>()  function is the wide-character equivalent of the"
>>> "B<strncpy>(3)  function.  It copies at most I<n> wide characters from the"
>>> "wide-character string pointed to by I<src>, including the terminating null"
>>> "wide character (L\\(aq\\e0\\(aq), to the array pointed to by I<dest>."
>>> "Exactly I<n> wide characters are written at I<dest>.  If the length"
>>> "I<wcslen(src)> is smaller than I<n>, the remaining wide characters in the"
>>> "array pointed to by I<dest> are filled with null wide characters.  If the"
>>> "length I<wcslen(src)> is greater than or equal to I<n>, the string pointed"
>>> "to by I<dest> will not be terminated by a null wide character."
>>
>> As an unrelated note.  I've had this running in my mind for some time... your various bug reports for strncpy(3) and similar wide character functions have triggered those thougts.
>>
>> I'm going to mark strncpy(3) and similar functions as deprecated, even if no libc or standard has done so.  There's wide agreement (at least in some communities) that strncpy(3) _is evil_.  There's simply no use for it.
>>
> 
> Please don't do this unilaterally. Apple did this unilaterally for sprintf which has caused problems, as well.
> 
> It's going to cause confusion as people will inevitably ask where/who deprecated it and there won't
> be a solid answer.

I'm writing up a solid answer for that.  Please check 
<https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man3/strncpy.3>.

The manual page is now all about why it should never be used.  In fact, I was 
going to do that without adding the [[deprecated]] attribute to the SYNOPSIS, 
but I convinced myself that it's necessary after writing that.

I'll copy here the current state of the page.  If there's general concern about 
the attribute, I can remove it while keeping everything else against its use; I 
don't think it will be a big change.  But I hope this convinces readers about it 
being harmful.



strncpy(3)               Library Functions Manual              strncpy(3)

NAME
        strncpy  -  copy  a string into a fixed‐length buffer and zero the
        rest of it

LIBRARY
        Standard C library (libc, -lc)

SYNOPSIS
        #include <string.h>

        [[deprecated]] char *strncpy(char dest[restrict .n],
                                     const char src[restrict .n], size_t n);

DESCRIPTION
        Note: This is not the function you want to use.  For  safe  string
        copying,  see  strlcpy(3bsd).   For copying a string into a fixed‐
        length buffer with zeroing of the rest, see stpncpy(3).

        The strncpy() copies at most n bytes of src, and fills the rest of
        the dest buffer with null bytes.  Warning: If  there  is  no  null
        byte  among  the  first  n bytes of src, the string placed in dest
        will not be null‐terminated.

        A simple implementation of strncpy() might be:

            char *
            strncpy(char *dest, const char *src, size_t n)
            {
                bzero(dest, n);
                memccpy(dest, src, '\0', n);

                return dest;
            }

        The only valid use of strncpy() is to copy a C string to a  fixed‐
        length  buffer while ensuring that unused bytes in the destination
        buffer are zeroed out (perhaps to prevent information leaks if the
        buffer is to be written to media or transmitted to another process
        via an interprocess communication technique).  But  stpncpy(3)  is
        better for this purpose, since it detects truncation.  See CAVEATS
        below.

RETURN VALUE
        The strncpy() function returns a pointer to the destination buffer
        dest.

ATTRIBUTES
        For  an explanation of the terms used in this section, see attrib‐
        utes(7).
        ┌───────────────────────────────────────┬───────────────┬─────────┐
        │Interface                              │ Attribute     │ Value   │
        ├───────────────────────────────────────┼───────────────┼─────────┤
        │strncpy()                              │ Thread safety │ MT‐Safe │
        └───────────────────────────────────────┴───────────────┴─────────┘

STANDARDS
        POSIX.1‐2001, POSIX.1‐2008, C89, C99, SVr4, 4.3BSD.

CAVEATS
        strncpy() has a misleading name.  It doesn’t produce a  (null‐ter‐
        minated)  string;  and  it  should  never  be used for producing a
        string.

        It can’t detect truncation.  It’s probably  better  to  explicitly
        call  bzero(3)  and memccpy(3), or stpncpy(3) since they allow de‐
        tecting truncation.

SEE ALSO
        bzero(3), memccpy(3), stpncpy(3), string(3), wcsncpy(3)

Linux man‐pages (unreleased)      (date)                       strncpy(3)

> And if we can't get a libc to agree to deprecate it as well, then doing it in the man
> pages is wrong. Even if I understand the spirit of the idea.

I hope this can convince someone in glibc :)
If there's consensus against marking it as [[deprecated]], I'll revert that bit 
before the next release later this month.


Cheers,

Alex

> 
> Best,
> sam

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Issue in man page wcsncpy.3

[Alejandro Colomar via Libc-alpha]

[-- Attachment #1.1: Type: text/plain, Size: 1917 bytes --]

Hi Sam,

On 12/5/22 00:09, Sam James wrote:
> 
> 
>> On 4 Dec 2022, at 23:06, Sam James via Libc-alpha <libc-alpha@sourceware.org> wrote:
>>
>>
>>
>>> On 4 Dec 2022, at 20:42, Alejandro Colomar via Libc-alpha <libc-alpha@sourceware.org> wrote:
>>>
>>> Hi Helge, glibc developers,
>>>
>>> On 12/4/22 10:07, Helge Kreutzmann wrote:
>>>> Without further ado, the following was found:
>>>> Issue:    Is the "L" in the bracket (for the NULL character) correct?
>>>> "The B<wcsncpy>()  function is the wide-character equivalent of the"
>>>> "B<strncpy>(3)  function.  It copies at most I<n> wide characters from the"
>>>> "wide-character string pointed to by I<src>, including the terminating null"
>>>> "wide character (L\\(aq\\e0\\(aq), to the array pointed to by I<dest>."
>>>> "Exactly I<n> wide characters are written at I<dest>.  If the length"
>>>> "I<wcslen(src)> is smaller than I<n>, the remaining wide characters in the"
>>>> "array pointed to by I<dest> are filled with null wide characters.  If the"
>>>> "length I<wcslen(src)> is greater than or equal to I<n>, the string pointed"
>>>> "to by I<dest> will not be terminated by a null wide character."
>>>
>>> As an unrelated note.  I've had this running in my mind for some time... your various bug reports for strncpy(3) and similar wide character functions have triggered those thougts.
>>>
>>> I'm going to mark strncpy(3) and similar functions as deprecated, even if no libc or standard has done so.  There's wide agreement (at least in some communities) that strncpy(3) _is evil_.  There's simply no use for it.
>>>
>>
>> Please don't do this unilaterally. Apple did this unilaterally for sprintf which has caused problems, as well.
> 
> snprintf, that is

No, they deprecated sprintf(3), AFAIK.
<https://lists.gnu.org/r/bug-gnulib/2022-11/msg00013.html>

Cheers,

Alex

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]