Re: [PATCH 15/16] linux: Add {f}stat{at} y2038 support

[Adhemerval Zanella via Libc-alpha <libc-alpha@sourceware.org>]

[-- Attachment #1.1: Type: text/plain, Size: 6832 bytes --]



On 14/10/2020 10:15, Lukasz Majewski wrote:
> Dear Community,
> 
>> Hi Adhemerval,
>>
>>> On 13/10/2020 10:58, Lukasz Majewski wrote:  
>>>> Hi Adhemerval,
>>>>     
>>>>> On 07/10/2020 09:52, Adhemerval Zanella wrote:    
>>>>>>
>>>>>>
>>>>>> On 06/10/2020 06:48, Lukasz Majewski wrote:      
>>>>>>> Hi Adhemerval,
>>>>>>>      
>>>>>>>> A new struct __stat{64}_t64 type is added with the required
>>>>>>>> __timespec64 time definition.  Both non-LFS and LFS support
>>>>>>>> were done with an extra __NR_statx call plus a conversion to
>>>>>>>> the new __stat{64}_t64 type.  The statx call is done only for
>>>>>>>> architectures with support for 32-bit time_t ABI.
>>>>>>>>
>>>>>>>> Internally some extra routines to copy from/to struct stat{64}
>>>>>>>> to struct __stat{64} used on multiple implementations (stat,
>>>>>>>> fstat, lstat, and fstatat) are added on a extra file
>>>>>>>> (stat_t64_cp.c).  Aslo some extra routines to copy from statx
>>>>>>>> to __stat{64} is added on statx_cp.c.
>>>>>>>>
>>>>>>>> Checked with a build for all affected ABIs. I also checked on
>>>>>>>> x86_64, i686, powerpc, powerpc64le, sparcv9, sparc64, s390,
>>>>>>>> and s390x.      
>>>>>>>
>>>>>>> When do you plan to pull this patch set to -master?
>>>>>>> Those patches have been available for review on the mailing
>>>>>>> list for more than two months now.      
>>>>>>
>>>>>> Hi Lukasz, thanks to remind me. I will rebase against master and
>>>>>> run some regressions tests against some platforms and push it.
>>>>>>       
>>>>>
>>>>> One required change with the rebase is adapt the riscv32 ABI to
>>>>> exclude the __{f,l}xstat{at} symbol and replace with proper
>>>>> {f,l}stat ones. It is possible because the new ABI was added on
>>>>> current development branch, however one minor inconvenient is the
>>>>> toolchain need to be rebuild with a updated glibc branch to avoid
>>>>> linking failures with libstd++ (which uses __{f,l}xstat{at}).
>>>>>    
>>>>
>>>> I'm not sure if this is related, but on my ARMv7 (32 bit) sandbox
>>>> there is an issue with fstat accesses to files.
>>>>
>>>> When I try to run a program build against newest glibc (installed
>>>> in /opt/lib) I do see issues with {f}stat on other libraries (e.g.
>>>> /opt/lib/librt.so). To be more specific I do experience the
>>>> EOVERFLOW error:
>>>>
>>>> error while loading shared libraries: librt.so.1: cannot stat
>>>> shared object: Error 75
>>>>
>>>> The "base" glibc is 2.28 (installed in /lib). The glibc under test
>>>> is the newest master installed in /opt/lib.
>>>>
>>>> I'm now investigating this issue.    
>>>
>>> I am not sure what it might be based on these information, could you
>>> provide a strace so we can pinpoint what might the issue?    
>>
>> Things are getting more and more interesting.
>>
>> Let's consider the /opt/lib/librt.so.1
>>
>> After qemu boot:
>>
>> root@y2038arm:~# stat /opt/lib/librt.so.1
>>   File: /opt/lib/librt.so.1 -> librt-2.32.9000.so
>>   Size: 18              Blocks: 0          IO Block: 4096   symbolic
>> link Device: b300h/45824d    Inode: 17490       Links: 1
>> Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/
>> root) Access: 2020-10-13 23:14:56.800000000 +0000      ----> OK
>> Modify: 2020-10-13 23:14:52.770000000 +0000
>> Change: 2020-10-13 23:14:52.770000000 +0000
>>
>> Then I do run gdb (which was build with 2.28 glibc as a base and gcc
>> 8.1, which uses the same library):
>>
>> gdb test_y2038:
>> ...
>> (gdb) run
>> Starting program: /usr/bin/test_y2038
>> /usr/bin/test_y2038: error while loading shared libraries: librt.so.1:
>> cannot stat shared object: Error 75 [Inferior 1 (process 1045) exited
>> with code 0177]
>>
>> So I've accessed the librt.so.1 with some old - i.e. 2.28 ABI - now:
>>
>> root@y2038arm:~# stat /opt/lib/librt.so.1
>>   File: /opt/lib/librt.so.1 -> librt-2.32.9000.so
>>   Size: 18              Blocks: 0          IO Block: 4096   symbolic
>> link Device: b300h/45824d    Inode: 17490       Links: 1
>> Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/
>> root) Access: 1901-12-13 20:46:33.625721000 +0000  ----> Overflow
>> Modify: 2020-10-13 23:14:52.770000000 +0000
>> Change: 2020-10-13 23:14:52.770000000 +0000
>>
>> root@y2038arm:~# strace -v -Tf -e trace=file test_y2038
>>
>> openat(AT_FDCWD, "/opt/lib/librt.so.1",
>> O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 <0.000409> statx(3, "",
>> AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
>> STATX_BASIC_STATS, {stx_mask=STATX_ALL, stx_blksize=4096,
>> stx_attributes=0, stx_nlink=1, stx_uid=0, stx_gid=0,
>> stx_mode=S_IFREG|0755, stx_ino=17487, stx_size=425008, stx_blocks=832,
>> stx_attributes_mask=STATX_ATTR_COMPRESSED|STATX_ATTR_IMMUTABLE|STATX_ATTR_APPEND|STATX_ATTR_NODUMP|STATX_ATTR_ENCRYPTED,
>> stx_atime={tv_sec=1602625090, tv_nsec=600000000} /*
>> 2020-10-13T21:38:10.600000000+0000 */, stx_btime={tv_sec=1602622592,
>> tv_nsec=0} /* 2020-10-13T20:56:32+0000 */,
>> stx_ctime={tv_sec=2147484284, tv_nsec=335721000},
>> stx_mtime={tv_sec=2147484263, tv_nsec=545721000}, stx_rdev_major=0,
>> stx_rdev_minor=0, stx_dev_major=179, stx_dev_minor=0}) = 0 <0.000494>
>> test_y2038: error while loading shared libraries: librt.so.1: cannot
>> stat shared object: Error 75 +++ exited with 127 +++
>>
>>
>> Here the ctime and mtime are wrong - i.e. overflowed.
>>
>> The system date is not changed:
>> root@y2038arm:~# date
>> Tue Oct 13 21:40:02 UTC 2020
>> root@y2038arm:~# date +%s
>> 1602625224
>>
>> I will continue investigation tomorrow...
> 
> And now the penny has dropped...
> 
> The issue was that:
> 
> 1. Adhemerval has added extra checks for overflowing the time in
> {f}stat{at} patches - this is obviously good :-)
> 
> 2. In my test system - I've been using "time" syscall (glibc function)
> to get the seconds after epoch. This was not yet [*] converted to
> support 64 bit time.
> 
> 3. Some tests (like clock_gettime / clock_settime) require time
> modifications after Y2038. Broken time() caused wrong data after
> casting from __time_t to __time64_t and the proper time couldn't be
> restored.
> 
> 4. As a result - the date was wrong during the following tests
> execution and {mac}time for external files (like librt.so.1,
> libgcc_s.so) was set to wrong value.
> 
> 5. Each subsequent call of {fl}stat{64} returned error due to date
> overflow check.
> 
> Fix:
> 
> Convert time to support 64 bit time.
> 
> [*] I will send patches after build-many-glibcs.py and xcheck finish.

Good to know this is an issue not related to the stat patches itself.
I was confused because you initially didn't describe the test you
are doing in details, so it was hard to pinpoint the issue. 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]