Re: [PATCH 15/16] linux: Add {f}stat{at} y2038 support

[Lukasz Majewski <lukma@denx.de>]

[-- Attachment #1: Type: text/plain, Size: 7359 bytes --]

Dear Community,

> Hi Adhemerval,
> 
> > On 13/10/2020 10:58, Lukasz Majewski wrote:  
> > > Hi Adhemerval,
> > >     
> > >> On 07/10/2020 09:52, Adhemerval Zanella wrote:    
> > >>>
> > >>>
> > >>> On 06/10/2020 06:48, Lukasz Majewski wrote:      
> > >>>> Hi Adhemerval,
> > >>>>      
> > >>>>> A new struct __stat{64}_t64 type is added with the required
> > >>>>> __timespec64 time definition.  Both non-LFS and LFS support
> > >>>>> were done with an extra __NR_statx call plus a conversion to
> > >>>>> the new __stat{64}_t64 type.  The statx call is done only for
> > >>>>> architectures with support for 32-bit time_t ABI.
> > >>>>>
> > >>>>> Internally some extra routines to copy from/to struct stat{64}
> > >>>>> to struct __stat{64} used on multiple implementations (stat,
> > >>>>> fstat, lstat, and fstatat) are added on a extra file
> > >>>>> (stat_t64_cp.c).  Aslo some extra routines to copy from statx
> > >>>>> to __stat{64} is added on statx_cp.c.
> > >>>>>
> > >>>>> Checked with a build for all affected ABIs. I also checked on
> > >>>>> x86_64, i686, powerpc, powerpc64le, sparcv9, sparc64, s390,
> > >>>>> and s390x.      
> > >>>>
> > >>>> When do you plan to pull this patch set to -master?
> > >>>> Those patches have been available for review on the mailing
> > >>>> list for more than two months now.      
> > >>>
> > >>> Hi Lukasz, thanks to remind me. I will rebase against master and
> > >>> run some regressions tests against some platforms and push it.
> > >>>       
> > >>
> > >> One required change with the rebase is adapt the riscv32 ABI to
> > >> exclude the __{f,l}xstat{at} symbol and replace with proper
> > >> {f,l}stat ones. It is possible because the new ABI was added on
> > >> current development branch, however one minor inconvenient is the
> > >> toolchain need to be rebuild with a updated glibc branch to avoid
> > >> linking failures with libstd++ (which uses __{f,l}xstat{at}).
> > >>    
> > > 
> > > I'm not sure if this is related, but on my ARMv7 (32 bit) sandbox
> > > there is an issue with fstat accesses to files.
> > > 
> > > When I try to run a program build against newest glibc (installed
> > > in /opt/lib) I do see issues with {f}stat on other libraries (e.g.
> > > /opt/lib/librt.so). To be more specific I do experience the
> > > EOVERFLOW error:
> > > 
> > > error while loading shared libraries: librt.so.1: cannot stat
> > > shared object: Error 75
> > > 
> > > The "base" glibc is 2.28 (installed in /lib). The glibc under test
> > > is the newest master installed in /opt/lib.
> > > 
> > > I'm now investigating this issue.    
> > 
> > I am not sure what it might be based on these information, could you
> > provide a strace so we can pinpoint what might the issue?    
> 
> Things are getting more and more interesting.
> 
> Let's consider the /opt/lib/librt.so.1
> 
> After qemu boot:
> 
> root@y2038arm:~# stat /opt/lib/librt.so.1
>   File: /opt/lib/librt.so.1 -> librt-2.32.9000.so
>   Size: 18              Blocks: 0          IO Block: 4096   symbolic
> link Device: b300h/45824d    Inode: 17490       Links: 1
> Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/
> root) Access: 2020-10-13 23:14:56.800000000 +0000      ----> OK
> Modify: 2020-10-13 23:14:52.770000000 +0000
> Change: 2020-10-13 23:14:52.770000000 +0000
> 
> Then I do run gdb (which was build with 2.28 glibc as a base and gcc
> 8.1, which uses the same library):
> 
> gdb test_y2038:
> ...
> (gdb) run
> Starting program: /usr/bin/test_y2038
> /usr/bin/test_y2038: error while loading shared libraries: librt.so.1:
> cannot stat shared object: Error 75 [Inferior 1 (process 1045) exited
> with code 0177]
> 
> So I've accessed the librt.so.1 with some old - i.e. 2.28 ABI - now:
> 
> root@y2038arm:~# stat /opt/lib/librt.so.1
>   File: /opt/lib/librt.so.1 -> librt-2.32.9000.so
>   Size: 18              Blocks: 0          IO Block: 4096   symbolic
> link Device: b300h/45824d    Inode: 17490       Links: 1
> Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/
> root) Access: 1901-12-13 20:46:33.625721000 +0000  ----> Overflow
> Modify: 2020-10-13 23:14:52.770000000 +0000
> Change: 2020-10-13 23:14:52.770000000 +0000
> 
> root@y2038arm:~# strace -v -Tf -e trace=file test_y2038
> 
> openat(AT_FDCWD, "/opt/lib/librt.so.1",
> O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 <0.000409> statx(3, "",
> AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
> STATX_BASIC_STATS, {stx_mask=STATX_ALL, stx_blksize=4096,
> stx_attributes=0, stx_nlink=1, stx_uid=0, stx_gid=0,
> stx_mode=S_IFREG|0755, stx_ino=17487, stx_size=425008, stx_blocks=832,
> stx_attributes_mask=STATX_ATTR_COMPRESSED|STATX_ATTR_IMMUTABLE|STATX_ATTR_APPEND|STATX_ATTR_NODUMP|STATX_ATTR_ENCRYPTED,
> stx_atime={tv_sec=1602625090, tv_nsec=600000000} /*
> 2020-10-13T21:38:10.600000000+0000 */, stx_btime={tv_sec=1602622592,
> tv_nsec=0} /* 2020-10-13T20:56:32+0000 */,
> stx_ctime={tv_sec=2147484284, tv_nsec=335721000},
> stx_mtime={tv_sec=2147484263, tv_nsec=545721000}, stx_rdev_major=0,
> stx_rdev_minor=0, stx_dev_major=179, stx_dev_minor=0}) = 0 <0.000494>
> test_y2038: error while loading shared libraries: librt.so.1: cannot
> stat shared object: Error 75 +++ exited with 127 +++
> 
> 
> Here the ctime and mtime are wrong - i.e. overflowed.
> 
> The system date is not changed:
> root@y2038arm:~# date
> Tue Oct 13 21:40:02 UTC 2020
> root@y2038arm:~# date +%s
> 1602625224
> 
> I will continue investigation tomorrow...

And now the penny has dropped...

The issue was that:

1. Adhemerval has added extra checks for overflowing the time in
{f}stat{at} patches - this is obviously good :-)

2. In my test system - I've been using "time" syscall (glibc function)
to get the seconds after epoch. This was not yet [*] converted to
support 64 bit time.

3. Some tests (like clock_gettime / clock_settime) require time
modifications after Y2038. Broken time() caused wrong data after
casting from __time_t to __time64_t and the proper time couldn't be
restored.

4. As a result - the date was wrong during the following tests
execution and {mac}time for external files (like librt.so.1,
libgcc_s.so) was set to wrong value.

5. Each subsequent call of {fl}stat{64} returned error due to date
overflow check.

Fix:

Convert time to support 64 bit time.

[*] I will send patches after build-many-glibcs.py and xcheck finish.

> 
> > 
> > The arm-linux-gnueabihf testing I did was on a aarch64 kernel
> > (4.12.13). Besides the make check without regression, I could run
> > system binaries with ./testrun.sh.
> > 
> > I will check on a different kernel/system with a 32-bit kernel.
> >   
> 
> 
> 
> 
> Best regards,
> 
> Lukasz Majewski
> 
> --
> 
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email:
> lukma@denx.de




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]