[PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Paul Eggert]

In glibc 2.34 on Linux kernels where time_t is traditionally 32-bit,
defining _FILE_OFFSET_BITS=64 and _TIME_BITS=64 makes time_t 64-bit.
Apps must define both macros.  Gnulib applications that use either
the largefile or the year2038 modules will want this behavior;
largefile because it deals with the off_t and ino_t components of
struct stat already, and so should also deal with time_t.
* NEWS, doc/largefile.texi, doc/posix-headers/time.texi:
Mention this.
* m4/largefile.m4: Override two macros even in Autoconf 2.70 and later.
(_AC_SYS_LARGEFILE_MACRO_VALUE): #undef $1 before #defining it, in
case some other Gnulib macro has #defined it.
(AC_SYS_LARGEFILE): Use AS_IF and AS_CASE to propagate AC_REQUIREs.
Invoke gl_YEAR2038_BODY if we need to set _FILE_OFFSET_BITS=64.
* m4/year2038.m4 (gl_YEAR2038_TEST_INCLUDES): New macro.
(gl_YEAR2038_BODY): New macro, with gl_YEAR2038’s old body; this
macro is designed to be used directly instead of being
AC_REQUIREd.  It takes an argument specifying whether 64-bit is
required.  Set _TIME_BITS=64 if this makes a difference in time_t
width when setting _FILE_OFFSET_BITS=64.  Do not warn about
32-bit time_t more than once.
* modules/largefile (Files): Add year2038.m4.
(Depends-on): Require gl_YEAR2038_EARLY.
---
 ChangeLog                   | 24 ++++++++++++++
 NEWS                        |  4 +++
 doc/largefile.texi          |  5 +--
 doc/posix-headers/time.texi | 13 ++++++--
 m4/largefile.m4             | 28 +++++++++--------
 m4/year2038.m4              | 63 +++++++++++++++++++++++++++++--------
 modules/largefile           |  7 +++++
 7 files changed, 113 insertions(+), 31 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index c325d80b2..b00dfd1fd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,29 @@
 2021-07-01  Paul Eggert  <eggert@cs.ucla.edu>
 
+	year2038: support glibc 2.34 _TIME_BITS=64
+	In glibc 2.34 on Linux kernels where time_t is traditionally 32-bit,
+	defining _FILE_OFFSET_BITS=64 and _TIME_BITS=64 makes time_t 64-bit.
+	Apps must define both macros.  Gnulib applications that use either
+	the largefile or the year2038 modules will want this behavior;
+	largefile because it deals with the off_t and ino_t components of
+	struct stat already, and so should also deal with time_t.
+	* NEWS, doc/largefile.texi, doc/posix-headers/time.texi:
+	Mention this.
+	* m4/largefile.m4: Override two macros even in Autoconf 2.70 and later.
+	(_AC_SYS_LARGEFILE_MACRO_VALUE): #undef $1 before #defining it, in
+	case some other Gnulib macro has #defined it.
+	(AC_SYS_LARGEFILE): Use AS_IF and AS_CASE to propagate AC_REQUIREs.
+	Invoke gl_YEAR2038_BODY if we need to set _FILE_OFFSET_BITS=64.
+	* m4/year2038.m4 (gl_YEAR2038_TEST_INCLUDES): New macro.
+	(gl_YEAR2038_BODY): New macro, with gl_YEAR2038’s old body; this
+	macro is designed to be used directly instead of being
+	AC_REQUIREd.  It takes an argument specifying whether 64-bit is
+	required.  Set _TIME_BITS=64 if this makes a difference in time_t
+	width when setting _FILE_OFFSET_BITS=64.  Do not warn about
+	32-bit time_t more than once.
+	* modules/largefile (Files): Add year2038.m4.
+	(Depends-on): Require gl_YEAR2038_EARLY.
+
 	relocatable-prog-wrapper: add m4 files
 	Add .m4 files needed for standalone relocatable-prog-wrapper, so
 	that ‘./gnulib-tool --test relocatable-prog-wrapper’ does not fail
diff --git a/NEWS b/NEWS
index def6cba3e..345afb189 100644
--- a/NEWS
+++ b/NEWS
@@ -66,6 +66,10 @@ User visible incompatible changes
 
 Date        Modules         Changes
 
+2021-07-01  largefile       AC_SYS_LARGEFILE now also arranges for time_t
+                            to be 64-bit on 32-bit GNU/Linux platforms
+                            that support it (glibc 2.34 or later).
+
 2021-03-21  fatal-signal    The function at_fatal_signal now returns an error
                             indicator.
 
diff --git a/doc/largefile.texi b/doc/largefile.texi
index 6d04b45b8..13572b47e 100644
--- a/doc/largefile.texi
+++ b/doc/largefile.texi
@@ -1,8 +1,9 @@
 @node Large File Support
 @section Large File Support
 
-The module provides support for files larger than 2 GB.
-To this effect, it ensures that @code{off_t} is a 64-bit integer type
+The module provides support for files larger than 2 GB, or with device
+or inode numbers or timestamps exceeding 32 bits.  To this effect, it
+ensures that types like @code{off_t} and @code{time_t} are 64-bit when possible,
 at least on the following platforms:
 glibc, Mac OS X, FreeBSD, NetBSD, OpenBSD, AIX, HP-UX, IRIX, Solaris,
 Cygwin, mingw, MSVC.
diff --git a/doc/posix-headers/time.texi b/doc/posix-headers/time.texi
index f1bac719e..b9be347c7 100644
--- a/doc/posix-headers/time.texi
+++ b/doc/posix-headers/time.texi
@@ -8,6 +8,12 @@ Gnulib module: time
 Portability problems fixed by Gnulib:
 @itemize
 @item
+On some platforms where @code{time_t} defaults to 32-bit but can be
+changed to 64-bit, functions like @code{stat} can fail with
+@code{errno == EOVERFLOW} when a 32-bit timestamp is out of range,
+such as with a file timestamp in the far future or past:
+glibc 2.34.
+@item
 @samp{struct timespec} is not defined on some platforms.
 @item
 The macro @code{TIME_UTC} is not defined on many platforms:
@@ -21,9 +27,10 @@ NetBSD 5.0.
 Portability problems not fixed by Gnulib:
 @itemize
 @item
-On platforms with 32-bit @code{time_t}, functions like @code{stat} can
-fail with @code{errno == EOVERFLOW} when a timestamp is out of range,
-such as with a file timestamp in the far future or past; on others,
+On platforms where @code{time_t} is always 32-bit, functions like
+@code{stat} can fail with @code{errno == EOVERFLOW} when a timestamp
+is out of range, such as with a file timestamp in the far future or
+past; on other such platforms,
 the functions silently return the low-order 32 bits of the correct
 timestamp.  These platforms will be obsolete when 32-bit @code{time_t}
 rolls around, which will occur in 2038 for the typical case when
diff --git a/m4/largefile.m4 b/m4/largefile.m4
index cadb16dc9..172a4da96 100644
--- a/m4/largefile.m4
+++ b/m4/largefile.m4
@@ -22,7 +22,8 @@ AC_DEFUN([gl_SET_LARGEFILE_SOURCE],
   esac
 ])
 
-# The following implementation works around a problem in autoconf <= 2.69;
+# Work around a problem in Autoconf through at least 2.71 on glibc 2.34+
+# with _TIME_BITS.  Also, work around a problem in autoconf <= 2.69:
 # AC_SYS_LARGEFILE does not configure for large inodes on Mac OS X 10.5,
 # or configures them incorrectly in some cases.
 m4_version_prereq([2.70], [], [
@@ -40,6 +41,7 @@ m4_define([_AC_SYS_LARGEFILE_TEST_INCLUDES],
                        && LARGE_OFF_T % 2147483647 == 1)
                       ? 1 : -1]];[]dnl
 ])
+])# m4_version_prereq 2.70
 
 
 # _AC_SYS_LARGEFILE_MACRO_VALUE(C-MACRO, VALUE,
@@ -54,7 +56,8 @@ m4_define([_AC_SYS_LARGEFILE_MACRO_VALUE],
     [AC_LANG_PROGRAM([$5], [$6])],
     [$3=no; break])
   m4_ifval([$6], [AC_LINK_IFELSE], [AC_COMPILE_IFELSE])(
-    [AC_LANG_PROGRAM([#define $1 $2
+    [AC_LANG_PROGRAM([#undef $1
+#define $1 $2
 $5], [$6])],
     [$3=$2; break])
   $3=unknown
@@ -80,9 +83,8 @@ rm -rf conftest*[]dnl
 AC_DEFUN([AC_SYS_LARGEFILE],
 [AC_ARG_ENABLE(largefile,
                [  --disable-largefile     omit support for large files])
-if test "$enable_largefile" != no; then
-
-  AC_CACHE_CHECK([for special C compiler options needed for large files],
+AS_IF([test "$enable_largefile" != no],
+ [AC_CACHE_CHECK([for special C compiler options needed for large files],
     ac_cv_sys_largefile_CC,
     [ac_cv_sys_largefile_CC=no
      if test "$GCC" != yes; then
@@ -107,15 +109,15 @@ if test "$enable_largefile" != no; then
     ac_cv_sys_file_offset_bits,
     [Number of bits in a file offset, on hosts where this is settable.],
     [_AC_SYS_LARGEFILE_TEST_INCLUDES])
-  if test $ac_cv_sys_file_offset_bits = unknown; then
-    _AC_SYS_LARGEFILE_MACRO_VALUE(_LARGE_FILES, 1,
-      ac_cv_sys_large_files,
-      [Define for large files, on AIX-style hosts.],
-      [_AC_SYS_LARGEFILE_TEST_INCLUDES])
-  fi
-fi
+  AS_CASE([$ac_cv_sys_file_offset_bits],
+    [unknown],
+      [_AC_SYS_LARGEFILE_MACRO_VALUE([_LARGE_FILES], [1],
+         [ac_cv_sys_large_files],
+         [Define for large files, on AIX-style hosts.],
+         [_AC_SYS_LARGEFILE_TEST_INCLUDES])],
+    [64],
+      [gl_YEAR2038_BODY([false])])])
 ])# AC_SYS_LARGEFILE
-])# m4_version_prereq 2.70
 
 # Enable large files on systems where this is implemented by Gnulib, not by the
 # system headers.
diff --git a/m4/year2038.m4 b/m4/year2038.m4
index 2534622fc..635ef124f 100644
--- a/m4/year2038.m4
+++ b/m4/year2038.m4
@@ -1,4 +1,4 @@
-# year2038.m4 serial 3
+# year2038.m4 serial 4
 dnl Copyright (C) 2017-2021 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -18,13 +18,29 @@ AC_DEFUN([gl_YEAR2038_EARLY],
   esac
 ])
 
-AC_DEFUN([gl_YEAR2038],
+# gl_YEAR2038_TEST_INCLUDES
+# -------------------------
+AC_DEFUN([gl_YEAR2038_TEST_INCLUDES],
+[[
+  #include <time.h>
+  /* Check that time_t can represent 2**63 - 1 correctly.
+     We can't simply define LARGE_TIME_T to be 9223372036854775807,
+     since some C++ compilers masquerading as C compilers
+     incorrectly reject 9223372036854775807.  */
+  #define LARGE_TIME_T (((time_t) 1 << 31 << 31) - 1 + ((time_t) 1 << 31 << 31))
+  int verify_time_t_range[(LARGE_TIME_T % 2147483629 == 721
+                           && LARGE_TIME_T % 2147483647 == 1)
+                          ? 1 : -1];
+]])
+
+# gl_YEAR2038_BODY(REQUIRE-64-BIT)
+----------------------------------
+AC_DEFUN([gl_YEAR2038_BODY],
 [
   dnl On many systems, time_t is already a 64-bit type.
   dnl On those systems where time_t is still 32-bit, it requires kernel
-  dnl and libc support to make it 64-bit. For glibc on Linux/x86, this
-  dnl is work in progress; see
-  dnl <https://sourceware.org/glibc/wiki/Y2038ProofnessDesign>.
+  dnl and libc support to make it 64-bit. For glibc 2.34 and later on Linux,
+  dnl defining _TIME_BITS=64 and _FILE_OFFSET_BITS=64 is needed on x86 and ARM.
   dnl
   dnl On native Windows, the system include files define types __time32_t
   dnl and __time64_t. By default, time_t is an alias of
@@ -36,13 +52,28 @@ AC_DEFUN([gl_YEAR2038],
   dnl __time32_t.
   AC_CACHE_CHECK([for 64-bit time_t], [gl_cv_type_time_t_64],
     [AC_COMPILE_IFELSE(
-       [AC_LANG_PROGRAM(
-          [[#include <time.h>
-            int verify_time_t_size[sizeof (time_t) >= 8 ? 1 : -1];
-          ]],
-          [[]])],
+       [AC_LANG_SOURCE([gl_YEAR2038_TEST_INCLUDES])],
        [gl_cv_type_time_t_64=yes], [gl_cv_type_time_t_64=no])
     ])
+  if test "$gl_cv_type_time_t_64" = no; then
+    AC_CACHE_CHECK([for 64-bit time_t with _TIME_BITS=64],
+      [gl_cv_type_time_t_bits_macro],
+      [AC_COMPILE_IFELSE(
+         [AC_LANG_SOURCE([[#define _TIME_BITS 64
+                           #define _FILE_OFFSET_BITS 64
+                           ]gl_YEAR2038_TEST_INCLUDES])],
+         [gl_cv_type_time_t_bits_macro=yes],
+         [gl_cv_type_time_t_bits_macro=no])
+      ])
+    if test "$gl_cv_type_time_t_bits_macro" = yes; then
+      AC_DEFINE([_TIME_BITS], [64],
+        [Number of bits in a timestamp, on hosts where this is settable.])
+      dnl AC_SYS_LARGFILE also defines this; it's OK if we do too.
+      AC_DEFINE([_FILE_OFFSET_BITS], [64],
+        [Number of bits in a file offset, on hosts where this is settable.])
+      gl_cv_type_time_t_64=yes
+    fi
+  fi
   if test $gl_cv_type_time_t_64 = no; then
     AC_COMPILE_IFELSE(
       [AC_LANG_SOURCE(
@@ -53,18 +84,24 @@ AC_DEFUN([gl_YEAR2038],
            #endif
          ]])],
       [AC_MSG_FAILURE([This package requires a 64-bit 'time_t' type. Remove _USE_32BIT_TIME_T from the compiler flags.])],
-      [# If TIME_T_32_BIT_OK is "no" (the default) and not cross-compiling
+      [# If not cross-compiling and $1 says we should check,
        # and 'touch' works with a large timestamp, then evidently 64-bit time_t
        # is desired and supported, so fail and ask the builder to fix the
        # problem.  Otherwise, just warn the builder.
-       if test "${TIME_T_32_BIT_OK-no}" = no \
+       if $1 \
           && test $cross_compiling = no \
           && TZ=UTC0 touch -t 210602070628.16 conftest.time 2>/dev/null; then
          rm -f conftest.time
          AC_MSG_FAILURE([This package requires a 64-bit 'time_t' type, which your system appears to support. You might try configuring with 'CPPFLAGS="-m64" LDFLAGS="-m64"'. To build with a 32-bit time_t anyway (not recommended), configure with 'TIME_T_32_BIT_OK=yes'.])
-       else
+       elif test "$gl_warned_about_64_bit_time_t" != yes; then
          AC_MSG_WARN([This package requires a 64-bit 'time_t' type if there is any way to access timestamps outside the year range 1901-2038 on your platform. Perhaps you should configure with 'CPPFLAGS="-m64" LDFLAGS="-m64"'?])
+         gl_warned_about_64_bit_time_t=yes
        fi
       ])
   fi
 ])
+
+AC_DEFUN([gl_YEAR2038],
+[
+  gl_YEAR2038_BODY([test "${TIME_T_32_BIT_OK-no}" = no])
+])
diff --git a/modules/largefile b/modules/largefile
index fcae113db..8eb438a8e 100644
--- a/modules/largefile
+++ b/modules/largefile
@@ -1,13 +1,20 @@
 Description:
 Support for files larger than 2 GB.
 
+Comment:
+This module should not be used as a dependency from a test module,
+otherwise when this module occurs as a tests-related module, it will
+have side effects on the compilation of the main modules in lib/.
+
 Files:
 m4/largefile.m4
+m4/year2038.m4
 
 Depends-on:
 
 configure.ac-early:
 AC_REQUIRE([AC_SYS_LARGEFILE])
+AC_REQUIRE([gl_YEAR2038_EARLY])
 
 configure.ac:
 AC_REQUIRE([gl_LARGEFILE])
-- 
2.30.2

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Florian Weimer]

* Paul Eggert:

> In glibc 2.34 on Linux kernels where time_t is traditionally 32-bit,
> defining _FILE_OFFSET_BITS=64 and _TIME_BITS=64 makes time_t 64-bit.
> Apps must define both macros.  Gnulib applications that use either
> the largefile or the year2038 modules will want this behavior;
> largefile because it deals with the off_t and ino_t components of
> struct stat already, and so should also deal with time_t.

Won't this be a very disruptive change to distributions, whose system
libraries have not switched to 64-bit time_t on 32-bit?

gnulib should not try to force a different distribution default.  I'm
worried that this will lead to distributions abandoning 32-bit i386
support altogether because the support cost is too high—and you can't
even run legacy binaries anymore.

Thanks,
Florian

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Bruno Haible]

Hi Florian,

> > In glibc 2.34 on Linux kernels where time_t is traditionally 32-bit,
> > defining _FILE_OFFSET_BITS=64 and _TIME_BITS=64 makes time_t 64-bit.
> > Apps must define both macros.  Gnulib applications that use either
> > the largefile or the year2038 modules will want this behavior;
> > largefile because it deals with the off_t and ino_t components of
> > struct stat already, and so should also deal with time_t.
> 
> Won't this be a very disruptive change to distributions, whose system
> libraries have not switched to 64-bit time_t on 32-bit?
> 
> gnulib should not try to force a different distribution default.  I'm
> worried that this will lead to distributions abandoning 32-bit i386
> support altogether because the support cost is too high—and you can't
> even run legacy binaries anymore.

I don't understand your points regarding "very disruptive change",
"distribution default", and "can't even run legacy binaries". Probably
you have something in mind that differs from my understanding.

In my understanding, a change like this one propagates to the tarballs
that make use of Gnulib. For example, GNU tar will, starting with the
next version, contain logic that has the effect of adding
  #define _TIME_BITS 64
to the config.h of that package. Thus, GNU tar and GNU mt will, on
glibc ≥ 2.34 systems, be internally using a different time_t type than
programs that don't use Gnulib (e.g. util-linux) and programs that use
older versions of Gnulib (e.g. GNU clisp).

From the perspective of the distributions, this is a no-op, IMO.

The only problem that I see is with *libraries* that have an API that
references the time_t type. It is well-known that when a library
  - references off_t or 'struct stat' in its API, and
  - was built with AC_SYS_LARGEFILE in effect,
the packages that use this library also have to be built with
AC_SYS_LARGEFILE. This has caused problems in the past, when
_FILE_OFFSET_BITS=64 was introduced (ca. 2000-2005).

Now, Paul's change has the effect that when a library
  - references time_t in its API, and
  - either it was built with Gnulib module 'largefile',
    or it was built without -D_TIME_BITS=64 but the package that uses it
    is built with  Gnulib module 'largefile' or -D_TIME_BITS=64,
there will be similar problems.

Are there many such libraries? Not many, but come important ones.
A quick look in my /usr/include lists
  libcurl
  glib, gtk
  gdbm
  gnutls, openssl
  libpng
  wxWidgets
  X11/Xos_r.h

Also, there might be problems with socket/pipe protocols which
transfer time_t values within a stream of data. Which such protocols
exist?

I don't see big problems with distribution vendors, since 56%
of the distributions have already abandoned i386 ports by now [1],
and more will follow suit. The rest of the distros can easily
add -D_TIME_BITS=64 to their common compilation flags.

However, I do see some bug reports coming from people who still use
such old i386 distros and try to build packages that rely on libraries
that use time_t in their APIs.

What is the best approach?
  - A package-by-package approach that allows each package maintainer
    to choose their preference? That would mean, to keep 'largefile'
    and 'year2038' as independent modules.
  - Or a global approach, like Paul's patch does?

From the times _FILE_OFFSET_BITS was introduced, I remember that
the package-by-package approach led to major trouble, and that the
global approach — use -D_FILE_OFFSET_BITS=64 in all compilations —
was ultimately the simplest way forward.

Bruno

[1] https://distrowatch.com/search.php#advanced

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Paul Eggert]

On 7/2/21 3:29 PM, Bruno Haible wrote:

> the package-by-package approach led to major trouble, and that the
> global approach — use -D_FILE_OFFSET_BITS=64 in all compilations —
> was ultimately the simplest way forward.

Yes, that's pretty much my thought as well.

Like Bruno, I didn't understand Florian's comments. Any apps and/or 
libraries compiled with _TIME_BITS=64 will have the API issues Bruno 
mentioned, even if the apps and/or libraries don't use Gnulib. I don't 
see why these issues would cause distros to abandon 32-bit i386 simply 
because of Gnulib.

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Florian Weimer]

* Bruno Haible:

> Hi Florian,
>
>> > In glibc 2.34 on Linux kernels where time_t is traditionally 32-bit,
>> > defining _FILE_OFFSET_BITS=64 and _TIME_BITS=64 makes time_t 64-bit.
>> > Apps must define both macros.  Gnulib applications that use either
>> > the largefile or the year2038 modules will want this behavior;
>> > largefile because it deals with the off_t and ino_t components of
>> > struct stat already, and so should also deal with time_t.
>> 
>> Won't this be a very disruptive change to distributions, whose system
>> libraries have not switched to 64-bit time_t on 32-bit?
>> 
>> gnulib should not try to force a different distribution default.  I'm
>> worried that this will lead to distributions abandoning 32-bit i386
>> support altogether because the support cost is too high—and you can't
>> even run legacy binaries anymore.
>
> I don't understand your points regarding "very disruptive change",
> "distribution default", and "can't even run legacy binaries". Probably
> you have something in mind that differs from my understanding.
>
> In my understanding, a change like this one propagates to the tarballs
> that make use of Gnulib. For example, GNU tar will, starting with the
> next version, contain logic that has the effect of adding
>   #define _TIME_BITS 64
> to the config.h of that package. Thus, GNU tar and GNU mt will, on
> glibc ≥ 2.34 systems, be internally using a different time_t type than
> programs that don't use Gnulib (e.g. util-linux) and programs that use
> older versions of Gnulib (e.g. GNU clisp).

I assume GNU clisp (at least in a full build) need to link to some
system libraries which are not dual ABI (and probably never will be).
If gnulib forces the use of time64 mode, then it creates a push towards
time64 mode in those libraries, too.  At that point, these libraries
will no longer be usable for running older binaries (in at least some
cases; in others, the time_t symbols are not actually used).

> From the perspective of the distributions, this is a no-op, IMO.

It is not, gnulib is pushing things in one particular direction, a
direction that not everyone working on the GNU toolchain agrees with.

> The only problem that I see is with *libraries* that have an API that
> references the time_t type. It is well-known that when a library
>   - references off_t or 'struct stat' in its API, and
>   - was built with AC_SYS_LARGEFILE in effect,
> the packages that use this library also have to be built with
> AC_SYS_LARGEFILE. This has caused problems in the past, when
> _FILE_OFFSET_BITS=64 was introduced (ca. 2000-2005).

There are some major differences to _FILE_OFFSET_BITS=64:

There weren't 20+ years of backwards compatibility in 2005, with a large
set of legacy applications.  Today, i386 without the ability run legacy
programs is fairly useless and perhaps not worth maintaining.

64-bit file offsets enabled real use cases.  People usually couldn't
address those in another way, given that LP64 CPUs and userspace wasn't
yet mainstream.

> I don't see big problems with distribution vendors, since 56%
> of the distributions have already abandoned i386 ports by now [1],
> and more will follow suit. The rest of the distros can easily
> add -D_TIME_BITS=64 to their common compilation flags.

I think those 56% count installation images, not installable i386
library packages on x86-64 systems.

Thanks,
Florian

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Paul Eggert]

On 7/5/21 7:32 AM, Florian Weimer wrote:

> I assume GNU clisp (at least in a full build) need to link to some
> system libraries which are not dual ABI (and probably never will be).
> If gnulib forces the use of time64 mode, then it creates a push towards
> time64 mode in those libraries, too.

Only in libraries that expose time_t (or time_t-dependent types like 
struct timespec and struct stat) directly to their users, right? For 
example, libselinux is unaffected by this issue even though it calls 
'lstat' internally, because its user-side ABI is unaffected by time_t width.

I take your point, though, that there will be a hassle with libraries 
whose user-side ABIs depend on time_t width. I expect distros will 
migrate these libraries and their users to _TIME_BITS=64 in an 
all-or-nothing way.

But again, this is not strictly a Gnulib issue. Apps like Coreutils 
should be fine even with the new Gnulib, as Coreutils doesn't use any of 
the problematic libraries. Conversely, apps that '#define _TIME_BITS 64' 
directly and use GLib (or whatever) can have the problem, even if they 
don't use Gnulib.

> It is not, gnulib is pushing things in one particular direction, a
> direction that not everyone working on the GNU toolchain agrees with.

That's OK; we needn't have universal agreement on this point, which is a 
good thing because it'd be impractical to insist on universal agreement. 
  There is a way to build with 32-bit time_t even in Gnulib-using apps, 
so distros wanting to stick with 32-bit time_t can build Gnulib-using 
apps with the appropriate flags so that they're still living in the 
32-bit time_t world for now.

> There are some major differences to _FILE_OFFSET_BITS=64:
> 
> There weren't 20+ years of backwards compatibility in 2005

Not on Linux-based platforms of course, but there were on BSD-based 
platforms with 20+ years of backwards compatibility back then (SunOS 
comes to mind). Admittedly there were fewer computers back then but the 
compatibility issues were quite extensive.

> 64-bit file offsets enabled real use cases. 

Designing devices now, that will be used past 2038, is a real use case.

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Bruno Haible]

Hi Paul,

>   There is a way to build with 32-bit time_t even in Gnulib-using apps, 
> so distros wanting to stick with 32-bit time_t can build Gnulib-using 
> apps with the appropriate flags so that they're still living in the 
> 32-bit time_t world for now.

What are these appropriate flags?

I'd like to understand
  - whether these flags are simple or complicated,
  - whether we really do need a module dependency 'largefile' -> 'year2038',
  - how to best document these things. (Btw, currently these are two sections
    in the chapter "Native Windows Support". I should reorganize this.)

Bruno

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Bruno Haible]

Florian Weimer wrote:

> 64-bit file offsets enabled real use cases.

Year 2038 is also a real use-case. It is not so rare that machines are
being used for 15 years. (I still occasionally use a 14-years old
computer, and had a washing machine that lasted 25 years.)
Year 2038 is less than 17 years away. So, it is time to do something for
year 2038 now, not in five years.

> I assume GNU clisp (at least in a full build) need to link to some
> system libraries which are not dual ABI (and probably never will be).
> If gnulib forces the use of time64 mode, then it creates a push towards
> time64 mode in those libraries, too.  At that point, these libraries
> will no longer be usable for running older binaries (in at least some
> cases; in others, the time_t symbols are not actually used).
> ...
> gnulib is pushing things in one particular direction, a
> direction ...

Let me try to summarize your arguments, the way I understand them.

  1) The ability to run older binaries is essential for nearly all
     distros.

  2) On i386, 32-bit time_t and 64-bit time_t are not binary compatible,
     when used in the public API of a shared library. Assume an existing
     old binary relies on /usr/lib/libfoo.so.5 and uses its API with
     32-bit time_t assumption. Then this library must stay in place with
     the same API.

  3) The distribution can provide a libfoo.so compiled with 64-bit
     time_t, but it MUST reside in a different file.

The distribution can provide a second copy of libfoo.so. Two mechanisms
come to mind, but there are certainly more:
  - It could provide it as /usr/lib/time64/libfoo.so.5.
  - It could provide it with a higher major version number:
    /usr/lib/libfoo.so.105.

Either way, it's going to be a hassle for the distro.

In other words, the glibc 2.34 _TIME_BITS=64 support is an important
part of solving the year 2038 problem, but it's not complete yet.

Pieces that are missing, AFAICS, are:

 A) Possibly some glibc "magic" with shared library versioning would
    make this situation simpler? Or is the combination of ldconfig and
    LD_LIBRARY_PATH etc. sufficient?

 B) A writeup for distributors, what is the recommended way to handle
    the situation.
    There are several _possible_ ways to handle it. But Linux distros
    aim at being compatible at the binary level, and that requires
    a _common_ approch among distros. IMO, the Linux Standard Base (LSB)
    is the forum where such things should be standardized.
    Have the LSB people already been involved in the discussion?

Regarding Gnulib, it is pretty clear — and necessary in order to avoid
bug reports — that we need to give the choice, to the person who builds
a GNU package, to choose among 32-bit and 64-bit time_t. (Giving the
choice to the maintainer of that package, when he prepares the tarball,
is not enough.) In other words, we need an
  AC_ARG_ENABLE([year2038], ...)
like we already have an
  AC_ARG_ENABLE([largefile], ...)

Whether additional configure options are needed in order to pick the
shared libraries that are compatible with this choice, depends on the
outcome of (B) above.

Bruno

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1099 bytes --]

On 7/5/21 6:34 PM, Bruno Haible wrote:

> What are these appropriate flags?

It was "./configure TIME_T_32_BIT_OK=yes". Though this wasn't documented 
well and perhaps needed other options, and it's different in style from 
how largefile worked.

> I'd like to understand
>    - whether these flags are simple or complicated,

They should be simple.

>    - whether we really do need a module dependency 'largefile' -> 'year2038',
>    - how to best document these things.

It shouldn't be a module dependency, since year2038 means "I must have 
64-bit time_t or I won't build" whereas largefile merely means "I want 
the widest off_t/dev_t/ino_t/time_t available". That being said, 
largefile can use some of the year2038 macros.

> we need an
>   AC_ARG_ENABLE([year2038], ...)
> like we already have an
>   AC_ARG_ENABLE([largefile], ...)

Yes, something like that would be better than the TIME_T_32_BIT_OK flag.

I installed the attached patches into Gnulib to implement the above 
suggestions. The first one is just a doc fix; the second one does the 
real work.

[-- Attachment #2: 0001-doc-clarify-which-module-fixes-32-bit-time_t.patch --]
[-- Type: text/x-patch, Size: 2189 bytes --]

From 8d8007df5e302f7f74faefc0f87983adbbc1c8c8 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 6 Jul 2021 15:08:05 -0700
Subject: [PATCH 1/2] doc: clarify which module fixes 32-bit time_t

* doc/posix-headers/time.texi: Make it clear that the 32-bit
time_t problem is addressed by the year2038 module, not
by the time module.
---
 ChangeLog                   |  7 +++++++
 doc/posix-headers/time.texi | 16 ++++++++++------
 2 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index b00dfd1fd..3cd55e5ba 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2021-07-06  Paul Eggert  <eggert@cs.ucla.edu>
+
+	doc: clarify which module fixes 32-bit time_t
+	* doc/posix-headers/time.texi: Make it clear that the 32-bit
+	time_t problem is addressed by the year2038 module, not
+	by the time module.
+
 2021-07-01  Paul Eggert  <eggert@cs.ucla.edu>
 
 	year2038: support glibc 2.34 _TIME_BITS=64
diff --git a/doc/posix-headers/time.texi b/doc/posix-headers/time.texi
index b9be347c7..3d8164645 100644
--- a/doc/posix-headers/time.texi
+++ b/doc/posix-headers/time.texi
@@ -8,12 +8,6 @@ Gnulib module: time
 Portability problems fixed by Gnulib:
 @itemize
 @item
-On some platforms where @code{time_t} defaults to 32-bit but can be
-changed to 64-bit, functions like @code{stat} can fail with
-@code{errno == EOVERFLOW} when a 32-bit timestamp is out of range,
-such as with a file timestamp in the far future or past:
-glibc 2.34.
-@item
 @samp{struct timespec} is not defined on some platforms.
 @item
 The macro @code{TIME_UTC} is not defined on many platforms:
@@ -24,6 +18,16 @@ expressions:
 NetBSD 5.0.
 @end itemize
 
+Portability problems fixed by the Gnulib module @code{year2038}:
+@itemize
+@item
+On some platforms where @code{time_t} defaults to 32-bit but can be
+changed to 64-bit, functions like @code{stat} can fail with
+@code{errno == EOVERFLOW} when a 32-bit timestamp is out of range,
+such as with a file timestamp in the far future or past:
+glibc 2.34.
+@end itemize
+
 Portability problems not fixed by Gnulib:
 @itemize
 @item
-- 
2.30.2


[-- Attachment #3: 0002-year2038-Add-disable-year2038-option.patch --]
[-- Type: text/x-patch, Size: 6414 bytes --]

From 12169c73bc0008553e6786cd8ef39ccf28cc1519 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 6 Jul 2021 15:21:44 -0700
Subject: [PATCH 2/2] year2038: Add --disable-year2038 option
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Also, document this stuff better.  Suggested by Bruno Haible in:
https://lists.gnu.org/r/bug-gnulib/2021-07/msg00011.html
* doc/year2038.texi: Document --disable-year2038.
* m4/year2038.m4 (gl_YEAR2038_BODY): Support ‘./configure
--disable-year2038’ to disable 64-bit time_t when that is not the
default.  Arg is now either empty or nonempty (not a shell
command) and is evaluated at m4 expansion time instead of at
runtime; all callers changed.
---
 ChangeLog         | 10 ++++++++++
 doc/year2038.texi | 19 +++++++++++++++++--
 m4/largefile.m4   |  2 +-
 m4/year2038.m4    | 23 ++++++++++++++---------
 4 files changed, 42 insertions(+), 12 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 3cd55e5ba..2eaf00b59 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,15 @@
 2021-07-06  Paul Eggert  <eggert@cs.ucla.edu>
 
+	year2038: Add --disable-year2038 option
+	Also, document this stuff better.  Suggested by Bruno Haible in:
+	https://lists.gnu.org/r/bug-gnulib/2021-07/msg00011.html
+	* doc/year2038.texi: Document --disable-year2038.
+	* m4/year2038.m4 (gl_YEAR2038_BODY): Support ‘./configure
+	--disable-year2038’ to disable 64-bit time_t when that is not the
+	default.  Arg is now either empty or nonempty (not a shell
+	command) and is evaluated at m4 expansion time instead of at
+	runtime; all callers changed.
+
 	doc: clarify which module fixes 32-bit time_t
 	* doc/posix-headers/time.texi: Make it clear that the 32-bit
 	time_t problem is addressed by the year2038 module, not
diff --git a/doc/year2038.texi b/doc/year2038.texi
index de85bc6d2..213fda235 100644
--- a/doc/year2038.texi
+++ b/doc/year2038.texi
@@ -6,5 +6,20 @@ that will likely occur in the year 2038, for programs that use a 32-bit
 @samp{time_t} type.  See @url{https://en.wikipedia.org/wiki/Year_2038_problem}
 for details.
 
-The gnulib module @samp{year2038} attempts to avoid this problem, by
-ensuring that @code{time_t} is a 64-bit type.
+The Gnulib module @samp{year2038} attempts to avoid this problem, by
+ensuring that @code{time_t} is a 64-bit type and by causing
+@code{configure} to fail otherwise.
+
+The Gnulib module @samp{largefile} also attempts to avoid this problem
+when possible, because @samp{largefile} enables the widest
+file-related types supported by the system and @code{time_t} is one of
+those types.  However, @code{largefile} does not cause
+@code{configure} to fail when the year 2038 problem is not avoidable.
+
+The Gnulib @samp{largefile} and @samp{year2038} modules are
+recommended for any package that might be used after the year 2038 on
+32-bit platforms.  However, if you build such a package you can
+disable its use of 64-bit @code{time_t} by giving the
+@code{--disable-year2038} option to @code{configure}.  This may be
+useful if the package links to other libraries whose user-facing ABIs
+still require @code{time_t} to be 32-bit on your platform.
diff --git a/m4/largefile.m4 b/m4/largefile.m4
index 172a4da96..fbde5e664 100644
--- a/m4/largefile.m4
+++ b/m4/largefile.m4
@@ -116,7 +116,7 @@ AS_IF([test "$enable_largefile" != no],
          [Define for large files, on AIX-style hosts.],
          [_AC_SYS_LARGEFILE_TEST_INCLUDES])],
     [64],
-      [gl_YEAR2038_BODY([false])])])
+      [gl_YEAR2038_BODY([])])])
 ])# AC_SYS_LARGEFILE
 
 # Enable large files on systems where this is implemented by Gnulib, not by the
diff --git a/m4/year2038.m4 b/m4/year2038.m4
index 635ef124f..ad7f30358 100644
--- a/m4/year2038.m4
+++ b/m4/year2038.m4
@@ -1,4 +1,4 @@
-# year2038.m4 serial 4
+# year2038.m4 serial 5
 dnl Copyright (C) 2017-2021 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -37,6 +37,10 @@ AC_DEFUN([gl_YEAR2038_TEST_INCLUDES],
 ----------------------------------
 AC_DEFUN([gl_YEAR2038_BODY],
 [
+ AC_ARG_ENABLE([year2038],
+   [  --disable-year2038      omit support for timestamps past the year 2038])
+ AS_IF([test "$enable_year2038" != no],
+ [
   dnl On many systems, time_t is already a 64-bit type.
   dnl On those systems where time_t is still 32-bit, it requires kernel
   dnl and libc support to make it 64-bit. For glibc 2.34 and later on Linux,
@@ -88,20 +92,21 @@ AC_DEFUN([gl_YEAR2038_BODY],
        # and 'touch' works with a large timestamp, then evidently 64-bit time_t
        # is desired and supported, so fail and ask the builder to fix the
        # problem.  Otherwise, just warn the builder.
-       if $1 \
-          && test $cross_compiling = no \
-          && TZ=UTC0 touch -t 210602070628.16 conftest.time 2>/dev/null; then
-         rm -f conftest.time
-         AC_MSG_FAILURE([This package requires a 64-bit 'time_t' type, which your system appears to support. You might try configuring with 'CPPFLAGS="-m64" LDFLAGS="-m64"'. To build with a 32-bit time_t anyway (not recommended), configure with 'TIME_T_32_BIT_OK=yes'.])
-       elif test "$gl_warned_about_64_bit_time_t" != yes; then
+       m4_ifval([$1],
+         [if test $cross_compiling = no \
+             && TZ=UTC0 touch -t 210602070628.16 conftest.time 2>/dev/null; then
+            rm -f conftest.time
+            AC_MSG_FAILURE([This package requires a 64-bit 'time_t' type, which your system appears to support. You might try configuring with 'CPPFLAGS="-m64" LDFLAGS="-m64"'. To build with a 32-bit time_t anyway (not recommended), configure with '--disable-year2038'.])
+          fi])
+       if test "$gl_warned_about_64_bit_time_t" != yes; then
          AC_MSG_WARN([This package requires a 64-bit 'time_t' type if there is any way to access timestamps outside the year range 1901-2038 on your platform. Perhaps you should configure with 'CPPFLAGS="-m64" LDFLAGS="-m64"'?])
          gl_warned_about_64_bit_time_t=yes
        fi
       ])
-  fi
+  fi])
 ])
 
 AC_DEFUN([gl_YEAR2038],
 [
-  gl_YEAR2038_BODY([test "${TIME_T_32_BIT_OK-no}" = no])
+  gl_YEAR2038_BODY([require-64-bit])
 ])
-- 
2.30.2

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Florian Weimer]

* Bruno Haible:

> Florian Weimer wrote:
>
>> 64-bit file offsets enabled real use cases.
>
> Year 2038 is also a real use-case. It is not so rare that machines are
> being used for 15 years. (I still occasionally use a 14-years old
> computer, and had a washing machine that lasted 25 years.)
> Year 2038 is less than 17 years away. So, it is time to do something for
> year 2038 now, not in five years.

Y2038 support requires recompilation.  If you are able to do that, why
not recompile for a 64-bit architecture?

>> I assume GNU clisp (at least in a full build) need to link to some
>> system libraries which are not dual ABI (and probably never will be).
>> If gnulib forces the use of time64 mode, then it creates a push towards
>> time64 mode in those libraries, too.  At that point, these libraries
>> will no longer be usable for running older binaries (in at least some
>> cases; in others, the time_t symbols are not actually used).
>> ...
>> gnulib is pushing things in one particular direction, a
>> direction ...
>
> Let me try to summarize your arguments, the way I understand them.
>
>   1) The ability to run older binaries is essential for nearly all
>      distros.
>
>   2) On i386, 32-bit time_t and 64-bit time_t are not binary compatible,
>      when used in the public API of a shared library. Assume an existing
>      old binary relies on /usr/lib/libfoo.so.5 and uses its API with
>      32-bit time_t assumption. Then this library must stay in place with
>      the same API.
>
>   3) The distribution can provide a libfoo.so compiled with 64-bit
>      time_t, but it MUST reside in a different file.

I think there is also the possibility of a dual ABI, see below.

> Pieces that are missing, AFAICS, are:
>
>  A) Possibly some glibc "magic" with shared library versioning would
>     make this situation simpler? Or is the combination of ldconfig and
>     LD_LIBRARY_PATH etc. sufficient?

This probably needs per-package/component work to enable dual ABI,
similar to what glibc did for its time_t interfaces.  There is no linker
magic involved, it's either symbol redirects (using a GCC extension) or
a preprocessor macro.  A dual ABI avoids the need for new soname and the
introduction of symbol versioning (so that the object can be loaded
twice into the same process with different ABIs).

I don't expect many upstreams to support this effort.

>  B) A writeup for distributors, what is the recommended way to handle
>     the situation.
>     There are several _possible_ ways to handle it. But Linux distros
>     aim at being compatible at the binary level, and that requires
>     a _common_ approch among distros. IMO, the Linux Standard Base (LSB)
>     is the forum where such things should be standardized.
>     Have the LSB people already been involved in the discussion?

LSB is quite dead, and it never covered the interesting packages anyway.

I can see distributions building 32-bit Arm and a *new*, separate
variant of i386 for 64-bit time_t, and the original i386 port remains at
32-bit.  The new i386 port would have a glibc that defaults to 64-bit
time_t.  Two separate i386 ports seem to require the least human
resources to maintain.  If that's the chosen approach, gnulib should
just use whatever the default time_t size is, and not attempt to
override it.

Thanks,
Florian

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Paul Eggert]

On 7/7/21 1:45 AM, Florian Weimer wrote:

> Y2038 support requires recompilation.  If you are able to do that, why
> not recompile for a 64-bit architecture?

Doesn't this argue against _TIME_BITS=64 in general? It seems to be 
saying that one should just recompile for 64-bit, and never use 
_TIME_BITS=64.

> This probably needs per-package/component work to enable dual ABI,
> similar to what glibc did for its time_t interfaces....
> I don't expect many upstreams to support this effort.

Agreed.
> Two separate i386 ports seem to require the least human
> resources to maintain.

That's a reasonable approach and if people want to do that they can, 
even with the latest Gnulib and the next version of Glibc.

However, people who want to run old binaries will surely stick to the 
32-bit-time_t i386 port, which means they won't use the 64-bit-time_t 
i386 port. So it's not clear to me that they will cotton to this approach.

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Florian Weimer]

* Paul Eggert:

> On 7/7/21 1:45 AM, Florian Weimer wrote:
>
>> Y2038 support requires recompilation.  If you are able to do that, why
>> not recompile for a 64-bit architecture?
>
> Doesn't this argue against _TIME_BITS=64 in general? It seems to be
> saying that one should just recompile for 64-bit, and never use 
> _TIME_BITS=64.

I think it does, but apparently 32-bit Arm is an outlier, related to
DRAM sizes.  I'm still not convinced that glibc needs to support that,
but the community wasn't opposed to it.

>> This probably needs per-package/component work to enable dual ABI,
>> similar to what glibc did for its time_t interfaces....
>> I don't expect many upstreams to support this effort.
>
> Agreed.
>> Two separate i386 ports seem to require the least human
>> resources to maintain.
>
> That's a reasonable approach and if people want to do that they can,
> even with the latest Gnulib and the next version of Glibc.
>
> However, people who want to run old binaries will surely stick to the
> 32-bit-time_t i386 port, which means they won't use the 64-bit-time_t 
> i386 port. So it's not clear to me that they will cotton to this approach.

Sorry, I don't understand.  Which approach?

I expect the legacy i386 port to be the main one.

Thanks,
Florian

Re: [PATCH] year2038: support glibc 2.34 _TIME_BITS=64

[Paul Eggert]

On 7/8/21 12:36 AM, Florian Weimer wrote:
>
>> Doesn't this argue against _TIME_BITS=64 in general? It seems to be
>> saying that one should just recompile for 64-bit, and never use
>> _TIME_BITS=64.
> I think it does, but apparently 32-bit Arm is an outlier, related to
> DRAM sizes.  I'm still not convinced that glibc needs to support that,
> but the community wasn't opposed to it.

It's a tricky situation then, since 32-bit ARM is evidently intended to 
last past 2038, which means Coreutils etc. built now should be built to 
survive past 2038, hence the Gnulib change. And once that snowball 
starts rolling down the hill it's going to keep rolling until it reaches 
the bottom.


>>
>>> Two separate i386 ports seem to require the least human
>>> resources to maintain.
>> That's a reasonable approach and if people want to do that they can,
>> even with the latest Gnulib and the next version of Glibc.
>>
>> However, people who want to run old binaries will surely stick to the
>> 32-bit-time_t i386 port, which means they won't use the 64-bit-time_t
>> i386 port. So it's not clear to me that they will cotton to this approach.
> Sorry, I don't understand.  Which approach?

I meant the approach of having two i386 ports (or two ARM ports, for 
that matter). People who want to run old binaries will insist on the 
32-bit time_t port, which means the 64-bit time_t port won't be used 
those people. In other words I think we're agreeing.