From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 0F0BB1F461 for ; Wed, 17 Jul 2019 21:58:05 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type; q=dns; s=default; b=SDkN7 BUX/YlNoNsC0FOkMq00qZnYh1RE93BSchyZQBDPGV6ENbOu+vp/chsVmz0veKFyd EP0nhpSVGaVVtVxH82nP4Ejr5jaNpgjNmn+QBV8qp1DazgYd69sc/T9zyER/t7So ZRqi+bDVZrMH76/O52IDW4exX2ZuYEGU1u+yn0= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type; s=default; bh=qVWbDQUOq+g GAq1Ry/O9UNP8kd8=; b=McOvdGdO7clmNswJKYUD8Thr27ch3jbcG1sC4ganlzM 0WSYQFHAAfLOn3y2mQtEkFi+IErqnCfnKEbsJDxDvpfPi3hUY6lBezSfP6NKLuPm 43NUI0ibuJsHxH19jZccSTNddGesr6Twv8QvooC106mWmEjPjFtukJmbNqMtQjpc = Received: (qmail 117827 invoked by alias); 17 Jul 2019 21:58:02 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 117464 invoked by uid 89); 17 Jul 2019 21:58:02 -0000 Authentication-Results: sourceware.org; auth=none X-HELO: mail-out.m-online.net Date: Wed, 17 Jul 2019 23:57:48 +0200 From: Lukasz Majewski To: Rich Felker Cc: Arnd Bergmann , Wolfgang Denk , Florian Weimer , GNU C Library , Joseph Myers Subject: Re: Accelerating Y2038 glibc fixes Message-ID: <20190717235748.2552834a@jawa> In-Reply-To: <20190717175026.GM1506@brightrain.aerifal.cx> References: <20190712072103.D3DBC24003A@gemini.denx.de> <874l3mjgi6.fsf@oldenburg2.str.redhat.com> <20190716145216.1C7CE240085@gemini.denx.de> <20190717175026.GM1506@brightrain.aerifal.cx> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/YrTS5Qx9kJFxvPX/ERY1/=X"; protocol="application/pgp-signature" --Sig_/YrTS5Qx9kJFxvPX/ERY1/=X Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi Rich, Thanks for sharing your view. > On Wed, Jul 17, 2019 at 04:15:25PM +0200, Arnd Bergmann wrote: > > On Tue, Jul 16, 2019 at 4:52 PM Wolfgang Denk wrote: =20 > > > In message <874l3mjgi6.fsf@oldenburg2.str.redhat.com> you wrote: > > > =20 > > > > Since I'm opposed to this entire project, I have largely > > > > refrained from reviews, except for things that looked obviously > > > > wrong to me (e.g., things that definitely break compatibility > > > > with older kernels or existing ABIs), but even for those cases, > > > > my feedback probably wasn't very helpful. =20 > > > > > > I can understand your point of view, but indeed this is not > > > helpful here. glibc is a central resource for the whole FOSS > > > system, and we are not pushing these changes for a fancy, but > > > because a solution is needed for a large number of affected > > > projects and products. > > > > > > We need to find a constructive way to proceed with this matter. > > > It will not go away if we try to ignore it. =20 > >=20 > > My impression is that we have two mostly separate groups > > of users for 32-bit user space: > >=20 > > a) Those that absolutely require 100% ABI compatibility to run > > legacy code but will eventually migrate to 64-bit user space, > > almost certainly within the coming ten years before we really > > need to worry (too much) about things like key expiration dates > > beyond 2038. > >=20 > > b) Those that already need support for 64-bit time_t because > > they are deploying new 32-bit binaries that are expected to run > > beyond 2038, while not caring at all about compatibility > > with existing binaries that are already known to be broken > > for this purpose. > >=20 > > If we just work to satisfy both these groups but ignore the > > intersection (those that have existing 32-bit binaries and want > > those to run after 2038?), things could become noticeably easier. =20 >=20 > Well it replaces one sticky problem with two separate, individually > easier problems. But I'm not sure it reduces the overall amount of > work needed; it might increase it. >=20 > The above roughly corresponds to what my original idea for musl and > the ".2 ABI" was, and I'm now fairly skeptical of it. Binary distros > are probably not going to want to expend extra effort to satisfy the > needs of both groups, so they'll either EOL the 32-bit ports they have > now (effectively leaving group (a) screwed, not because glibc dropped > them, but because their distro did) to produce something that will > continue to be relevant into the future, or they'll stick with the old > ABI and EOL 32-bit entirely in the near-ish future. >=20 > The segment of group (b) that's building everything themselves > probably doesn't care in the latter case, but then do they even care > about using glibc (vs something else) anyway? If I may add my 2$ - I think they care. Otherwise there wouldn't be the effort to adapt "upstream first" policy with Y2038 support. >=20 > From this analysis it seems like, if you attack the problems > separately, "no action" is the right choice for distros, but that also > leads to EOL for distro support of all existing 32-bit targets. But as it was mentioned in the other mail. There are _real_ users of 32 bit targets, so distros may re-consider dropping them. >=20 > As a result, I'm leaning strongly towards a conclusion that a solution > that doesn't split the two cases is better, and it's probably what > we'll do for musl. The approach would be very similar to what was done with _FILE_OFFSET_BITS=3D=3D64. The end user would need to add -D_TIME_BITS=3D= =3D64 when compiling programs against glibc on 32 bit architecture. >=20 > > The price for that is a mid-2030s EOL time on any existing 32-bit =20 >=20 > That's really wishful thinking. I suspect we'll hit critical failures > much sooner due to need to represent dates in the future. >=20 > > glibc ports, any binaries linked to them, and any distros built upon > > that, and replacing them with something else that is binary > > incompatible but can coexist within e.g. a container. =20 >=20 > I'm not sure how "coexist within a container" is helpful; that might > as well just be a separate system. > 1 Multi-arch type approaches might > work a little bit better, but in order for these to be useful to end > users, there need to be automated ways of setting them up. That's a > big burden for distros to spend on archs that many of them would > rather just EOL/drop... >=20 > > There are several options what such a replacement might look > > like, some that come to mind would include: > >=20 > > - A patchset against glibc that replaces all the time32 interfaces > > with time64 ones and that is maintained outside of the glibc > > code base by whoever is interested in this working. =20 >=20 > If you're trying to use incompatible old and new ABI alongside each > other, I would think you only want to do that as long as you still > have old stuff around. But if you're using patches with ad-hoc ABI > that have bypassed the process of upstream review to ensure that > they're making good decisions compatible with the long term, it's > likely that it's going to keep breaking ABI, and you're going to need > to do multiple painful replacements or keep N ABIs alongside each > other rather than 2 (possibly using something like nix to make it > practical). >=20 > So I don't think "use out-of-tree patches for time64 support" is a > good approach. I do agree here - the "out-of-tree" is not a good approach. Basically one would need to rebase those "hacks" on top of each glibc release. If you are wondering how such set of "out-of-tree" patches ("hacks" is a better word I think) look like - please check following link [1]. They provide support for Y2038 on 32 bit arch (and even passes some tests) [2]. >=20 > > - A set of glibc architecture ports that can coexist with the > > existing ports but have new target triples and/or soname. > > These could be implemented and merged one architecture > > at a time and share most of the code with the existing > > ports. =20 >=20 > This is basically the same proposal as the musl ".2 ABI", and it's one > of the least-bad, but as explained above I no longer really favor it. >=20 > > - Leave glibc unchanged and require users/distros to migrate to > > musl or something else if they need the time64 support. =20 >=20 > As much as I'd like this, I don't think distros that have stability > contracts with their users for how things work can replace the libc > with one which has significantly different behaviors in application- > and user-facing ways. +1 > It's a viable solution for users to run new > 32-bit stuff alongside a 32-bit glibc target that's being phased-out > and nearing EOL, but if this is the only option I think it would just > lead to distros dropping the targets. >=20 > > Any of the above approaches (including the current plan of > > supporting time32 and time64 in a single glibc binary) can of > > coexist, but the more incompatible approaches get implemented, > > the more fragmentation of ABIs we get. =20 >=20 > I'm moderately in favor of supporting both in a single binary. As > noted elsewhere, this fundamentally does not break ABI between libc > consumers (main apps or 3rd party libs) and libc, only between pairs > of libc consumers that have used the affected types as part of their > public interfaces. As I've written above. For the end user it shall be enough to add -D_TIME_BITS=3D=3D64 during compilation. >=20 > To answer Wolfgang Denk's original question, in my opinion, the single > most valuable thing for accelerating the time64 process that third > parties could contribute is detailed analysis of the interfaces that > will break. Such analysis is already done and available on glib'c wiki page [3]. > This (1) will give us an idea of the extent of breakage > that could happen in the worst case, and (2) can be translated > directly into a graph of "requires/conflicts-with" relationships > between library packages that a distro can use to ensure consistency > when updating packages. As stated above - please visit [3]. >=20 > Are others willing to start a discussion of exactly what form this > analysis should take, how results should be reported, and how to use > them to make such a transition go smoothly? There is even the OE/Yocto's meta-y2038 layer (thud YPRR) [4] , which allows building glibc for testing with: 1. Kernel 5.1+ (with 64 bit time syscalls support on 32 bit archs).=20 2. Kernel < 5.1 (to check if it works with older syscalls) 3. A set of tests for syscalls mentioned in [3][2] (with and without Y2038 support in glibc) 4. Several different architectures for testing (qemu): - ARM 32 bit - vexpress-a9 - i386 / x86_64 - x32 - the "special" one - All the OE/Yocto supported qemu* targets (mips, ppc) 5. One can run the glibc xtest with mentioned above qemu running system with test-wrapper=3D'/opt/Y2038/glibc/src/scripts/cross-test-ssh.sh (detailed instruction in [4]). 6. It is also possible to run plain glibc tests for archs mentioned in point 4 above. >=20 > > Note that we have never had such a forced transition on x86, but > > there are several examples on other architectures: > > - arm oabi/gnueabi/gnueabihf > > - ppc64/ppc64le > > - mips o32/n32/64 > > The migration is not much fun for users to say the least, but it's > > quite possible that the alternative is even worse. =20 >=20 > I wouldn't characterize all of those as transitions, more as different > options. As far as I know glibc still supports them all except ARM > OABI, which was dropped long before there was serious interest in > end-user-facing Linux on ARM. >=20 > Rich Note: [1] - https://github.com/lmajewski/y2038_glibc/commits/Y2038-2.29-glibc-11-03-2019 [2] - https://github.com/lmajewski/y2038-tests [3] - https://sourceware.org/glibc/wiki/Y2038ProofnessDesign?highlight=3D%28y2038= %29 [4] - https://github.com/lmajewski/meta-y2038/tree/master Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de --Sig_/YrTS5Qx9kJFxvPX/ERY1/=X Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgAyFJ+N6uu6+XupJAR8vZIA0zr0FAl0vmdwACgkQAR8vZIA0 zr2XqwgA08bIvGNUXY9YbtuxSaImbJpWdmwtdlVdQjDbLtAaNbtB8PiriaidoOUi xwHIdF+XEVuIpqCL0i81zwg29ccGnLU+S0W13xtKe0XP6/sUXR8riNXOTlrwX6sc pDGpAwjr+MwY+PM2anKmN6+H57vYHCOQ1uGg9y7B0uS1ABE3ZtFxlcLpxSwD/Fud HDJ5Rt3joDS28APkEXUCBQmOA0TUsE5aCHM59K+Vm5pAtNRqK0l91fPE7QEqTzwl RIBh0TRWZF+m1wrpsVFrKF8hfYgRxCTV+BTkxtDwm89HwJdJgW5blLZ98pPboP+j +5smAU1dwrTOCULlCmw7ye/69b0VIg== =KRxB -----END PGP SIGNATURE----- --Sig_/YrTS5Qx9kJFxvPX/ERY1/=X--