From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 13BC21F47C for ; Sun, 15 Jan 2023 11:01:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pH0lD-00025F-Ig; Sun, 15 Jan 2023 06:01:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pH0lC-00024y-5I for bug-gnulib@gnu.org; Sun, 15 Jan 2023 06:01:10 -0500 Received: from uggla.sjd.se ([2001:9b1:8633::107]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pH0l7-00044z-GA for bug-gnulib@gnu.org; Sun, 15 Jan 2023 06:01:09 -0500 DKIM-Signature: v=1; a=ed25519-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=ed2110; h=Content-Type:MIME-Version:Message-ID:In-Reply-To :Date:References:Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding :Content-ID:Content-Description; bh=Nz7yixa/Hfz9DYzFLwLMdqan0mmKiwCK15xUVZlAOs0=; t=1673780462; x=1674990062; b=Zk4JewBzkYPwS6563SGVT0N3/bn+JFlrH9q49KKECzy/7nxemFLFEHEcMhbt7qAqHxi93vqh//S 7VJizWYxzBg==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=rsa2110; h=Content-Type:MIME-Version:Message-ID: In-Reply-To:Date:References:Subject:To:From:Sender:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Nz7yixa/Hfz9DYzFLwLMdqan0mmKiwCK15xUVZlAOs0=; t=1673780462; x=1674990062; b=e3degfLdN3A1wPu9nbXrOjZu3m4j0yvTFVlGJ3PV2DfE/VL2N4tttcfqg2fkgqZqt9/VDMndLGb NPYIGrqq3ETQfQCLgWRyGWRbFntxTCwEkafZUhO2M3L8K/xQQf18cSkXuVOI15I4hMHEmo26rbU9y //y20eWUtmC3xJ+7Jx+fESI5mCtZmdI+YI4j8HMnJScdg0yo4XH6B8/TdLKmLseLw1Iz386N5GK5L Vla0igmUEth1P9CnBLBbmo7GusWazAtcAvMYEiWIuzJV9Cxg0FOk5x1vxBNrnAdvjLAMviQIfzPLJ Mu84TTrJoarNcr8Ne25mp5S1z44cLxWsFcCr/PnwQomsxIjS4aM+x3A3JboaEDKXF+VOkTtiI+dhe xzkfIUOoQVY0bUmFDKoVyVBNrbTuW1UZe4gPtKG7NGMwPEhu8rIGqj0rhrPWHUy+39C6dRsuF; Received: from [2001:9b1:41ac:ff00:4177:6622:48f5:9b4a] (port=55038 helo=kaka) by uggla.sjd.se with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1pH0l1-002mXB-0g for bug-gnulib@gnu.org; Sun, 15 Jan 2023 12:00:59 +0100 X-Hashcash: 1:22:230115:bug-gnulib@gnu.org::IJorx7WROf4Zs/Gz:1wMH To: bug-gnulib@gnu.org Subject: RFC: git-commit based mtime-reproducible tarballs References: <87h6wtgmhy.fsf__22556.7857896507$1673713908$gmane$org@redhat.com> OpenPGP: id=B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE; url=https://josefsson.org/key-20190320.txt X-Hashcash: 1:22:230115:dje.gcc@gmail.com::TbW93Y1Bzai0+iT3:8TLB X-Hashcash: 1:22:230115:binutils@sourceware.org::wTIvMkns8t3TqcMk:5HlT X-Hashcash: 1:22:230115:info-gnu@gnu.org::sF6cTF9q5MXp0UFZ:VWeu X-Hashcash: 1:22:230115:nickc@redhat.com::DJSLZ7yhq/ULV7yM:09C6X Date: Sun, 15 Jan 2023 12:01:00 +0100 In-Reply-To: <87h6wtgmhy.fsf__22556.7857896507$1673713908$gmane$org@redhat.com> (Nick Clifton's message of "Sat, 14 Jan 2023 15:32:41 +0000") Message-ID: <87lem4cb9v.fsf@josefsson.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=2001:9b1:8633::107; envelope-from=simon@josefsson.org; helo=uggla.sjd.se X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Simon Josefsson From: Simon Josefsson via Gnulib discussion list Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi. Quoting the recent binutils announcement: > As an experiment these tarballs were made with the new "-r " > option supported by the src-release.sh script. This attempts to make > reproducible tarballs by sorting the files and passing the > "--mtime=3D" option to tar. The date used for these tarballs was > obtained by running: >=20=20=20 > git log -1 --format=3D%cd --date=3Dformat:%F bfd/version.m4 This got me thinking about git-version-gen and GNUmakefile, and I came up with the patch below to use the most recent commit as the timestamp for all files in the tarball. What do you think? There are some concerns about this: 1) Having the same mtime on all files in a tarball may cause problems for some projects that have fragile dependency-systems. While I think all dependency checks really should be using >=3D timestamp tests, I wouldn't rule out that some use > timestamp tests, which would cause (sometimes unwanted) rebuilding of some files. Are there dependency-constructs where the same mtime for all files in a tarball is just a bad idea, with no better approach available? 2) The use of TAR_OPTIONS in GNUmakefile is complex and somewhat hard to debug. I can't find any cleaner way to provide options to tar for 'make dist' though. Automake defines $(AMTAR) but looks like an internal symbol which also isn't used (bug?), instead $(am__tar) is used and defined as am__tar =3D $${TAR-tar} chof - "$$tardir". So we can override TAR in Makefile.am but it looks like a user-variable that we shouldn't override. So pending support for a AMTAR (or AM_TAR?) variable in Makefile.am that actually works, I guess we are stuck with the TAR_OPTIONS approach. We could do 'TAR =3D env TAR_OPTIONS_=3D... tar' in Makefile.am but it looks like the wrong approach. 3) The Makefile.am snippet in git-version-gen is difficult to maintain, can't we put such snippets in a gnulib-owned file and suggest use of 'include gl/top-gl-Makefile.am-include.mk' instead? The same applies to gen-ChangeLog rule. The logic would have to be a bit more complex to support per-project modifications to these rules though. Two small bugs that are possible to fix but not important before we know if mtime-reproducible tarballs is useful or not: 4) If there is no .version file when you type 'make dist' my patch below would fail to provide --mtime=3D... to tar. So it fails if you didn't do 'make' before 'make dist' after ./bootstrap + ./configure in a clean checkout. 5) It is also a bit fragile that it assume 'git log -1' works without checking for errors before invoking touch. /Simon diff --git a/build-aux/git-version-gen b/build-aux/git-version-gen index a72057bf2c..0a98cb12dd 100755 =2D-- a/build-aux/git-version-gen +++ b/build-aux/git-version-gen @@ -66,6 +66,7 @@ scriptversion=3D2022-07-09.08; # UTC # BUILT_SOURCES =3D $(top_srcdir)/.version # $(top_srcdir)/.version: # echo '$(VERSION)' > $@-t +# touch -m -d @$(shell git log -1 --format=3D%cd --date=3Dunix) $@-t # mv $@-t $@ # dist-hook: # echo '$(VERSION)' > $(distdir)/.tarball-version diff --git a/top/GNUmakefile b/top/GNUmakefile index 07b331fe53..f0dd41b5b4 100644 =2D-- a/top/GNUmakefile +++ b/top/GNUmakefile @@ -25,8 +25,14 @@ _gl-Makefile :=3D $(wildcard [M]akefile) ifneq ($(_gl-Makefile),) =20 +_gl-.version :=3D $(wildcard .version) +ifneq ($(_gl-.version),) +_tar_mtime :=3D --mtime=3D.version +endif + # Make tar archive easier to reproduce. =2Dexport TAR_OPTIONS =3D --owner=3D0 --group=3D0 --numeric-owner --sort=3D= name +export TAR_OPTIONS =3D --owner=3D0 --group=3D0 --numeric-owner --sort=3Dna= me \ + $(_tar_mtime) =20 # Allow the user to add to this in the Makefile. ALL_RECURSIVE_TARGETS =3D --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCY8Pc7BQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFokZmAQC4o+8oTRVS4woPVHTlVNzxRjJ1KM0M k1BAfPGHMsWpAAEAwccwFcjtElUybNWz3Ub1QTI6ZHqjRlvFj79J4DGJXwc= =ALbe -----END PGP SIGNATURE----- --=-=-=--