From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 8E0751F47C for ; Mon, 16 Jan 2023 08:29:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pHKrL-00049l-Da; Mon, 16 Jan 2023 03:28:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHKrF-00047o-HF for bug-gnulib@gnu.org; Mon, 16 Jan 2023 03:28:46 -0500 Received: from uggla.sjd.se ([2001:9b1:8633::107]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHKrB-0001Ct-WE for bug-gnulib@gnu.org; Mon, 16 Jan 2023 03:28:45 -0500 DKIM-Signature: v=1; a=ed25519-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=ed2110; h=Content-Type:MIME-Version:Message-ID:In-Reply-To :Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding :Content-ID:Content-Description; bh=+gz6GpusWn+os36iZa4MWvuFle0dst3Mygk+xBpFFA8=; t=1673857721; x=1675067321; b=RwwXxQrnIi3RiSTsx1n19WdjaVphIAAPF0g/wF7AGs5NBzrPDCJ5327T0+NRSdeSZ5+XmKhSZj+ 6t5w87n3QCw==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=josefsson.org; s=rsa2110; h=Content-Type:MIME-Version:Message-ID: In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=+gz6GpusWn+os36iZa4MWvuFle0dst3Mygk+xBpFFA8=; t=1673857721; x=1675067321; b=BagydwuVx9M88NvLjPMQ6HYKhZyx6QRHDIRPxndWa1n5bRFBljWVssAXX4yYYYBLTaUOtphX1Yt C5B2izrh9DaY1WNsgcUwos4caMVP30rBpheqiCtV8OQhA9Q93cTHVBNBfRpO86ezwbuTbroahE2SD iCRJcFG87w9B5DBPHIcAvDq7KtAYfWpOw/ivEJv/aSm1Q4xq23rNvbXDtb+pFwXiPPm4+fH6HpsSr NVA1TkjSe42KG/771iEVuO8laK1oyf5CZK09rTiwHJ13wrzHXsRzA0wo1vdT0+ptuDK3WCtv+WJYC WIR/xsjU1KrM6eWnIgDSmFrHMKbYxRNY+eFekhU5U22zLTx/C8UgZTw7I4QhkgEcYuCMtazB3Fs57 JeyT0oclfXWgdXSCdrRJzqWm6YIcq7iG+QdFQyyjpzAqug4anaYgG/Z0KHdOfZKF+uvuQriXt; Received: from [2001:9b1:41ac:ff00:2422:ac13:8078:4a4e] (port=39264 helo=kaka) by uggla.sjd.se with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1pHKr9-004HNK-3g; Mon, 16 Jan 2023 09:28:39 +0100 To: Bruno Haible Cc: bug-gnulib@gnu.org Subject: Re: RFC: git-commit based mtime-reproducible tarballs References: <87h6wtgmhy.fsf__22556.7857896507$1673713908$gmane$org@redhat.com> <87lem4cb9v.fsf@josefsson.org> <5459006.YCjZZlMYnJ@nimes> OpenPGP: id=B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE; url=https://josefsson.org/key-20190320.txt X-Hashcash: 1:22:230116:bruno@clisp.org::aP/tUcLd4JzgzQON:EvWj X-Hashcash: 1:22:230116:bug-gnulib@gnu.org::HUSqH9Sb3JGgAcO6:Phtr Date: Mon, 16 Jan 2023 09:28:45 +0100 In-Reply-To: <5459006.YCjZZlMYnJ@nimes> (Bruno Haible's message of "Sun, 15 Jan 2023 14:21:01 +0100") Message-ID: <87cz7edgsi.fsf@josefsson.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=2001:9b1:8633::107; envelope-from=simon@josefsson.org; helo=uggla.sjd.se X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Simon Josefsson From: Simon Josefsson via Gnulib discussion list Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Bruno, > Hi Simon, > >> > This attempts to make >> > reproducible tarballs by sorting the files and passing the >> > "--mtime=3D" option to tar. ... >> Having the same mtime on all files in a tarball > > First question: What is the point of doing that? Good question, I don't know the motivation for the binutils people. For me, the motivation would be to get rid of arbitrary/random differences in non-source artifacts. Those makes auditing non-source for reproducibility more difficult and error prone, and even a source for side-channels. I think the exact motivations are still not fully understood and that this is an evolving space -- articulating the goals is useful to measure if we will actually meet them. To me this is similar to including a build timestamp in a binary. In theory it would not cause any problems for anyone, but in practice it will be one more source of differences that may hide or complicate finding other more important differences. Thus from a helicopter perspective it does make sense to fix that particular non-reproducible behaviour even though it is difficult to argue that the timestamp by itself is a serious bug that is important to fix. > Reproducibility is about verifying that an artifact A was generated > from a source S. Right, and I think the proponents of reproducability suggest that an even stronger verification should be possible: that there is a one-to-one correspondence between source S and artifact A [for a particular environment where A is relevant]. If different artifacts A can be generated from the same source S this will be a source of unreproducability and non-deterministic behaviour, which ultimately can be a security/safety/reliability problem. > When I, as a GNU maintainer or uploader, create a tarball and upload it > to ftp.gnu.org, that tarball is the source S. Because that's what I sign > with my GPG key. The commits in the git repo aren't the source, and even > the git checkout on my disk aren't the source =E2=80=94 because I am free= to > unpack and repack the tarball as I like, before I upload it to ftp.gnu.or= g. Yeah, and I think this is what is being challenged recently -- some people don't consider tarballs the only relevant source code any more. To me this makes some sense: we all have tried to fix a small bug in a package by making changes to some source code, and then see the build fail catastrophically and sometimes in ways that can't even be resolved because the necessary tools or source codes were forgotten from the released tarball. I think it is good practice to verify that our tarballs can be regenerated reproducibly from version controlled sources and free tools. >> 1) Having the same mtime on all files in a tarball may cause problems > > Definitely. HP-UX 'make' attempts to rebuilds a file Y that depends on > a file X, if Y and X have the same timestamp (mtime). It is long known > that you have to have actually different timestamps for some files. Interesting -- I wonder if supporting HP-UX [without GNU make] is worth more than the benefits from reproducible tarballs. /Simon --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCY8UKvRQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFogG0AP9Ijhuwc0MW2X/2rVvCDFNSxsabEsGl uf3ReFa8Ce3YggEA1iemQRmEVKV7NDOKqDIKy9fBS0sVSq34S+drhL/DGA4= =FzVy -----END PGP SIGNATURE----- --=-=-=--