From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.7 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 786421F47C for ; Sun, 15 Jan 2023 22:26:21 +0000 (UTC) Authentication-Results: dcvr.yhbt.net; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=clisp.org header.i=@clisp.org header.a=rsa-sha256 header.s=strato-dkim-0002 header.b=Ip/GOnm9; dkim-atps=neutral Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pHBS4-0004nb-Ns; Sun, 15 Jan 2023 17:26:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHBS3-0004nS-Pp for bug-gnulib@gnu.org; Sun, 15 Jan 2023 17:26:07 -0500 Received: from mo4-p01-ob.smtp.rzone.de ([85.215.255.50]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHBRy-0002zN-KJ for bug-gnulib@gnu.org; Sun, 15 Jan 2023 17:26:07 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1673821558; cv=none; d=strato.com; s=strato-dkim-0002; b=ZEiIbJ/TFPeBBCDRzX++DZWAtczULlZei7TGpVd2CZ3/8VBWityq80oZGxHiGRHpq/ zQNDc3GyqxAQ1jHfIYZuEwR6fJFRy1YlXjbukexmZZL/FSNuTtDtvaA5meoMs8EO/m8b W0O+4LTg0BYxeT5slk7UjxB4kvho03x3x8GM9vG/+RNBn3BOjgI70wr05FrcgS3uSp8N 4keDCZxxxapg4S8JG5HH6Civsfj1sRqreMZTePdquWPLpUyRbC7pfZtD3hbsq4xQY5we Hde1XfTpLhDinw7SNjiS8uhZEic9AYUWh3/02qmO+LCOxl3ojdtL1nSax0iED85jpa8I G6Jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1673821558; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2ZdkknxoAck3RKFNWZILVCwzmWVx7KmbGc9yDe1k0TE=; b=OE4GqL/ETlSW63pjhDHrMtiV8uttYJ+S0yHc82ThKhA9vS/UHnyx5vr2ZTUn+DksSc g9eGjLAYlfkdsYQlE/6iTkkO2TQ+Scut6aJ8GgIA7ZAFO6GePIvLc7owLYHLUZJVS2yr zNfOE7vvW/WXU7X8N7jZVBf01XaVANiDL1LVAazK0SPeftool2cQja4mwIRYqQP0sWaN SfQfylEmvGFwZWPoT8GzTfMyy5UvLg8P4elSv7BwK09/bgrhlq+F7nZUmLA5YQJoffiM pFr3ANQOpApAib6aIwLQLjVUjJx4ATN9uWmF19PW03a/y5YHYIGwVC1tWB07AiFgl20s Dyvw== ARC-Authentication-Results: i=1; strato.com; arc=none; dkim=none X-RZG-CLASS-ID: mo01 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1673821558; s=strato-dkim-0002; d=clisp.org; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2ZdkknxoAck3RKFNWZILVCwzmWVx7KmbGc9yDe1k0TE=; b=Ip/GOnm9gjn/vM1ff/xUzxqqsU5SMau4pSsMahNS8wbYVl2va1o+964QMmInDxFYoe tMjJTGiETm9qDO+VLIx2you2IEBhKAX8pWpoN3MUz8NyrXMdrlj/WjM+sml3EJxbupdD ahuhR6aWpRgGwOCdevT8oJlKzQHW9Pq7i/htSBccueV/tS7jD8VdIA11us1ZCMKmlUht ghPaMe3sL9X6pSnYABpAFec2VYnUKeMiX05eWCRnrjHKpKEMmOL92bJk7+tbxZk2il0N CW/FXyNWsLhuvxQrtYA+Mjcf9JYGS0GCtE5bLl4Yck5ogyoV3iNaV6BqEUTRNO4nCVT6 q/qw== X-RZG-AUTH: ":Ln4Re0+Ic/6oZXR1YgKryK8brlshOcZlIWs+iCP5vnk6shH0WWb0LN8XZoH94zq68+3cfpPC36MKRCY/l9/dcAUOVs3n6M4G" Received: from nimes.localnet by smtp.strato.de (RZmta 48.6.2 AUTH) with ESMTPSA id I8f358z0FMPwIlV (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Sun, 15 Jan 2023 23:25:58 +0100 (CET) From: Bruno Haible To: Paul Eggert Cc: Simon Josefsson , bug-gnulib@gnu.org Subject: Re: RFC: git-commit based mtime-reproducible tarballs Date: Sun, 15 Jan 2023 23:25:58 +0100 Message-ID: <2740098.11c6FMkHaZ@nimes> In-Reply-To: References: <87h6wtgmhy.fsf__22556.7857896507$1673713908$gmane$org@redhat.com> <5459006.YCjZZlMYnJ@nimes> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Received-SPF: none client-ip=85.215.255.50; envelope-from=bruno@clisp.org; helo=mo4-p01-ob.smtp.rzone.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Paul Eggert wrote: > some users want to "trust but verify" and a reproducible > tarball is easier to audit than a non-reproducible one, so for these > users it can be a win to omit the irrelevant data from the tarball. Reproducibility can be implemented in different ways: - by omitting irrelevant data from the tarball, - by having a customized comparison program 'diff', such that "diff --ignore-irrelevant-metadata contents1 contents2" would ignore the irrelevant parts. > when I do an 'ls > -l' of a source directory that I got from a distribution tarball, it's > useful to see the last time the contents of each source file was changed > upstream. OK, now we're discussing different ways to make a tarball reproducible. That's nice, because Simon's proposal was to make all timestamps equal, and that puts me off. In binutils-2.40.tar.bz2 all files are from 2023-01-14. In android-studio-2021.3.1.17-linux.tar.gz all files are from 2010-01-01. It gives me as a user no idea whether this tarball is 13 years old, 2 years old, or from yesterday. I much prefer Paul's approach, since it still conveys meaningful timestamps: > For TZDB, where users have long wanted reproducibility, I use something > like this in a Makefile recipe for each source file $$file: > > time=`git log -1 --format='tformat:%ct' $$file` && > touch -cmd @$$time $$file That's good for the files that are under version control. > 2. What about platform-independent files that are automatically created > from source files from the repository, and that are shipped in the > release tarball? For these, you could unpack the tarball, see in which order the timestamps are, and then assign artificial timestamps, in the same order but exactly 2 seconds apart. For example, if the tarball contains under version control: hello.c 2023-01-14 13:28:14 configure.ac 2023-01-01 14:03:07 and not under version control: configure 2023-01-15 04:09:10 config.h.in 2023-01-15 04:05:19 then you would determine the max_timestamp_under_vc = max { 2023-01-14 13:28:14, 2023-01-01 14:03:07 } = 2023-01-14 13:28:14 and then, since config.h.in is older than configure: touch -m (max_timestamp_under_vc + 2 seconds) config.h.in touch -m (max_timestamp_under_vc + 4 seconds) configure You can do this without knowing the Makefile rules or scripts which created config.h.in and configure. The increment of 2 seconds is, of course, for VFAT file systems, which have only 2 seconds of resolution for file modification times. > GNUTARFLAGS uses delete=atime,delete=ctime so that atime and > ctime do not leak into the tarball and make it less reproducible I agree, it's pointless to have atime and ctime in a tarball. Bruno