bug-gnulib@gnu.org mirror (unofficial)
 help / color / mirror / Atom feed
From: Simon Josefsson via Gnulib discussion list <bug-gnulib@gnu.org>
To: bug-gnulib@gnu.org
Subject: RFC: git-commit based mtime-reproducible tarballs
Date: Sun, 15 Jan 2023 12:01:00 +0100	[thread overview]
Message-ID: <87lem4cb9v.fsf@josefsson.org> (raw)
In-Reply-To: <87h6wtgmhy.fsf__22556.7857896507$1673713908$gmane$org@redhat.com> (Nick Clifton's message of "Sat, 14 Jan 2023 15:32:41 +0000")

[-- Attachment #1: Type: text/plain, Size: 3772 bytes --]

Hi.  Quoting the recent binutils announcement:

>   As an experiment these tarballs were made with the new "-r <date>"
>   option supported by the src-release.sh script.  This attempts to make
>   reproducible tarballs by sorting the files and passing the
>   "--mtime=<date>" option to tar.  The date used for these tarballs was
>   obtained by running:
>   
>     git log -1 --format=%cd --date=format:%F bfd/version.m4

This got me thinking about git-version-gen and GNUmakefile, and I came
up with the patch below to use the most recent commit as the timestamp
for all files in the tarball.  What do you think?

There are some concerns about this:

1) Having the same mtime on all files in a tarball may cause problems
for some projects that have fragile dependency-systems.  While I think
all dependency checks really should be using >= timestamp tests, I
wouldn't rule out that some use > timestamp tests, which would cause
(sometimes unwanted) rebuilding of some files.  Are there
dependency-constructs where the same mtime for all files in a tarball is
just a bad idea, with no better approach available?

2) The use of TAR_OPTIONS in GNUmakefile is complex and somewhat hard to
debug.  I can't find any cleaner way to provide options to tar for 'make
dist' though.  Automake defines $(AMTAR) but looks like an internal
symbol which also isn't used (bug?), instead $(am__tar) is used and
defined as am__tar = $${TAR-tar} chof - "$$tardir".  So we can override
TAR in Makefile.am but it looks like a user-variable that we shouldn't
override.  So pending support for a AMTAR (or AM_TAR?) variable in
Makefile.am that actually works, I guess we are stuck with the
TAR_OPTIONS approach.  We could do 'TAR = env TAR_OPTIONS_=... tar' in
Makefile.am but it looks like the wrong approach.

3) The Makefile.am snippet in git-version-gen is difficult to maintain,
can't we put such snippets in a gnulib-owned file and suggest use of
'include gl/top-gl-Makefile.am-include.mk' instead?  The same applies to
gen-ChangeLog rule.  The logic would have to be a bit more complex to
support per-project modifications to these rules though.

Two small bugs that are possible to fix but not important before we know
if mtime-reproducible tarballs is useful or not:

4) If there is no .version file when you type 'make dist' my patch below
would fail to provide --mtime=... to tar.  So it fails if you didn't do
'make' before 'make dist' after ./bootstrap + ./configure in a clean
checkout.

5) It is also a bit fragile that it assume 'git log -1' works without
checking for errors before invoking touch.

/Simon

diff --git a/build-aux/git-version-gen b/build-aux/git-version-gen
index a72057bf2c..0a98cb12dd 100755
--- a/build-aux/git-version-gen
+++ b/build-aux/git-version-gen
@@ -66,6 +66,7 @@ scriptversion=2022-07-09.08; # UTC
 # BUILT_SOURCES = $(top_srcdir)/.version
 # $(top_srcdir)/.version:
 #      echo '$(VERSION)' > $@-t
+#      touch -m -d @$(shell git log -1 --format=%cd --date=unix) $@-t
 #      mv $@-t $@
 # dist-hook:
 #      echo '$(VERSION)' > $(distdir)/.tarball-version
diff --git a/top/GNUmakefile b/top/GNUmakefile
index 07b331fe53..f0dd41b5b4 100644
--- a/top/GNUmakefile
+++ b/top/GNUmakefile
@@ -25,8 +25,14 @@
 _gl-Makefile := $(wildcard [M]akefile)
 ifneq ($(_gl-Makefile),)
 
+_gl-.version := $(wildcard .version)
+ifneq ($(_gl-.version),)
+_tar_mtime := --mtime=.version
+endif
+
 # Make tar archive easier to reproduce.
-export TAR_OPTIONS = --owner=0 --group=0 --numeric-owner --sort=name
+export TAR_OPTIONS = --owner=0 --group=0 --numeric-owner --sort=name \
+       $(_tar_mtime)
 
 # Allow the user to add to this in the Makefile.
 ALL_RECURSIVE_TARGETS =

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

       reply	other threads:[~2023-01-15 11:01 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <87h6wtgmhy.fsf__22556.7857896507$1673713908$gmane$org@redhat.com>
2023-01-15 11:01 ` Simon Josefsson via Gnulib discussion list [this message]
2023-01-15 13:21   ` RFC: git-commit based mtime-reproducible tarballs Bruno Haible
2023-01-15 16:03     ` Paul Eggert
2023-01-15 22:25       ` Bruno Haible
2023-01-16  8:40         ` Simon Josefsson via Gnulib discussion list
2023-01-16  8:51           ` Jim Meyering
2023-01-16  9:45       ` Vivien Kraus
2023-01-16 11:48         ` Bruno Haible
2023-01-16 23:00         ` Simon Josefsson via Gnulib discussion list
2023-01-16  8:28     ` Simon Josefsson via Gnulib discussion list

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.gnu.org/mailman/listinfo/bug-gnulib

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87lem4cb9v.fsf@josefsson.org \
    --to=bug-gnulib@gnu.org \
    --cc=simon@josefsson.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).