Re: GNU gnulib: calling for beta-testers

[Bruno Haible <bruno@clisp.org>]

[-- Attachment #1: Type: text/plain, Size: 1813 bytes --]

Thanks for the report, Paul.
Thanks for the preliminary investigation, Collin.

> >   ./bootstrap
> >   ./configure
> >   make -k distclean
> >   git submodule foreach git pull origin master
> >   git commit -m 'build: update gnulib submodule to latest' gnulib
> >   ./bootstrap --no-git --gnulib-srcdir=gnulib
> > 
> > The problem is that the Python-based build leaves behind a __pycache__ directory, which causes the comparison to fail.

I reproduce the issue. It's because executing gnulib-tool.py creates
gnulib/pygnulib/__pycache__, while gnulib-tool.sh does not do so.

Two workarounds are possible. I'm committing both, since the first
workaround works only with Python ≥ 3.8.
  * Let Python create its cache not in gnulib/pygnulib/__pycache__,
    but instead in
    /tmp/gnulib-python-cache-$USER/<absolute_file_name>/gnulib/pygnulib/ .
  * Ignore the __pycache__ directory during the comparison.

The first workaround should fix trouble similar to what we regularly
see with 'autom4te.cache': Unnecessary difference while comparing source
trees, unnecessary "git status" noise. Clutter.


2024-04-22  Bruno Haible  <bruno@clisp.org>

	gnulib-tool: Fix trouble caused by Python's bytecode cache.
	Reported by Paul Eggert in
	<https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00367.html>.
	* gnulib-tool: In sh+py mode, ignore the __pycache__ directory during
	comparison.

2024-04-22  Bruno Haible  <bruno@clisp.org>

	gnulib-tool.py: Fix trouble caused by Python's bytecode cache.
	Reported by Paul Eggert in
	<https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00367.html>.
	* gnulib-tool.py: Set PYTHONPYCACHEPREFIX, so as to avoid creating a
	__pycache__ directory in the developer's gnulib checkout (only effective
	with Python ≥ 3.8).


[-- Attachment #2: 0001-gnulib-tool.py-Fix-trouble-caused-by-Python-s-byteco.patch --]
[-- Type: text/x-patch, Size: 1951 bytes --]

From eda62139d838f53e4953db26019e5a4b8b805847 Mon Sep 17 00:00:00 2001
From: Bruno Haible <bruno@clisp.org>
Date: Mon, 22 Apr 2024 13:11:05 +0200
Subject: [PATCH 1/2] gnulib-tool.py: Fix trouble caused by Python's bytecode
 cache.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Reported by Paul Eggert in
<https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00367.html>.

* gnulib-tool.py: Set PYTHONPYCACHEPREFIX, so as to avoid creating a
__pycache__ directory in the developer's gnulib checkout (only effective
with Python ≥ 3.8).
---
 ChangeLog      | 9 +++++++++
 gnulib-tool.py | 6 ++++++
 2 files changed, 15 insertions(+)

diff --git a/ChangeLog b/ChangeLog
index b3cef64936..4a272d326e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,12 @@
+2024-04-22  Bruno Haible  <bruno@clisp.org>
+
+	gnulib-tool.py: Fix trouble caused by Python's bytecode cache.
+	Reported by Paul Eggert in
+	<https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00367.html>.
+	* gnulib-tool.py: Set PYTHONPYCACHEPREFIX, so as to avoid creating a
+	__pycache__ directory in the developer's gnulib checkout (only effective
+	with Python ≥ 3.8).
+
 2024-04-21  Collin Funk  <collin.funk1@gmail.com>
 
 	gnulib-tool.py: Make temporary directories recognizable.
diff --git a/gnulib-tool.py b/gnulib-tool.py
index cdcd316909..81537c272c 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -144,6 +144,12 @@
   func_fatal_error "python3 not found; try setting GNULIB_TOOL_IMPL=sh"
 fi
 
+# Tell Python to store the compiled bytecode outside the gnulib directory.
+if test -z "$PYTHONPYCACHEPREFIX"; then
+  PYTHONPYCACHEPREFIX="${TMPDIR-/tmp}/gnulib-python-cache-${USER-$LOGNAME}"
+  export PYTHONPYCACHEPREFIX
+fi
+
 profiler_args=
 # For profiling, cf. <https://docs.python.org/3/library/profile.html>.
 #profiler_args="-m cProfile -s tottime"
-- 
2.34.1


[-- Attachment #3: 0002-gnulib-tool-Fix-trouble-caused-by-Python-s-bytecode-.patch --]
[-- Type: text/x-patch, Size: 1609 bytes --]

From ab5390ae6d8db323420874d1c1334feb77af9cb1 Mon Sep 17 00:00:00 2001
From: Bruno Haible <bruno@clisp.org>
Date: Mon, 22 Apr 2024 13:12:35 +0200
Subject: [PATCH 2/2] gnulib-tool: Fix trouble caused by Python's bytecode
 cache.

Reported by Paul Eggert in
<https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00367.html>.

* gnulib-tool: In sh+py mode, ignore the __pycache__ directory during
comparison.
---
 ChangeLog   | 8 ++++++++
 gnulib-tool | 2 +-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/ChangeLog b/ChangeLog
index 4a272d326e..462823888d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2024-04-22  Bruno Haible  <bruno@clisp.org>
+
+	gnulib-tool: Fix trouble caused by Python's bytecode cache.
+	Reported by Paul Eggert in
+	<https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00367.html>.
+	* gnulib-tool: In sh+py mode, ignore the __pycache__ directory during
+	comparison.
+
 2024-04-22  Bruno Haible  <bruno@clisp.org>
 
 	gnulib-tool.py: Fix trouble caused by Python's bytecode cache.
diff --git a/gnulib-tool b/gnulib-tool
index 6d430e56e6..85b62883c6 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -199,7 +199,7 @@ case "$GNULIB_TOOL_IMPL" in
         else
           diff_options=
         fi
-        diff -r $diff_options -q . "$tmp" >/dev/null ||
+        diff -r $diff_options --exclude=__pycache__ -q . "$tmp" >/dev/null ||
           func_fatal_error "gnulib-tool.py produced different files than gnulib-tool.sh! Compare `pwd` and $tmp."
         # Compare the two outputs.
         diff -q "$tmp-sh-out" "$tmp-py-out" >/dev/null ||
-- 
2.34.1