gnulib-tool.py: Optimize directory creation.

gnulib-tool.py: Optimize directory creation.

[Collin Funk]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

Hi Bruno,

Separating this from the other thread so it is easier to follow.

This patch handles the directory creation upfront instead of checking
for every single file, as you mentioned earlier [1].

The previously unused variable was slightly incorrect. We have to join
the directory name with self.testdir. I ran it without this and the
directories were made in the current working directory, causing the
tests to crash.

[1] https://lists.gnu.org/archive/html/bug-gnulib/2024-04/msg00245.html

Collin

[-- Attachment #2: 0001-gnulib-tool.py-Optimize-directory-creation.patch --]
[-- Type: text/x-patch, Size: 2181 bytes --]

From 709ff282c0093f696594810298bdc21c251489cd Mon Sep 17 00:00:00 2001
From: Collin Funk <collin.funk1@gmail.com>
Date: Mon, 15 Apr 2024 09:16:25 -0700
Subject: [PATCH] gnulib-tool.py: Optimize directory creation.

* pygnulib/GLTestDir.py (GLTestDir.execute): Use a list of possible
subdirectories and create them upfront instead of checking every file.
---
 ChangeLog             |  6 ++++++
 pygnulib/GLTestDir.py | 10 +++++-----
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 5b7b3a36fc..cff3b1c048 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2024-04-15  Collin Funk  <collin.funk1@gmail.com>
+
+	gnulib-tool.py: Optimize directory creation.
+	* pygnulib/GLTestDir.py (GLTestDir.execute): Use a list of possible
+	subdirectories and create them upfront instead of checking every file.
+
 2024-04-14  Collin Funk  <collin.funk1@gmail.com>
 
 	gnulib-tool.py: Fix incorrect type hint.
diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index a7709a1259..b668629f4d 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -345,9 +345,12 @@ def execute(self) -> None:
         filelist = sorted(set(filelist))
 
         # Create directories.
-        directories = [os.path.dirname(file)
-                       for file in self.rewrite_files(filelist)]
+        directories = [ joinpath(self.testdir, os.path.dirname(file))
+                        for file in self.rewrite_files(filelist) ]
         directories = sorted(set(directories))
+        for directory in directories:
+            if not isdir(directory):
+                os.makedirs(directory)
 
         # Copy files or make symbolic links or hard links.
         filetable = []
@@ -358,9 +361,6 @@ def execute(self) -> None:
             src = row[1]
             dest = row[0]
             destpath = joinpath(self.testdir, dest)
-            dirname = os.path.dirname(destpath)
-            if not isdir(dirname):
-                os.makedirs(dirname)
             if src.startswith('tests=lib/'):
                 src = constants.substart('tests=lib/', 'lib/', src)
             lookedup, flag = self.filesystem.lookup(src)
-- 
2.44.0

Re: gnulib-tool.py: Optimize directory creation.

[Bruno Haible]

Hi Collin,

> This patch handles the directory creation upfront instead of checking
> for every single file, as you mentioned earlier [1].

Thanks! This looks good. Applied.

Bruno