git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"Derrick Stolee" <derrickstolee@github.com>,
	"Adam Spiers" <git@adamspiers.org>, "Jeff King" <peff@peff.net>,
	"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH 07/13] test-lib: create test data with "git init --no-template" (almost)
Date: Sun, 12 Dec 2021 21:13:17 +0100	[thread overview]
Message-ID: <patch-07.13-9402fb23b40-20211212T201308Z-avarab@gmail.com> (raw)
In-Reply-To: <cover-00.13-00000000000-20211212T201308Z-avarab@gmail.com>

Change "t/test-lib.sh" to set the newly added
"GIT_NO_TEMPLATE_DIR=true" option in the environment, which makes any
"git init" invocation take the equivalent of a "--no-template" option,
unless explicitly overridden by config or CLI option.

Well, "almost". We're also adding a test-only
"GIT_TEST_BARE_TEMPLATE=true", when it's set we'll interpret
"GIT_NO_TEMPLATE_DIR=true" as meaning no template, except for empty
"hooks" and "branches" directories.

This is because we've added various implicit dependencies on having
these created by "git init". In subsequent commits we'll address some
of those. More on that below.

The code being added here in "builtin/init-db.c" doesn't require
careful review, since this lazy_mkdir_strbuf_or_die_setlen() function
and the "no_template" condition will be gone in a few commits as we
fix up a few tests to create their own "hooks" and "branches"
directories.

This change reduces the I/O the test suite generates by quite a
bit. Before this running it with --debug (so that trash is retained)
results in ~770MB of trash* (according to "du -shc"), after it's
reduced to around 590M, so almost 1/4 bytes we wrote were the same
sample hooks and other repetitive data. The number of files created
went down from ~151k to ~110k, which around the same reduction
of (more than) 1/4.

This will also make our tests more reliable as we're now forced to
check whether our software works with any arbitrary --template that
may be in use in the wild, as opposed to our relatively "fat" current
default.

On the "more on that below", these are the directories we're
creating (only in the test suite) for now:

- "hooks": Removing it will be relatively simple, mostly a matter of a
  few tests needing a "mkdir .git/hooks", or the tests assume that
  they'll need to explicitly disable .git/hooks.

- "branches": Similarly trivial. The last attempt to get rid of it was
  aborted in c8a58ac5a52 (Revert "Don't create the $GIT_DIR/branches
  directory on init", 2009-10-31), but only a couple of tests files
  depend on it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/init-db.c | 41 ++++++++++++++++++++++++++++++++++++++++-
 t/t0001-init.sh   | 19 ++++++++-----------
 t/test-lib.sh     |  6 ++++--
 wrap-for-bin.sh   |  4 +---
 4 files changed, 53 insertions(+), 17 deletions(-)

diff --git a/builtin/init-db.c b/builtin/init-db.c
index fcf538193c8..8dddb47bdc4 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -93,10 +93,36 @@ static void copy_templates_1(struct strbuf *path, struct strbuf *template_path,
 	}
 }
 
+static void lazy_mkdir_strbuf_or_die_setlen(struct strbuf *path, size_t oldlen,
+					    const char *dir)
+{
+	strbuf_addstr(path, dir);
+	if (mkdir(path->buf, 0777) < 0) {
+		int saved_errno = errno;
+		struct stat st;
+
+		/*
+		 * Unfortunately there's no EEXIST_{DIR,FILE}, and
+		 * we'd like to pass these only if the path is already
+		 * what we want it to be, not if it's a normal.
+		 */
+		if (lstat(path->buf, &st))
+			die_errno(_("cannot stat '%s'"), path->buf);
+		else if (S_ISDIR(st.st_mode))
+			goto cleanup;
+
+		errno = saved_errno;
+		die_errno(_("cannot mkdir '%s'"), path->buf);
+	}
+cleanup:
+	strbuf_setlen(path, oldlen);
+}
+
 static void copy_templates(int no_template, const char *template_dir,
 			   const char *init_template_dir)
 {
 	struct strbuf path = STRBUF_INIT;
+	size_t len;
 	struct strbuf template_path = STRBUF_INIT;
 	size_t template_len;
 	struct repository_format template_format = REPOSITORY_FORMAT_INIT;
@@ -108,7 +134,7 @@ static void copy_templates(int no_template, const char *template_dir,
 		return;
 	if (!template_dir && !init_template_dir &&
 	    git_env_bool(GIT_NO_TEMPLATE_DIR_ENVIRONMENT, 0))
-		return;
+		goto no_template;
 	if (!template_dir)
 		template_dir = getenv(TEMPLATE_DIR_ENVIRONMENT);
 	if (!template_dir)
@@ -157,6 +183,19 @@ static void copy_templates(int no_template, const char *template_dir,
 	strbuf_release(&path);
 	strbuf_release(&template_path);
 	clear_repository_format(&template_format);
+	return;
+no_template:
+	if (!git_env_bool("GIT_TEST_BARE_TEMPLATE", 0))
+		return;
+
+	strbuf_addstr(&path, get_git_common_dir());
+	strbuf_complete(&path, '/');
+	len = path.len;
+
+	lazy_mkdir_strbuf_or_die_setlen(&path, len, "hooks");
+	lazy_mkdir_strbuf_or_die_setlen(&path, len, "branches");
+
+	strbuf_release(&path);
 }
 
 /*
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 39cf132e9a0..a91de28aadc 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -235,7 +235,7 @@ test_expect_success 'init with --template' '
 
 test_expect_success 'init with --template (blank)' '
 	git init template-plain &&
-	test_path_is_file template-plain/.git/info/exclude &&
+	test_path_is_dir template-plain/.git/info &&
 	git init --template= template-blank &&
 	test_path_is_missing template-blank/.git/info/exclude
 '
@@ -246,19 +246,12 @@ no_templatedir_env () {
 	export NO_SET_GIT_TEMPLATE_DIR
 }
 
-init_no_templatedir_env () {
-	(
-		no_templatedir_env &&
-		git init "$1"
-	)
-}
-
 test_expect_success 'init with init.templatedir set' '
 	mkdir templatedir-source &&
 	echo Content >templatedir-source/file &&
 	test_config_global init.templatedir "${HOME}/templatedir-source" &&
 
-	init_no_templatedir_env templatedir-set &&
+	git init templatedir-set &&
 	test_cmp templatedir-source/file templatedir-set/.git/file
 '
 
@@ -267,7 +260,7 @@ test_expect_success 'init with init.templatedir using ~ expansion' '
 	echo Content >templatedir-source/file &&
 	test_config_global init.templatedir "~/templatedir-source" &&
 
-	init_no_templatedir_env templatedir-expansion &&
+	git init templatedir-expansion &&
 	test_cmp templatedir-source/file templatedir-expansion/.git/file
 '
 
@@ -563,15 +556,19 @@ test_expect_success 'remote init from does not use config from cwd' '
 '
 
 test_expect_success 're-init from a linked worktree' '
-	git init main-worktree &&
 	(
+		git init main-worktree &&
+
 		cd main-worktree &&
 		test_commit first &&
 		git worktree add ../linked-worktree &&
+		>empty &&
+		cp empty .git/info/exclude &&
 		mv .git/info/exclude expected-exclude &&
 		cp .git/config expected-config &&
 		find .git/worktrees -print | sort >expected &&
 		git -C ../linked-worktree init &&
+		cp empty .git/info/exclude &&
 		test_cmp expected-exclude .git/info/exclude &&
 		test_cmp expected-config .git/config &&
 		find .git/worktrees -print | sort >actual &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 57efcc5e97a..bd09d691da3 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1355,11 +1355,13 @@ else # normal case, use ../bin-wrappers only unless $with_dashes:
 		PATH="$GIT_BUILD_DIR:$GIT_BUILD_DIR/t/helper:$PATH"
 	fi
 fi
-GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt
+GIT_NO_TEMPLATE_DIR=true
+GIT_TEST_BARE_TEMPLATE=true
+export GIT_NO_TEMPLATE_DIR GIT_TEST_BARE_TEMPLATE
 GIT_CONFIG_NOSYSTEM=1
 GIT_ATTR_NOSYSTEM=1
 GIT_CEILING_DIRECTORIES="$TRASH_DIRECTORY/.."
-export PATH GIT_EXEC_PATH GIT_TEMPLATE_DIR GIT_CONFIG_NOSYSTEM GIT_ATTR_NOSYSTEM GIT_CEILING_DIRECTORIES
+export PATH GIT_EXEC_PATH GIT_CONFIG_NOSYSTEM GIT_ATTR_NOSYSTEM GIT_CEILING_DIRECTORIES
 
 if test -z "$GIT_TEST_CMP"
 then
diff --git a/wrap-for-bin.sh b/wrap-for-bin.sh
index 95851b85b6b..26efe0b60ac 100644
--- a/wrap-for-bin.sh
+++ b/wrap-for-bin.sh
@@ -7,10 +7,8 @@
 # @@BUILD_DIR@@ and @@PROG@@.
 
 GIT_EXEC_PATH='@@BUILD_DIR@@'
-if test -n "$NO_SET_GIT_TEMPLATE_DIR"
+if test -z "$GIT_TEMPLATE_DIR" && test -z "$GIT_NO_TEMPLATE_DIR"
 then
-	unset GIT_TEMPLATE_DIR
-else
 	GIT_TEMPLATE_DIR='@@BUILD_DIR@@/templates/blt'
 	export GIT_TEMPLATE_DIR
 fi
-- 
2.34.1.1020.gb1392dd1877


  parent reply	other threads:[~2021-12-12 20:14 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-12 20:13 [PATCH 00/13] tests + init: don't rely on templates & add --no-template + config Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 01/13] t0001: fix gaps in "TEMPLATE DIRECTORY" coverage Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 02/13] init: split out template population from create_default_files() Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 03/13] init: unconditionally create the "info" directory Ævar Arnfjörð Bjarmason
2021-12-20 15:59   ` Derrick Stolee
2021-12-20 16:13     ` Ævar Arnfjörð Bjarmason
2021-12-20 17:39       ` Derrick Stolee
2021-12-20 18:16         ` Ævar Arnfjörð Bjarmason
2021-12-20 19:06         ` Junio C Hamano
2021-12-21  1:15           ` Ævar Arnfjörð Bjarmason
2021-12-21  2:10             ` Junio C Hamano
2021-12-21  2:39               ` Ævar Arnfjörð Bjarmason
2021-12-21  6:38                 ` Junio C Hamano
2021-12-24 17:26                   ` Ævar Arnfjörð Bjarmason
2021-12-25  1:58                     ` Junio C Hamano
2022-01-12 12:42         ` Ævar Arnfjörð Bjarmason
2022-01-18 19:43           ` Derrick Stolee
2022-01-19  1:00             ` Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 04/13] t0008: don't rely on default ".git/info/exclude" Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 05/13] init & clone: add a --no-template option Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 06/13] init & clone: add init.templateDir=[bool] Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` Ævar Arnfjörð Bjarmason [this message]
2021-12-12 20:13 ` [PATCH 08/13] tests: don't depend on template-created .git/branches Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 09/13] t5540: don't rely on "hook/post-update.sample" Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 10/13] test-lib-functions: add and use a "write_hook" wrapper Ævar Arnfjörð Bjarmason
2021-12-13 14:15   ` Eric Sunshine
2021-12-13 16:29     ` Ævar Arnfjörð Bjarmason
2021-12-13 16:45       ` Eric Sunshine
2021-12-13 19:37         ` Ævar Arnfjörð Bjarmason
2021-12-13 21:33           ` Eric Sunshine
2021-12-12 20:13 ` [PATCH 11/13] tests: change "cat && chmod +x" to use "write_hook" Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 12/13] tests: migrate miscellaneous "write_script" to "write_hooks" Ævar Arnfjörð Bjarmason
2021-12-12 20:13 ` [PATCH 13/13] tests: don't depend on template-created .git/hooks Ævar Arnfjörð Bjarmason
2022-06-03 11:15 ` [PATCH v2 0/7] tests: don't depend on "git init" using the template Ævar Arnfjörð Bjarmason
2022-06-03 11:15   ` [PATCH v2 1/7] t0008: don't rely on default ".git/info/exclude" Ævar Arnfjörð Bjarmason
2022-06-03 11:15   ` [PATCH v2 2/7] tests: don't depend on template-created .git/branches Ævar Arnfjörð Bjarmason
2022-06-03 11:15   ` [PATCH v2 3/7] tests: don't assume a .git/info for .git/info/grafts Ævar Arnfjörð Bjarmason
2022-06-03 11:15   ` [PATCH v2 4/7] tests: don't assume a .git/info for .git/info/attributes Ævar Arnfjörð Bjarmason
2022-06-03 11:15   ` [PATCH v2 5/7] tests: don't assume a .git/info for .git/info/refs Ævar Arnfjörð Bjarmason
2022-06-03 11:15   ` [PATCH v2 6/7] tests: don't assume a .git/info for .git/info/exclude Ævar Arnfjörð Bjarmason
2022-06-03 11:15   ` [PATCH v2 7/7] tests: don't assume a .git/info for .git/info/sparse-checkout Ævar Arnfjörð Bjarmason
2022-06-03 19:17   ` [PATCH v2 0/7] tests: don't depend on "git init" using the template Junio C Hamano
2022-06-04  0:41     ` Ævar Arnfjörð Bjarmason
2022-06-06 19:08       ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=patch-07.13-9402fb23b40-20211212T201308Z-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=derrickstolee@github.com \
    --cc=git@adamspiers.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).