git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Matheus Tavares <matheus.bernardino@usp.br>
To: git@vger.kernel.org
Cc: christian.couder@gmail.com, git@jeffhostetler.com,
	Jonathan Nieder <jrnieder@gmail.com>
Subject: [PATCH 4/7] parallel-checkout: add tests for basic operations
Date: Thu, 22 Apr 2021 12:17:50 -0300	[thread overview]
Message-ID: <6379b8df6a59361dd44733e379880a11c6cd977c.1619104091.git.matheus.bernardino@usp.br> (raw)
In-Reply-To: <cover.1619104091.git.matheus.bernardino@usp.br>

Add tests to populate the working tree during clone and checkout using
sequential and parallel mode, to confirm that they produce identical
results. Also test basic checkout mechanics, such as checking for
symlinks in the leading directories and the abidance to --force.

Note: some helper functions are added to a common lib file which is only
included by t2080 for now. But they will also be used by other
parallel-checkout tests in the following patches.

Original-patch-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 t/lib-parallel-checkout.sh          |  37 +++++++
 t/t2080-parallel-checkout-basics.sh | 150 ++++++++++++++++++++++++++++
 2 files changed, 187 insertions(+)
 create mode 100644 t/lib-parallel-checkout.sh
 create mode 100755 t/t2080-parallel-checkout-basics.sh

diff --git a/t/lib-parallel-checkout.sh b/t/lib-parallel-checkout.sh
new file mode 100644
index 0000000000..39fd36fdf6
--- /dev/null
+++ b/t/lib-parallel-checkout.sh
@@ -0,0 +1,37 @@
+# Helpers for t208* tests
+
+set_checkout_config () {
+	if test $# -ne 2
+	then
+		BUG "set_checkout_config() requires two arguments"
+	fi &&
+
+	test_config_global checkout.workers $1 &&
+	test_config_global checkout.thresholdForParallelism $2
+}
+
+# Run "${@:2}" and check that $1 checkout workers were used
+test_checkout_workers () {
+	if test $# -lt 2
+	then
+		BUG "too few arguments to test_checkout_workers()"
+	fi &&
+
+	expected_workers=$1 &&
+	shift &&
+
+	rm -f trace &&
+	GIT_TRACE2="$(pwd)/trace" "$@" &&
+
+	workers=$(grep "child_start\[..*\] git checkout--worker" trace | wc -l) &&
+	test $workers -eq $expected_workers &&
+	rm -f trace
+}
+
+# Verify that both the working tree and the index were created correctly
+verify_checkout () {
+	git -C "$1" diff-index --quiet HEAD -- &&
+	git -C "$1" diff-index --quiet --cached HEAD -- &&
+	git -C "$1" status --porcelain >"$1".status &&
+	test_must_be_empty "$1".status
+}
diff --git a/t/t2080-parallel-checkout-basics.sh b/t/t2080-parallel-checkout-basics.sh
new file mode 100755
index 0000000000..0cb1493cdc
--- /dev/null
+++ b/t/t2080-parallel-checkout-basics.sh
@@ -0,0 +1,150 @@
+#!/bin/sh
+
+test_description='parallel-checkout basics
+
+Ensure that parallel-checkout basically works on clone and checkout, spawning
+the required number of workers and correctly populating both the index and the
+working tree.
+'
+
+TEST_NO_CREATE_REPO=1
+. ./test-lib.sh
+. "$TEST_DIRECTORY/lib-parallel-checkout.sh"
+
+# Test parallel-checkout with a branch switch containing file creations,
+# deletions, and modification; with different entry types. Switching from B1 to
+# B2 will have the following changes:
+#
+# - a (file):      modified
+# - e/x (file):    deleted
+# - b (symlink):   deleted
+# - b/f (file):    created
+# - e (symlink):   created
+# - d (submodule): created
+#
+test_expect_success SYMLINKS 'setup repo for checkout with various types of changes' '
+	git init various &&
+	(
+		cd various &&
+		git checkout -b B1 &&
+		echo a >a &&
+		mkdir e &&
+		echo e/x >e/x &&
+		ln -s e b &&
+		git add -A &&
+		git commit -m B1 &&
+
+		git checkout -b B2 &&
+		echo modified >a &&
+		rm -rf e &&
+		rm b &&
+		mkdir b &&
+		echo b/f >b/f &&
+		ln -s b e &&
+		git init d &&
+		test_commit -C d f &&
+		git submodule add ./d &&
+		git add -A &&
+		git commit -m B2 &&
+
+		git checkout --recurse-submodules B1
+	)
+'
+
+test_expect_success SYMLINKS 'sequential checkout' '
+	cp -R various various_sequential &&
+	set_checkout_config 1 0 &&
+	test_checkout_workers 0 \
+		git -C various_sequential checkout --recurse-submodules B2 &&
+	verify_checkout various_sequential
+'
+
+test_expect_success SYMLINKS 'parallel checkout' '
+	cp -R various various_parallel &&
+	set_checkout_config 2 0 &&
+	test_checkout_workers 2 \
+		git -C various_parallel checkout --recurse-submodules B2 &&
+	verify_checkout various_parallel
+'
+
+test_expect_success SYMLINKS 'fallback to sequential checkout (threshold)' '
+	cp -R various various_sequential_fallback &&
+	set_checkout_config 2 100 &&
+	test_checkout_workers 0 \
+		git -C various_sequential_fallback checkout --recurse-submodules B2 &&
+	verify_checkout various_sequential_fallback
+'
+
+test_expect_success SYMLINKS 'parallel checkout on clone' '
+	git -C various checkout --recurse-submodules B2 &&
+	set_checkout_config 2 0 &&
+	test_checkout_workers 2 \
+		git clone --recurse-submodules various various_parallel_clone &&
+	verify_checkout various_parallel_clone
+'
+
+test_expect_success SYMLINKS 'fallback to sequential checkout on clone (threshold)' '
+	git -C various checkout --recurse-submodules B2 &&
+	set_checkout_config 2 100 &&
+	test_checkout_workers 0 \
+		git clone --recurse-submodules various various_sequential_fallback_clone &&
+	verify_checkout various_sequential_fallback_clone
+'
+
+# Just to be paranoid, actually compare the working trees' contents directly.
+test_expect_success SYMLINKS 'compare the working trees' '
+	rm -rf various_*/.git &&
+	rm -rf various_*/d/.git &&
+
+	diff -r various_sequential various_parallel &&
+	diff -r various_sequential various_sequential_fallback &&
+	diff -r various_sequential various_parallel_clone &&
+	diff -r various_sequential various_sequential_fallback_clone
+'
+
+test_expect_success 'parallel checkout respects --[no]-force' '
+	set_checkout_config 2 0 &&
+	git init dirty &&
+	(
+		cd dirty &&
+		mkdir D &&
+		test_commit D/F &&
+		test_commit F &&
+
+		rm -rf D &&
+		echo changed >D &&
+		echo changed >F.t &&
+
+		# We expect 0 workers because there is nothing to be done
+		test_checkout_workers 0 git checkout HEAD &&
+		test_path_is_file D &&
+		grep changed D &&
+		grep changed F.t &&
+
+		test_checkout_workers 2 git checkout --force HEAD &&
+		test_path_is_dir D &&
+		grep D/F D/F.t &&
+		grep F F.t
+	)
+'
+
+test_expect_success SYMLINKS 'parallel checkout checks for symlinks in leading dirs' '
+	set_checkout_config 2 0 &&
+	git init symlinks &&
+	(
+		cd symlinks &&
+		mkdir D untracked &&
+		# Commit 2 files to have enough work for 2 parallel workers
+		test_commit D/A &&
+		test_commit D/B &&
+		rm -rf D &&
+		ln -s untracked D &&
+
+		test_checkout_workers 2 git checkout --force HEAD &&
+		! test -h D &&
+		grep D/A D/A.t &&
+		grep D/B D/B.t
+	)
+'
+
+test_done
-- 
2.30.1


  parent reply	other threads:[~2021-04-22 15:18 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-22 15:17 [PATCH 0/7] Parallel Checkout (part 3) Matheus Tavares
2021-04-22 15:17 ` [PATCH 1/7] make_transient_cache_entry(): optionally alloc from mem_pool Matheus Tavares
2021-04-22 15:17 ` [PATCH 2/7] builtin/checkout.c: complete parallel checkout support Matheus Tavares
2021-04-23 16:19   ` Derrick Stolee
2021-04-26 21:54     ` Matheus Tavares Bernardino
2021-04-22 15:17 ` [PATCH 3/7] checkout-index: add " Matheus Tavares
2021-04-23 18:32   ` Derrick Stolee
2021-04-26 22:30     ` Matheus Tavares Bernardino
2021-04-22 15:17 ` Matheus Tavares [this message]
2021-04-23 19:18   ` [PATCH 4/7] parallel-checkout: add tests for basic operations Derrick Stolee
2021-04-27  2:30     ` Matheus Tavares Bernardino
2021-04-22 15:17 ` [PATCH 5/7] parallel-checkout: add tests related to path collisions Matheus Tavares
2021-04-22 15:17 ` [PATCH 6/7] parallel-checkout: add tests related to .gitattributes Matheus Tavares
2021-04-23 19:48   ` Derrick Stolee
2021-04-22 15:17 ` [PATCH 7/7] ci: run test round with parallel-checkout enabled Matheus Tavares
2021-04-23 19:56   ` Derrick Stolee
2021-04-30 21:40 ` [PATCH v2 0/8] Parallel Checkout (part 3) Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 1/8] make_transient_cache_entry(): optionally alloc from mem_pool Matheus Tavares
2021-05-01 17:06     ` Christian Couder
2021-05-03 14:11       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 2/8] builtin/checkout.c: complete parallel checkout support Matheus Tavares
2021-05-01 17:08     ` Christian Couder
2021-05-03 14:21       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 3/8] checkout-index: add " Matheus Tavares
2021-05-01 17:08     ` Christian Couder
2021-05-03 14:22       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 4/8] parallel-checkout: add tests for basic operations Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 5/8] parallel-checkout: add tests related to path collisions Matheus Tavares
2021-05-02  7:59     ` Torsten Bögershausen
2021-05-03 14:58       ` Matheus Tavares Bernardino
2021-04-30 21:40   ` [PATCH v2 6/8] t0028: extract encoding helpers to lib-encoding.sh Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 7/8] parallel-checkout: add tests related to .gitattributes Matheus Tavares
2021-04-30 21:40   ` [PATCH v2 8/8] ci: run test round with parallel-checkout enabled Matheus Tavares
2021-05-02 10:12   ` [PATCH v2 0/8] Parallel Checkout (part 3) Torsten Bögershausen
2021-05-03 15:01     ` Matheus Tavares Bernardino
2021-05-04 16:27   ` [PATCH v3 " Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 1/8] make_transient_cache_entry(): optionally alloc from mem_pool Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 2/8] builtin/checkout.c: complete parallel checkout support Matheus Tavares
2021-05-05 13:55       ` Derrick Stolee
2021-05-04 16:27     ` [PATCH v3 3/8] checkout-index: add " Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 4/8] parallel-checkout: add tests for basic operations Matheus Tavares
2021-05-26 18:36       ` AIX failures on parallel checkout (new in v2.32.0-rc*) Ævar Arnfjörð Bjarmason
2021-05-26 22:01         ` Matheus Tavares Bernardino
2021-05-26 23:00           ` Junio C Hamano
2021-05-04 16:27     ` [PATCH v3 5/8] parallel-checkout: add tests related to path collisions Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 6/8] t0028: extract encoding helpers to lib-encoding.sh Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 7/8] parallel-checkout: add tests related to .gitattributes Matheus Tavares
2021-05-04 16:27     ` [PATCH v3 8/8] ci: run test round with parallel-checkout enabled Matheus Tavares
2021-05-05 13:57     ` [PATCH v3 0/8] Parallel Checkout (part 3) Derrick Stolee
2021-05-06  0:40       ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6379b8df6a59361dd44733e379880a11c6cd977c.1619104091.git.matheus.bernardino@usp.br \
    --to=matheus.bernardino@usp.br \
    --cc=christian.couder@gmail.com \
    --cc=git@jeffhostetler.com \
    --cc=git@vger.kernel.org \
    --cc=jrnieder@gmail.com \
    --subject='Re: [PATCH 4/7] parallel-checkout: add tests for basic operations' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.version-control.git
	nntp://ie5yzdi7fg72h7s4sdcztq5evakq23rdt33mfyfcddc5u3ndnw24ogqd.onion/inbox.comp.version-control.git
	nntp://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git