git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Denton Liu <liu.denton@gmail.com>
To: Git Mailing List <git@vger.kernel.org>
Cc: Eric Sunshine <sunshine@sunshineco.com>,
	Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>
Subject: [PATCH v6 1/5] t7700: consolidate code into test_no_missing_in_packs()
Date: Wed, 4 Dec 2019 14:03:09 -0800
Message-ID: <3008ce8deb4055f39fd4e1331b30666b40b26eaa.1575496684.git.liu.denton@gmail.com> (raw)
In-Reply-To: <cover.1575496683.git.liu.denton@gmail.com>

The code to test that objects were not missing from the packfile was
duplicated many times. Extract the duplicated code into
test_no_missing_in_packs() and use that instead.

Refactor the resulting extraction so that if any git commands fail,
their return codes are not silently lost.

Instead of verifying each file of `alt_objects/pack/*.idx` individually
in a for-loop, batch them together into one verification step.

The original testing construct was O(n^2): it used a grep in a loop to
test whether any objects were missing in the packfile. Rewrite this to
extract the hash using sed or cut, sort the files, then use `comm -23`
so that finding missing lines from the original file is done more
efficiently.

While we're at it, add a space to `commit_and_pack ()` for style.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/t7700-repack.sh | 55 +++++++++++++----------------------------------
 1 file changed, 15 insertions(+), 40 deletions(-)

diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh
index 1d14ddcbdb..5fb9e99f34 100755
--- a/t/t7700-repack.sh
+++ b/t/t7700-repack.sh
@@ -4,12 +4,23 @@ test_description='git repack works correctly'
 
 . ./test-lib.sh
 
-commit_and_pack() {
+commit_and_pack () {
 	test_commit "$@" 1>&2 &&
 	SHA1=$(git pack-objects --all --unpacked --incremental .git/objects/pack/pack </dev/null) &&
 	echo pack-${SHA1}.pack
 }
 
+test_no_missing_in_packs () {
+	myidx=$(ls -1 .git/objects/pack/*.idx) &&
+	test_path_is_file "$myidx" &&
+	git verify-pack -v alt_objects/pack/*.idx >orig.raw &&
+	sed -n -e "s/^\([0-9a-f]\{40\}\).*/\1/p" orig.raw | sort >orig &&
+	git verify-pack -v $myidx >dest.raw &&
+	cut -d" " -f1 dest.raw | sort >dest &&
+	comm -23 orig dest >missing &&
+	test_must_be_empty missing
+}
+
 test_expect_success 'objects in packs marked .keep are not repacked' '
 	echo content1 >file1 &&
 	echo content2 >file2 &&
@@ -105,19 +116,7 @@ test_expect_success 'packed obs in alt ODB are repacked even when local repo is
 	mkdir alt_objects/pack &&
 	mv .git/objects/pack/* alt_objects/pack &&
 	git repack -a &&
-	myidx=$(ls -1 .git/objects/pack/*.idx) &&
-	test_path_is_file "$myidx" &&
-	for p in alt_objects/pack/*.idx
-	do
-		git verify-pack -v $p | sed -n -e "/^[0-9a-f]\{40\}/p"
-	done | while read sha1 rest
-	do
-		if ! ( git verify-pack -v $myidx | grep "^$sha1" )
-		then
-			echo "Missing object in local pack: $sha1"
-			return 1
-		fi
-	done
+	test_no_missing_in_packs
 '
 
 test_expect_success 'packed obs in alt ODB are repacked when local repo has packs' '
@@ -128,19 +127,7 @@ test_expect_success 'packed obs in alt ODB are repacked when local repo has pack
 	git commit -m more_content &&
 	git repack &&
 	git repack -a -d &&
-	myidx=$(ls -1 .git/objects/pack/*.idx) &&
-	test_path_is_file "$myidx" &&
-	for p in alt_objects/pack/*.idx
-	do
-		git verify-pack -v $p | sed -n -e "/^[0-9a-f]\{40\}/p"
-	done | while read sha1 rest
-	do
-		if ! ( git verify-pack -v $myidx | grep "^$sha1" )
-		then
-			echo "Missing object in local pack: $sha1"
-			return 1
-		fi
-	done
+	test_no_missing_in_packs
 '
 
 test_expect_success 'packed obs in alternate ODB kept pack are repacked' '
@@ -156,19 +143,7 @@ test_expect_success 'packed obs in alternate ODB kept pack are repacked' '
 		fi
 	done &&
 	git repack -a -d &&
-	myidx=$(ls -1 .git/objects/pack/*.idx) &&
-	test_path_is_file "$myidx" &&
-	for p in alt_objects/pack/*.idx
-	do
-		git verify-pack -v $p | sed -n -e "/^[0-9a-f]\{40\}/p"
-	done | while read sha1 rest
-	do
-		if ! ( git verify-pack -v $myidx | grep "^$sha1" )
-		then
-			echo "Missing object in local pack: $sha1"
-			return 1
-		fi
-	done
+	test_no_missing_in_packs
 '
 
 test_expect_success 'packed unreachable obs in alternate ODB are not loosened' '
-- 
2.24.0.578.g4820254054


  reply index

Thread overview: 169+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-15  1:00 [PATCH 00/27] t: general test cleanup + `set -o pipefail` Denton Liu
2019-11-15  1:00 ` [PATCH 01/27] lib-bash.sh: move `then` onto its own line Denton Liu
2019-11-15 18:22   ` Eric Sunshine
2019-11-16  2:50     ` Junio C Hamano
2019-11-15  1:00 ` [PATCH 02/27] t0014: remove git command upstream of pipe Denton Liu
2019-11-15  1:00 ` [PATCH 03/27] t0090: stop losing return codes of git commands Denton Liu
2019-11-15  1:00 ` [PATCH 04/27] t3301: " Denton Liu
2019-11-15  1:00 ` [PATCH 05/27] t3600: use test_line_count() where possible Denton Liu
2019-11-15  1:00 ` [PATCH 06/27] t3600: stop losing return codes of git commands Denton Liu
2019-11-15  1:00 ` [PATCH 07/27] t3600: comment on inducing SIGPIPE in `git rm` Denton Liu
2019-11-15  1:00 ` [PATCH 08/27] t4015: stop losing return codes of git commands Denton Liu
2019-11-15  1:00 ` [PATCH 09/27] t4015: use test_write_lines() Denton Liu
2019-11-15  1:00 ` [PATCH 10/27] t4138: stop losing return codes of git commands Denton Liu
2019-11-16  9:00   ` Eric Sunshine
2019-11-15  1:00 ` [PATCH 11/27] t5317: " Denton Liu
2019-11-15  1:00 ` [PATCH 12/27] t5317: use ! grep to check for no matching lines Denton Liu
2019-11-16  9:27   ` Eric Sunshine
2019-11-15  1:01 ` [PATCH 13/27] t5703: stop losing return codes of git commands Denton Liu
2019-11-16 10:11   ` Eric Sunshine
2019-11-15  1:01 ` [PATCH 14/27] t7501: remove spaces after redirect operators Denton Liu
2019-11-15  1:01 ` [PATCH 15/27] t7501: stop losing return codes of git commands Denton Liu
2019-11-16 10:35   ` Eric Sunshine
2019-11-15  1:01 ` [PATCH 16/27] t7700: drop redirections to /dev/null Denton Liu
2019-11-15  1:01 ` [PATCH 17/27] t7700: remove spaces after redirect operators Denton Liu
2019-11-15  1:01 ` [PATCH 18/27] t7700: move keywords onto their own line Denton Liu
2019-11-15  1:01 ` [PATCH 19/27] t7700: s/test -f/test_path_is_file/ Denton Liu
2019-11-15  1:01 ` [PATCH 20/27] t7700: stop losing return codes of git commands Denton Liu
2019-11-15  1:01 ` [PATCH 21/27] t: define test_grep_return_success() Denton Liu
2019-11-15  5:26   ` Junio C Hamano
2019-11-15  1:01 ` [PATCH 22/27] t0090: mask failing grep status Denton Liu
2019-11-15  1:01 ` [PATCH 23/27] t3600: mark git command as failing Denton Liu
2019-11-15  5:35   ` Junio C Hamano
2019-11-15  1:01 ` [PATCH 24/27] t5004: ignore SIGPIPE in zipinfo Denton Liu
2019-11-15  5:36   ` Junio C Hamano
2019-11-15  1:01 ` [PATCH 25/27] t5703: mask failing grep status Denton Liu
2019-11-15  1:01 ` [PATCH 26/27] t9902: disable pipefail Denton Liu
2019-11-15  1:01 ` [PATCH 27/27] t: run tests with `set -o pipefail` on Bash Denton Liu
2019-11-15  4:09 ` [PATCH 00/27] t: general test cleanup + `set -o pipefail` Jeff King
2019-11-21  0:45 ` [PATCH v2 00/21] t: test cleanup stemming from experimentally enabling pipefail Denton Liu
2019-11-21  0:45   ` [PATCH v2 01/21] lib-bash.sh: move `then` onto its own line Denton Liu
2019-11-21  0:45   ` [PATCH v2 02/21] t0014: remove git command upstream of pipe Denton Liu
2019-11-21  0:45   ` [PATCH v2 03/21] t0090: stop losing return codes of git commands Denton Liu
2019-11-21  0:45   ` [PATCH v2 04/21] t3301: " Denton Liu
2019-11-21  0:45   ` [PATCH v2 05/21] t3600: use test_line_count() where possible Denton Liu
2019-11-21  0:46   ` [PATCH v2 06/21] t3600: stop losing return codes of git commands Denton Liu
2019-11-21  0:46   ` [PATCH v2 07/21] t3600: comment on inducing SIGPIPE in `git rm` Denton Liu
2019-11-21  0:46   ` [PATCH v2 08/21] t4015: stop losing return codes of git commands Denton Liu
2019-11-21  0:46   ` [PATCH v2 09/21] t4015: use test_write_lines() Denton Liu
2019-11-21  0:46   ` [PATCH v2 10/21] t4138: stop losing return codes of git commands Denton Liu
2019-11-21  0:46   ` [PATCH v2 11/21] t5317: " Denton Liu
2019-11-21  0:46   ` [PATCH v2 12/21] t5317: use ! grep to check for no matching lines Denton Liu
2019-11-21 12:59     ` Eric Sunshine
2019-11-21  0:46   ` [PATCH v2 13/21] t5703: simplify one-time-sed generation logic Denton Liu
2019-11-21 13:12     ` Eric Sunshine
2019-11-21 23:22       ` Denton Liu
2019-11-21  0:46   ` [PATCH v2 14/21] t5703: stop losing return codes of git commands Denton Liu
2019-11-21  0:46   ` [PATCH v2 15/21] t7501: remove spaces after redirect operators Denton Liu
2019-11-21  0:46   ` [PATCH v2 16/21] t7501: stop losing return codes of git commands Denton Liu
2019-11-21  0:46   ` [PATCH v2 17/21] t7700: drop redirections to /dev/null Denton Liu
2019-11-21  0:46   ` [PATCH v2 18/21] t7700: remove spaces after redirect operators Denton Liu
2019-11-21  0:46   ` [PATCH v2 19/21] t7700: move keywords onto their own line Denton Liu
2019-11-21  0:46   ` [PATCH v2 20/21] t7700: s/test -f/test_path_is_file/ Denton Liu
2019-11-21  0:46   ` [PATCH v2 21/21] t7700: stop losing return codes of git commands Denton Liu
2019-11-22 18:59   ` [PATCH v3 00/22] t: test cleanup stemming from experimentally enabling pipefail Denton Liu
2019-11-22 18:59     ` [PATCH v3 01/22] lib-bash.sh: move `then` onto its own line Denton Liu
2019-11-22 18:59     ` [PATCH v3 02/22] apply-one-time-sed.sh: modernize style Denton Liu
2019-11-23  1:32       ` Junio C Hamano
2019-11-22 18:59     ` [PATCH v3 03/22] t0014: remove git command upstream of pipe Denton Liu
2019-11-22 18:59     ` [PATCH v3 04/22] t0090: stop losing return codes of git commands Denton Liu
2019-11-22 18:59     ` [PATCH v3 05/22] t3301: " Denton Liu
2019-11-22 18:59     ` [PATCH v3 06/22] t3600: use test_line_count() where possible Denton Liu
2019-11-22 18:59     ` [PATCH v3 07/22] t3600: stop losing return codes of git commands Denton Liu
2019-11-22 19:00     ` [PATCH v3 08/22] t3600: comment on inducing SIGPIPE in `git rm` Denton Liu
2019-11-22 19:00     ` [PATCH v3 09/22] t4015: stop losing return codes of git commands Denton Liu
2019-11-22 19:00     ` [PATCH v3 10/22] t4015: use test_write_lines() Denton Liu
2019-11-22 19:00     ` [PATCH v3 11/22] t4138: stop losing return codes of git commands Denton Liu
2019-11-22 19:00     ` [PATCH v3 12/22] t5317: " Denton Liu
2019-11-22 19:00     ` [PATCH v3 13/22] t5317: use ! grep to check for no matching lines Denton Liu
2019-11-23  6:21       ` Eric Sunshine
2019-11-25 21:43         ` Denton Liu
2019-11-22 19:00     ` [PATCH v3 14/22] t5703: simplify one-time-sed generation logic Denton Liu
2019-11-22 19:00     ` [PATCH v3 15/22] t5703: stop losing return codes of git commands Denton Liu
2019-11-22 19:00     ` [PATCH v3 16/22] t7501: remove spaces after redirect operators Denton Liu
2019-11-22 19:00     ` [PATCH v3 17/22] t7501: stop losing return codes of git commands Denton Liu
2019-11-22 19:00     ` [PATCH v3 18/22] t7700: drop redirections to /dev/null Denton Liu
2019-11-22 19:00     ` [PATCH v3 19/22] t7700: remove spaces after redirect operators Denton Liu
2019-11-22 19:00     ` [PATCH v3 20/22] t7700: move keywords onto their own line Denton Liu
2019-11-22 19:00     ` [PATCH v3 21/22] t7700: s/test -f/test_path_is_file/ Denton Liu
2019-11-22 19:00     ` [PATCH v3 22/22] t7700: stop losing return codes of git commands Denton Liu
2019-11-23  1:49       ` Junio C Hamano
2019-11-25 23:57         ` Denton Liu
2019-11-26  0:58           ` Eric Sunshine
2019-11-26  1:34             ` Junio C Hamano
2019-11-26  4:47               ` Denton Liu
2019-11-26  1:17     ` [PATCH v4 00/27] t: test cleanup stemming from experimentally enabling pipefail Denton Liu
2019-11-26  1:17       ` [PATCH v4 01/27] lib-bash.sh: move `then` onto its own line Denton Liu
2019-11-26  1:17       ` [PATCH v4 02/27] apply-one-time-sed.sh: modernize style Denton Liu
2019-11-26  1:17       ` [PATCH v4 03/27] t0014: remove git command upstream of pipe Denton Liu
2019-11-26  1:17       ` [PATCH v4 04/27] t0090: stop losing return codes of git commands Denton Liu
2019-11-26  1:17       ` [PATCH v4 05/27] t3301: " Denton Liu
2019-11-26  1:17       ` [PATCH v4 06/27] t3600: use test_line_count() where possible Denton Liu
2019-11-26  1:18       ` [PATCH v4 07/27] t3600: stop losing return codes of git commands Denton Liu
2019-11-26  1:18       ` [PATCH v4 08/27] t3600: comment on inducing SIGPIPE in `git rm` Denton Liu
2019-11-26  1:18       ` [PATCH v4 09/27] t4015: stop losing return codes of git commands Denton Liu
2019-11-26  1:18       ` [PATCH v4 10/27] t4015: use test_write_lines() Denton Liu
2019-11-26  1:18       ` [PATCH v4 11/27] t4138: stop losing return codes of git commands Denton Liu
2019-11-26  1:18       ` [PATCH v4 12/27] t5317: " Denton Liu
2019-11-26  1:18       ` [PATCH v4 13/27] t5317: use ! grep to check for no matching lines Denton Liu
2019-11-26  1:18       ` [PATCH v4 14/27] t5703: simplify one-time-sed generation logic Denton Liu
2019-11-26  1:18       ` [PATCH v4 15/27] t5703: stop losing return codes of git commands Denton Liu
2019-11-26  1:18       ` [PATCH v4 16/27] t7501: remove spaces after redirect operators Denton Liu
2019-11-26  1:18       ` [PATCH v4 17/27] t7501: stop losing return codes of git commands Denton Liu
2019-11-26  1:18       ` [PATCH v4 18/27] t7700: drop redirections to /dev/null Denton Liu
2019-11-26  1:18       ` [PATCH v4 19/27] t7700: remove spaces after redirect operators Denton Liu
2019-11-26  1:18       ` [PATCH v4 20/27] t7700: move keywords onto their own line Denton Liu
2019-11-26  1:18       ` [PATCH v4 21/27] t7700: s/test -f/test_path_is_file/ Denton Liu
2019-11-26  1:18       ` [PATCH v4 22/27] t7700: consolidate code into test_no_missing_in_packs() Denton Liu
2019-11-26  2:35         ` Eric Sunshine
2019-11-26  1:18       ` [PATCH v4 23/27] squash! " Denton Liu
2019-11-26  1:18       ` [PATCH v4 24/27] t7700: consolidate code into test_has_duplicate_object() Denton Liu
2019-11-26  1:18       ` [PATCH v4 25/27] t7700: replace egrep with grep Denton Liu
2019-11-26  1:18       ` [PATCH v4 26/27] t7700: make references to SHA-1 generic Denton Liu
2019-11-26  3:15         ` Eric Sunshine
2019-11-26  1:18       ` [PATCH v4 27/27] t7700: stop losing return codes of git commands Denton Liu
2019-11-27 19:53       ` [PATCH v5 00/26] t: test cleanup stemming from experimentally enabling pipefail Denton Liu
2019-11-27 19:53         ` [PATCH v5 01/26] lib-bash.sh: move `then` onto its own line Denton Liu
2019-11-27 19:53         ` [PATCH v5 02/26] apply-one-time-sed.sh: modernize style Denton Liu
2019-11-27 19:53         ` [PATCH v5 03/26] t0014: remove git command upstream of pipe Denton Liu
2019-11-27 19:53         ` [PATCH v5 04/26] t0090: stop losing return codes of git commands Denton Liu
2019-11-27 19:53         ` [PATCH v5 05/26] t3301: " Denton Liu
2019-11-27 19:53         ` [PATCH v5 06/26] t3600: use test_line_count() where possible Denton Liu
2019-11-27 19:53         ` [PATCH v5 07/26] t3600: stop losing return codes of git commands Denton Liu
2019-11-27 19:53         ` [PATCH v5 08/26] t3600: comment on inducing SIGPIPE in `git rm` Denton Liu
2019-11-27 19:53         ` [PATCH v5 09/26] t4015: stop losing return codes of git commands Denton Liu
2019-11-27 19:53         ` [PATCH v5 10/26] t4015: use test_write_lines() Denton Liu
2019-11-27 19:53         ` [PATCH v5 11/26] t4138: stop losing return codes of git commands Denton Liu
2019-11-27 19:53         ` [PATCH v5 12/26] t5317: " Denton Liu
2019-11-27 19:53         ` [PATCH v5 13/26] t5317: use ! grep to check for no matching lines Denton Liu
2019-11-27 19:53         ` [PATCH v5 14/26] t5703: simplify one-time-sed generation logic Denton Liu
2019-11-27 19:53         ` [PATCH v5 15/26] t5703: stop losing return codes of git commands Denton Liu
2019-11-27 19:53         ` [PATCH v5 16/26] t7501: remove spaces after redirect operators Denton Liu
2019-11-27 19:53         ` [PATCH v5 17/26] t7501: stop losing return codes of git commands Denton Liu
2019-11-27 19:53         ` [PATCH v5 18/26] t7700: drop redirections to /dev/null Denton Liu
2019-11-27 19:53         ` [PATCH v5 19/26] t7700: remove spaces after redirect operators Denton Liu
2019-11-27 19:53         ` [PATCH v5 20/26] t7700: move keywords onto their own line Denton Liu
2019-11-27 19:53         ` [PATCH v5 21/26] t7700: s/test -f/test_path_is_file/ Denton Liu
2019-11-27 19:53         ` [PATCH v5 22/26] t7700: consolidate code into test_no_missing_in_packs() Denton Liu
2019-11-29 21:39           ` Junio C Hamano
2019-12-02 20:50             ` Denton Liu
2019-12-02 22:53               ` Junio C Hamano
2019-12-02 23:28                 ` Denton Liu
2019-12-03 15:41                   ` Junio C Hamano
2019-12-04  7:24                     ` Denton Liu
2019-12-04 18:13                       ` Junio C Hamano
2019-12-04 22:03                         ` [PATCH v6 0/5] t: test cleanup stemming from experimentally enabling pipefail Denton Liu
2019-12-04 22:03                           ` Denton Liu [this message]
2019-12-04 22:03                           ` [PATCH v6 2/5] t7700: consolidate code into test_has_duplicate_object() Denton Liu
2019-12-04 22:03                           ` [PATCH v6 3/5] t7700: replace egrep with grep Denton Liu
2019-12-04 22:03                           ` [PATCH v6 4/5] t7700: make references to SHA-1 generic Denton Liu
2019-12-04 22:03                           ` [PATCH v6 5/5] t7700: stop losing return codes of git commands Denton Liu
2019-12-04 22:07                           ` [PATCH v6 0/5] t: test cleanup stemming from experimentally enabling pipefail Junio C Hamano
2019-11-27 19:53         ` [PATCH v5 23/26] t7700: consolidate code into test_has_duplicate_object() Denton Liu
2019-11-27 19:53         ` [PATCH v5 24/26] t7700: replace egrep with grep Denton Liu
2019-11-27 19:54         ` [PATCH v5 25/26] t7700: make references to SHA-1 generic Denton Liu
2019-11-27 19:54         ` [PATCH v5 26/26] t7700: stop losing return codes of git commands Denton Liu
2019-11-30 10:48           ` Danh Doan
2019-11-30 11:31             ` Eric Sunshine
2019-11-30 17:00               ` Junio C Hamano
2019-12-04 12:59                 ` t: remove inappropriate uses of test_must_fail(), was " Denton Liu

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3008ce8deb4055f39fd4e1331b30666b40b26eaa.1575496684.git.liu.denton@gmail.com \
    --to=liu.denton@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git