From: Taylor Blau <me@ttaylorr.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, peff@peff.net, stolee@gmail.com
Subject: [PATCH v2 6/8] t5326: move tests to t/lib-bitmap.sh
Date: Mon, 13 Dec 2021 20:55:41 -0500 [thread overview]
Message-ID: <bf42b116e1872fd69164fe15f098b7a72e938c24.1639446906.git.me@ttaylorr.com> (raw)
In-Reply-To: <cover.1639446906.git.me@ttaylorr.com>
In t5326, we have a handful of tests that we would like to run twice:
once using the MIDX's new `RIDX` chunk as the source of the
reverse-index cache, and once using the separate `.rev` file.
But because these tests mutate the state of the underlying repository,
and then make assumptions about those mutations occurring in a certain
sequence, simply running the tests twice in the same repository is
awkward.
Instead, extract the core of interesting tests into t/lib-bitmap.sh to
prepare for them to be run twice, each in a separate test script. This
means that they can each operate on a separate repository, removing any
concerns about mutating state.
For now, this patch is a strict cut-and-paste of some tests from t5326.
The tests which did not move are not interesting with respect to the
source of their reverse index data.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
t/lib-bitmap.sh | 178 ++++++++++++++++++++++++++++++++++
t/t5326-multi-pack-bitmaps.sh | 174 +--------------------------------
2 files changed, 180 insertions(+), 172 deletions(-)
diff --git a/t/lib-bitmap.sh b/t/lib-bitmap.sh
index 21d0392dda..771c41c2ea 100644
--- a/t/lib-bitmap.sh
+++ b/t/lib-bitmap.sh
@@ -1,6 +1,9 @@
# Helpers for scripts testing bitmap functionality; see t5310 for
# example usage.
+objdir=.git/objects
+midx=$objdir/pack/multi-pack-index
+
# Compare a file containing rev-list bitmap traversal output to its non-bitmap
# counterpart. You can't just use test_cmp for this, because the two produce
# subtly different output:
@@ -264,3 +267,178 @@ have_delta () {
midx_checksum () {
test-tool read-midx --checksum "$1"
}
+
+# midx_pack_source <obj>
+midx_pack_source () {
+ test-tool read-midx --show-objects .git/objects | grep "^$1 " | cut -f2
+}
+
+test_rev_exists () {
+ commit="$1"
+
+ test_expect_success 'reverse index exists' '
+ GIT_TRACE2_EVENT_NESTING=10 \
+ GIT_TRACE2_EVENT=$(pwd)/event.trace \
+ git rev-list --test-bitmap "$commit" &&
+
+ test_path_is_file $midx-$(midx_checksum $objdir).rev &&
+ grep "\"category\":\"load_midx_revindex\",\"key\":\"source\",\"value\":\"rev\"" event.trace
+ '
+}
+
+midx_bitmap_core () {
+ setup_bitmap_history
+
+ test_expect_success 'create single-pack midx with bitmaps' '
+ git repack -ad &&
+ git multi-pack-index write --bitmap &&
+ test_path_is_file $midx &&
+ test_path_is_file $midx-$(midx_checksum $objdir).bitmap
+ '
+
+ test_rev_exists HEAD
+
+ basic_bitmap_tests
+
+ test_expect_success 'create new additional packs' '
+ for i in $(test_seq 1 16)
+ do
+ test_commit "$i" &&
+ git repack -d || return 1
+ done &&
+
+ git checkout -b other2 HEAD~8 &&
+ for i in $(test_seq 1 8)
+ do
+ test_commit "side-$i" &&
+ git repack -d || return 1
+ done &&
+ git checkout second
+ '
+
+ test_expect_success 'create multi-pack midx with bitmaps' '
+ git multi-pack-index write --bitmap &&
+
+ ls $objdir/pack/pack-*.pack >packs &&
+ test_line_count = 25 packs &&
+
+ test_path_is_file $midx &&
+ test_path_is_file $midx-$(midx_checksum $objdir).bitmap
+ '
+
+ test_rev_exists HEAD
+
+ basic_bitmap_tests
+
+ test_expect_success '--no-bitmap is respected when bitmaps exist' '
+ git multi-pack-index write --bitmap &&
+
+ test_commit respect--no-bitmap &&
+ git repack -d &&
+
+ test_path_is_file $midx &&
+ test_path_is_file $midx-$(midx_checksum $objdir).bitmap &&
+
+ git multi-pack-index write --no-bitmap &&
+
+ test_path_is_file $midx &&
+ test_path_is_missing $midx-$(midx_checksum $objdir).bitmap &&
+ test_path_is_missing $midx-$(midx_checksum $objdir).rev
+ '
+
+ test_expect_success 'setup midx with base from later pack' '
+ # Write a and b so that "a" is a delta on top of base "b", since Git
+ # prefers to delete contents out of a base rather than add to a shorter
+ # object.
+ test_seq 1 128 >a &&
+ test_seq 1 130 >b &&
+
+ git add a b &&
+ git commit -m "initial commit" &&
+
+ a=$(git rev-parse HEAD:a) &&
+ b=$(git rev-parse HEAD:b) &&
+
+ # In the first pack, "a" is stored as a delta to "b".
+ p1=$(git pack-objects .git/objects/pack/pack <<-EOF
+ $a
+ $b
+ EOF
+ ) &&
+
+ # In the second pack, "a" is missing, and "b" is not a delta nor base to
+ # any other object.
+ p2=$(git pack-objects .git/objects/pack/pack <<-EOF
+ $b
+ $(git rev-parse HEAD)
+ $(git rev-parse HEAD^{tree})
+ EOF
+ ) &&
+
+ git prune-packed &&
+ # Use the second pack as the preferred source, so that "b" occurs
+ # earlier in the MIDX object order, rendering "a" unusable for pack
+ # reuse.
+ git multi-pack-index write --bitmap --preferred-pack=pack-$p2.idx &&
+
+ have_delta $a $b &&
+ test $(midx_pack_source $a) != $(midx_pack_source $b)
+ '
+
+ rev_list_tests 'full bitmap with backwards delta'
+
+ test_expect_success 'clone with bitmaps enabled' '
+ git clone --no-local --bare . clone-reverse-delta.git &&
+ test_when_finished "rm -fr clone-reverse-delta.git" &&
+
+ git rev-parse HEAD >expect &&
+ git --git-dir=clone-reverse-delta.git rev-parse HEAD >actual &&
+ test_cmp expect actual
+ '
+
+ test_expect_success 'changing the preferred pack does not corrupt bitmaps' '
+ rm -fr repo &&
+ git init repo &&
+ test_when_finished "rm -fr repo" &&
+ (
+ cd repo &&
+
+ test_commit A &&
+ test_commit B &&
+
+ git rev-list --objects --no-object-names HEAD^ >A.objects &&
+ git rev-list --objects --no-object-names HEAD^.. >B.objects &&
+
+ A=$(git pack-objects $objdir/pack/pack <A.objects) &&
+ B=$(git pack-objects $objdir/pack/pack <B.objects) &&
+
+ cat >indexes <<-EOF &&
+ pack-$A.idx
+ pack-$B.idx
+ EOF
+
+ git multi-pack-index write --bitmap --stdin-packs \
+ --preferred-pack=pack-$A.pack <indexes &&
+ git rev-list --test-bitmap A &&
+
+ git multi-pack-index write --bitmap --stdin-packs \
+ --preferred-pack=pack-$B.pack <indexes &&
+ git rev-list --test-bitmap A
+ )
+ '
+}
+
+midx_bitmap_partial_tests () {
+ test_expect_success 'setup partial bitmaps' '
+ test_commit packed &&
+ git repack &&
+ test_commit loose &&
+ git multi-pack-index write --bitmap 2>err &&
+ test_path_is_file $midx &&
+ test_path_is_file $midx-$(midx_checksum $objdir).bitmap
+ '
+
+ test_rev_exists HEAD~
+
+ basic_bitmap_tests HEAD~
+}
diff --git a/t/t5326-multi-pack-bitmaps.sh b/t/t5326-multi-pack-bitmaps.sh
index 85a91c2675..100ac90d15 100755
--- a/t/t5326-multi-pack-bitmaps.sh
+++ b/t/t5326-multi-pack-bitmaps.sh
@@ -9,135 +9,7 @@ test_description='exercise basic multi-pack bitmap functionality'
GIT_TEST_MULTI_PACK_INDEX=0
GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=0
-objdir=.git/objects
-midx=$objdir/pack/multi-pack-index
-
-# midx_pack_source <obj>
-midx_pack_source () {
- test-tool read-midx --show-objects .git/objects | grep "^$1 " | cut -f2
-}
-
-test_rev_exists () {
- commit="$1"
-
- test_expect_success 'reverse index exists' '
- GIT_TRACE2_EVENT_NESTING=10 \
- GIT_TRACE2_EVENT=$(pwd)/event.trace \
- git rev-list --test-bitmap "$commit" &&
-
- test_path_is_file $midx-$(midx_checksum $objdir).rev &&
- grep "\"category\":\"load_midx_revindex\",\"key\":\"source\",\"value\":\"rev\"" event.trace
- '
-}
-
-setup_bitmap_history
-
-test_expect_success 'create single-pack midx with bitmaps' '
- git repack -ad &&
- git multi-pack-index write --bitmap &&
- test_path_is_file $midx &&
- test_path_is_file $midx-$(midx_checksum $objdir).bitmap
-'
-
-test_rev_exists HEAD
-
-basic_bitmap_tests
-
-test_expect_success 'create new additional packs' '
- for i in $(test_seq 1 16)
- do
- test_commit "$i" &&
- git repack -d || return 1
- done &&
-
- git checkout -b other2 HEAD~8 &&
- for i in $(test_seq 1 8)
- do
- test_commit "side-$i" &&
- git repack -d || return 1
- done &&
- git checkout second
-'
-
-test_expect_success 'create multi-pack midx with bitmaps' '
- git multi-pack-index write --bitmap &&
-
- ls $objdir/pack/pack-*.pack >packs &&
- test_line_count = 25 packs &&
-
- test_path_is_file $midx &&
- test_path_is_file $midx-$(midx_checksum $objdir).bitmap
-'
-
-test_rev_exists HEAD
-
-basic_bitmap_tests
-
-test_expect_success '--no-bitmap is respected when bitmaps exist' '
- git multi-pack-index write --bitmap &&
-
- test_commit respect--no-bitmap &&
- git repack -d &&
-
- test_path_is_file $midx &&
- test_path_is_file $midx-$(midx_checksum $objdir).bitmap &&
-
- git multi-pack-index write --no-bitmap &&
-
- test_path_is_file $midx &&
- test_path_is_missing $midx-$(midx_checksum $objdir).bitmap &&
- test_path_is_missing $midx-$(midx_checksum $objdir).rev
-'
-
-test_expect_success 'setup midx with base from later pack' '
- # Write a and b so that "a" is a delta on top of base "b", since Git
- # prefers to delete contents out of a base rather than add to a shorter
- # object.
- test_seq 1 128 >a &&
- test_seq 1 130 >b &&
-
- git add a b &&
- git commit -m "initial commit" &&
-
- a=$(git rev-parse HEAD:a) &&
- b=$(git rev-parse HEAD:b) &&
-
- # In the first pack, "a" is stored as a delta to "b".
- p1=$(git pack-objects .git/objects/pack/pack <<-EOF
- $a
- $b
- EOF
- ) &&
-
- # In the second pack, "a" is missing, and "b" is not a delta nor base to
- # any other object.
- p2=$(git pack-objects .git/objects/pack/pack <<-EOF
- $b
- $(git rev-parse HEAD)
- $(git rev-parse HEAD^{tree})
- EOF
- ) &&
-
- git prune-packed &&
- # Use the second pack as the preferred source, so that "b" occurs
- # earlier in the MIDX object order, rendering "a" unusable for pack
- # reuse.
- git multi-pack-index write --bitmap --preferred-pack=pack-$p2.idx &&
-
- have_delta $a $b &&
- test $(midx_pack_source $a) != $(midx_pack_source $b)
-'
-
-rev_list_tests 'full bitmap with backwards delta'
-
-test_expect_success 'clone with bitmaps enabled' '
- git clone --no-local --bare . clone-reverse-delta.git &&
- test_when_finished "rm -fr clone-reverse-delta.git" &&
-
- git rev-parse HEAD >expect &&
- git --git-dir=clone-reverse-delta.git rev-parse HEAD >actual &&
- test_cmp expect actual
-'
+midx_bitmap_core
bitmap_reuse_tests() {
from=$1
@@ -214,18 +86,7 @@ test_expect_success 'missing object closure fails gracefully' '
)
'
-test_expect_success 'setup partial bitmaps' '
- test_commit packed &&
- git repack &&
- test_commit loose &&
- git multi-pack-index write --bitmap 2>err &&
- test_path_is_file $midx &&
- test_path_is_file $midx-$(midx_checksum $objdir).bitmap
-'
-
-test_rev_exists HEAD~
-
-basic_bitmap_tests HEAD~
+midx_bitmap_partial_tests
test_expect_success 'removing a MIDX clears stale bitmaps' '
rm -fr repo &&
@@ -399,35 +260,4 @@ test_expect_success 'hash-cache values are propagated from pack bitmaps' '
)
'
-test_expect_success 'changing the preferred pack does not corrupt bitmaps' '
- rm -fr repo &&
- git init repo &&
- test_when_finished "rm -fr repo" &&
- (
- cd repo &&
-
- test_commit A &&
- test_commit B &&
-
- git rev-list --objects --no-object-names HEAD^ >A.objects &&
- git rev-list --objects --no-object-names HEAD^.. >B.objects &&
-
- A=$(git pack-objects $objdir/pack/pack <A.objects) &&
- B=$(git pack-objects $objdir/pack/pack <B.objects) &&
-
- cat >indexes <<-EOF &&
- pack-$A.idx
- pack-$B.idx
- EOF
-
- git multi-pack-index write --bitmap --stdin-packs \
- --preferred-pack=pack-$A.pack <indexes &&
- git rev-list --test-bitmap A &&
-
- git multi-pack-index write --bitmap --stdin-packs \
- --preferred-pack=pack-$B.pack <indexes &&
- git rev-list --test-bitmap A
- )
-'
-
test_done
--
2.34.1.25.gb3157a20e6
next prev parent reply other threads:[~2021-12-14 1:55 UTC|newest]
Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-08 19:26 [PATCH 0/2] midx: prevent bitmap corruption when permuting pack order Taylor Blau
2021-12-08 19:26 ` [PATCH 1/2] t5326: demonstrate bitmap corruption after permutation Taylor Blau
2021-12-08 19:26 ` [PATCH 2/2] midx.c: make changing the preferred pack safe Taylor Blau
2021-12-08 19:30 ` [PATCH 0/2] midx: prevent bitmap corruption when permuting pack order Derrick Stolee
2021-12-08 19:55 ` Jeff King
2021-12-10 18:36 ` Taylor Blau
2021-12-10 22:31 ` Taylor Blau
2021-12-11 1:39 ` Taylor Blau
2021-12-13 14:00 ` Derrick Stolee
2021-12-13 14:31 ` Taylor Blau
2021-12-14 1:55 ` [PATCH v2 0/8] " Taylor Blau
2021-12-14 1:55 ` [PATCH v2 1/8] t5326: demonstrate bitmap corruption after permutation Taylor Blau
2021-12-14 1:55 ` [PATCH v2 2/8] midx.c: make changing the preferred pack safe Taylor Blau
2021-12-14 1:55 ` [PATCH v2 3/8] pack-revindex.c: instrument loading on-disk reverse index Taylor Blau
2021-12-14 1:55 ` [PATCH v2 4/8] t5326: drop unnecessary setup Taylor Blau
2021-12-14 1:55 ` [PATCH v2 5/8] t5326: extract `test_rev_exists` Taylor Blau
2021-12-20 18:33 ` Derrick Stolee
2022-01-04 15:33 ` Taylor Blau
2021-12-14 1:55 ` Taylor Blau [this message]
2021-12-14 1:55 ` [PATCH v2 7/8] t/lib-bitmap.sh: parameterize tests over reverse index source Taylor Blau
2021-12-14 1:55 ` [PATCH v2 8/8] midx: read `RIDX` chunk when present Taylor Blau
2021-12-20 18:42 ` Derrick Stolee
2022-01-04 15:21 ` Taylor Blau
2021-12-15 19:46 ` [PATCH v2 0/8] midx: prevent bitmap corruption when permuting pack order Junio C Hamano
2021-12-15 21:37 ` Taylor Blau
2021-12-15 22:17 ` Junio C Hamano
2021-12-15 22:55 ` Junio C Hamano
2021-12-20 18:51 ` Derrick Stolee
2021-12-20 19:52 ` Taylor Blau
2021-12-20 20:09 ` Derrick Stolee
2021-12-15 22:58 ` Junio C Hamano
2021-12-15 23:01 ` Taylor Blau
2022-01-04 18:15 ` [PATCH v3 0/9] " Taylor Blau
2022-01-04 18:15 ` [PATCH v3 1/9] t5326: demonstrate bitmap corruption after permutation Taylor Blau
2022-01-20 17:55 ` Jonathan Tan
2022-01-20 22:11 ` Taylor Blau
2022-01-20 22:41 ` Junio C Hamano
2022-01-20 22:46 ` Taylor Blau
2022-01-24 17:40 ` Jonathan Tan
2022-01-04 18:15 ` [PATCH v3 2/9] midx.c: make changing the preferred pack safe Taylor Blau
2022-01-14 21:35 ` Junio C Hamano
2022-01-14 21:43 ` Junio C Hamano
2022-01-15 0:59 ` Taylor Blau
2022-01-15 6:27 ` Junio C Hamano
2022-01-20 18:08 ` Jonathan Tan
2022-01-20 22:13 ` Taylor Blau
2022-01-04 18:15 ` [PATCH v3 3/9] pack-revindex.c: instrument loading on-disk reverse index Taylor Blau
2022-01-20 18:15 ` Jonathan Tan
2022-01-20 22:18 ` Taylor Blau
2022-01-24 17:53 ` Jonathan Tan
2022-01-04 18:15 ` [PATCH v3 4/9] t5326: drop unnecessary setup Taylor Blau
2022-01-04 18:15 ` [PATCH v3 5/9] t5326: extract `test_rev_exists` Taylor Blau
2022-01-04 18:15 ` [PATCH v3 6/9] t5326: move tests to t/lib-bitmap.sh Taylor Blau
2022-01-04 18:15 ` [PATCH v3 7/9] t/lib-bitmap.sh: parameterize tests over reverse index source Taylor Blau
2022-01-24 19:15 ` Jonathan Tan
2022-01-25 21:40 ` Taylor Blau
2022-01-26 21:00 ` Jonathan Tan
2022-01-04 18:16 ` [PATCH v3 8/9] midx: read `RIDX` chunk when present Taylor Blau
2022-01-24 19:27 ` Jonathan Tan
2022-01-25 21:45 ` Taylor Blau
2022-01-26 21:28 ` Jonathan Tan
2022-01-04 18:16 ` [PATCH v3 9/9] pack-bitmap.c: gracefully fallback after opening pack/MIDX Taylor Blau
2022-01-24 19:29 ` Jonathan Tan
2022-01-25 21:46 ` Taylor Blau
2022-01-25 22:40 ` [PATCH v4 0/9] midx: prevent bitmap corruption when permuting pack order Taylor Blau
2022-01-25 22:41 ` [PATCH v4 1/9] t5326: demonstrate bitmap corruption after permutation Taylor Blau
2022-01-26 15:01 ` Ævar Arnfjörð Bjarmason
2022-01-26 20:18 ` Taylor Blau
2022-01-25 22:41 ` [PATCH v4 2/9] midx.c: make changing the preferred pack safe Taylor Blau
2022-01-25 22:41 ` [PATCH v4 3/9] pack-revindex.c: instrument loading on-disk reverse index Taylor Blau
2022-01-26 15:03 ` Ævar Arnfjörð Bjarmason
2022-01-25 22:41 ` [PATCH v4 4/9] t5326: drop unnecessary setup Taylor Blau
2022-01-25 22:41 ` [PATCH v4 5/9] t5326: extract `test_rev_exists` Taylor Blau
2022-01-26 15:04 ` Ævar Arnfjörð Bjarmason
2022-01-26 20:19 ` Taylor Blau
2022-01-25 22:41 ` [PATCH v4 6/9] t5326: move tests to t/lib-bitmap.sh Taylor Blau
2022-01-25 22:41 ` [PATCH v4 7/9] t/lib-bitmap.sh: parameterize tests over reverse index source Taylor Blau
2022-01-25 22:41 ` [PATCH v4 8/9] midx: read `RIDX` chunk when present Taylor Blau
2022-01-26 15:10 ` Ævar Arnfjörð Bjarmason
2022-01-26 20:23 ` Taylor Blau
2022-01-25 22:41 ` [PATCH v4 9/9] pack-bitmap.c: gracefully fallback after opening pack/MIDX Taylor Blau
2022-01-26 15:08 ` Ævar Arnfjörð Bjarmason
2022-01-26 17:50 ` [PATCH v4 0/9] midx: prevent bitmap corruption when permuting pack order Ævar Arnfjörð Bjarmason
2022-01-26 20:24 ` Taylor Blau
2022-01-27 17:15 ` Jonathan Tan
2022-02-24 22:50 ` Taylor Blau
2022-01-27 14:13 ` Derrick Stolee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bf42b116e1872fd69164fe15f098b7a72e938c24.1639446906.git.me@ttaylorr.com \
--to=me@ttaylorr.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=stolee@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).