From: Taylor Blau <me@ttaylorr.com>
To: git@vger.kernel.org
Cc: dstolee@microsoft.com, peff@peff.net, sluongng@gmail.com
Subject: [PATCH v2] builtin/repack.c: invalidate MIDX only when necessary
Date: Tue, 25 Aug 2020 12:04:36 -0400 [thread overview]
Message-ID: <87a3b7a5a2f091e2a23c163a7d86effbbbedfa3a.1598371475.git.me@ttaylorr.com> (raw)
In-Reply-To: <20200825144515.GB7671@syl.lan>
In 525e18c04b (midx: clear midx on repack, 2018-07-12), 'git repack'
learned to remove a multi-pack-index file if it added or removed a pack
from the object store.
This mechanism is a little over-eager, since it is only necessary to
drop a MIDX if 'git repack' removes a pack that the MIDX references.
Adding a pack outside of the MIDX does not require invalidating the
MIDX, and likewise for removing a pack the MIDX does not know about.
Teach 'git repack' to check for this by loading the MIDX, and checking
whether the to-be-removed pack is known to the MIDX. This requires a
slightly odd alternation to a test in t5319, which is explained with a
comment. A new test is added to show that the MIDX is left alone when
both packs known to it are marked as .keep, but two packs unknown to it
are removed and combined into one new pack.
Helped-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
Range-diff against v1:
1: ef9186a8df ! 1: 87a3b7a5a2 builtin/repack.c: invalidate MIDX only when necessary
@@ Commit message
Teach 'git repack' to check for this by loading the MIDX, and checking
whether the to-be-removed pack is known to the MIDX. This requires a
slightly odd alternation to a test in t5319, which is explained with a
- comment.
+ comment. A new test is added to show that the MIDX is left alone when
+ both packs known to it are marked as .keep, but two packs unknown to it
+ are removed and combined into one new pack.
+ Helped-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
## builtin/repack.c ##
@@ t/t5319-multi-pack-index.sh: test_expect_success 'repack with the --no-progress
test_path_is_missing $objdir/pack/multi-pack-index
'
-+test_expect_success 'repack preserves multi-pack-index when deleting unknown packs' '
-+ git multi-pack-index write &&
-+ cp $objdir/pack/multi-pack-index $objdir/pack/multi-pack-index.bak &&
-+ test_when_finished "rm -f $objdir/pack/multi-pack-index.bak" &&
-+
-+ # Write a new pack that is unknown to the multi-pack-index.
-+ git hash-object -w </dev/null >blob &&
-+ git pack-objects $objdir/pack/pack <blob &&
-+
-+ GIT_TEST_MULTI_PACK_INDEX=0 git -c core.multiPackIndex repack -d &&
-+ test_cmp_bin $objdir/pack/multi-pack-index \
-+ $objdir/pack/multi-pack-index.bak
++test_expect_success 'repack preserves multi-pack-index when creating packs' '
++ git init preserve &&
++ test_when_finished "rm -fr preserve" &&
++ (
++ cd preserve &&
++ packdir=.git/objects/pack &&
++ midx=$packdir/multi-pack-index &&
++
++ test_commit 1 &&
++ pack1=$(git pack-objects --all $packdir/pack) &&
++ touch $packdir/pack-$pack1.keep &&
++ test_commit 2 &&
++ pack2=$(git pack-objects --revs $packdir/pack) &&
++ touch $packdir/pack-$pack2.keep &&
++
++ git multi-pack-index write &&
++ cp $midx $midx.bak &&
++
++ cat >pack-input <<-EOF &&
++ HEAD
++ ^HEAD~1
++ EOF
++ test_commit 3 &&
++ pack3=$(git pack-objects --revs $packdir/pack <pack-input) &&
++ test_commit 4 &&
++ pack4=$(git pack-objects --revs $packdir/pack <pack-input) &&
++
++ GIT_TEST_MULTI_PACK_INDEX=0 git -c core.multiPackIndex repack -ad &&
++ ls -la $packdir &&
++ test_path_is_file $packdir/pack-$pack1.pack &&
++ test_path_is_file $packdir/pack-$pack2.pack &&
++ test_path_is_missing $packdir/pack-$pack3.pack &&
++ test_path_is_missing $packdir/pack-$pack4.pack &&
++ test_cmp_bin $midx.bak $midx
++ )
+'
+
compare_results_with_midx "after repack"
builtin/repack.c | 12 +++++-----
t/t5319-multi-pack-index.sh | 44 +++++++++++++++++++++++++++++++++++--
2 files changed, 47 insertions(+), 9 deletions(-)
diff --git a/builtin/repack.c b/builtin/repack.c
index 04c5ceaf7e..98fac03946 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -133,7 +133,11 @@ static void get_non_kept_pack_filenames(struct string_list *fname_list,
static void remove_redundant_pack(const char *dir_name, const char *base_name)
{
struct strbuf buf = STRBUF_INIT;
- strbuf_addf(&buf, "%s/%s.pack", dir_name, base_name);
+ struct multi_pack_index *m = get_multi_pack_index(the_repository);
+ strbuf_addf(&buf, "%s.pack", base_name);
+ if (m && midx_contains_pack(m, buf.buf))
+ clear_midx_file(the_repository);
+ strbuf_insertf(&buf, 0, "%s/", dir_name);
unlink_pack_path(buf.buf, 1);
strbuf_release(&buf);
}
@@ -286,7 +290,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
int keep_unreachable = 0;
struct string_list keep_pack_list = STRING_LIST_INIT_NODUP;
int no_update_server_info = 0;
- int midx_cleared = 0;
struct pack_objects_args po_args = {NULL};
struct option builtin_repack_options[] = {
@@ -439,11 +442,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
for (ext = 0; ext < ARRAY_SIZE(exts); ext++) {
char *fname, *fname_old;
- if (!midx_cleared) {
- clear_midx_file(the_repository);
- midx_cleared = 1;
- }
-
fname = mkpathdup("%s/pack-%s%s", packdir,
item->string, exts[ext].name);
if (!file_exists(fname)) {
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 43b1b5b2af..f340b376bc 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -382,12 +382,52 @@ test_expect_success 'repack with the --no-progress option' '
test_line_count = 0 err
'
-test_expect_success 'repack removes multi-pack-index' '
+test_expect_success 'repack removes multi-pack-index when deleting packs' '
test_path_is_file $objdir/pack/multi-pack-index &&
- GIT_TEST_MULTI_PACK_INDEX=0 git repack -adf &&
+ # Set GIT_TEST_MULTI_PACK_INDEX to 0 to avoid writing a new
+ # multi-pack-index after repacking, but set "core.multiPackIndex" to
+ # true so that "git repack" can read the existing MIDX.
+ GIT_TEST_MULTI_PACK_INDEX=0 git -c core.multiPackIndex repack -adf &&
test_path_is_missing $objdir/pack/multi-pack-index
'
+test_expect_success 'repack preserves multi-pack-index when creating packs' '
+ git init preserve &&
+ test_when_finished "rm -fr preserve" &&
+ (
+ cd preserve &&
+ packdir=.git/objects/pack &&
+ midx=$packdir/multi-pack-index &&
+
+ test_commit 1 &&
+ pack1=$(git pack-objects --all $packdir/pack) &&
+ touch $packdir/pack-$pack1.keep &&
+ test_commit 2 &&
+ pack2=$(git pack-objects --revs $packdir/pack) &&
+ touch $packdir/pack-$pack2.keep &&
+
+ git multi-pack-index write &&
+ cp $midx $midx.bak &&
+
+ cat >pack-input <<-EOF &&
+ HEAD
+ ^HEAD~1
+ EOF
+ test_commit 3 &&
+ pack3=$(git pack-objects --revs $packdir/pack <pack-input) &&
+ test_commit 4 &&
+ pack4=$(git pack-objects --revs $packdir/pack <pack-input) &&
+
+ GIT_TEST_MULTI_PACK_INDEX=0 git -c core.multiPackIndex repack -ad &&
+ ls -la $packdir &&
+ test_path_is_file $packdir/pack-$pack1.pack &&
+ test_path_is_file $packdir/pack-$pack2.pack &&
+ test_path_is_missing $packdir/pack-$pack3.pack &&
+ test_path_is_missing $packdir/pack-$pack4.pack &&
+ test_cmp_bin $midx.bak $midx
+ )
+'
+
compare_results_with_midx "after repack"
test_expect_success 'multi-pack-index and pack-bitmap' '
--
2.28.0.338.g87a3b7a5a2
next prev parent reply other threads:[~2020-08-25 16:04 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-25 2:01 [PATCH] builtin/repack.c: invalidate MIDX only when necessary Taylor Blau
2020-08-25 2:26 ` Jeff King
2020-08-25 2:37 ` Taylor Blau
2020-08-25 13:14 ` Derrick Stolee
2020-08-25 14:41 ` Taylor Blau
2020-08-25 15:14 ` Derrick Stolee
2020-08-25 15:42 ` Taylor Blau
2020-08-25 16:56 ` Jeff King
2020-08-25 15:58 ` Junio C Hamano
2020-08-25 16:08 ` Taylor Blau
2020-08-25 16:18 ` Derrick Stolee
2020-08-25 17:34 ` Jeff King
2020-08-25 17:22 ` Jeff King
2020-08-25 18:05 ` Junio C Hamano
2020-08-25 18:27 ` Jeff King
2020-08-25 22:45 ` [PATCH] pack-redundant: gauge the usage before proposing its removal Junio C Hamano
2020-08-25 23:09 ` Taylor Blau
2020-08-25 23:22 ` Junio C Hamano
2020-08-26 1:17 ` [PATCH v1 0/3] War on dashed-git Junio C Hamano
2020-08-26 1:17 ` [PATCH v1 1/3] transport-helper: do not run git-remote-ext etc. in dashed form Junio C Hamano
2020-08-26 1:24 ` Eric Sunshine
2020-08-26 7:55 ` Johannes Schindelin
2020-08-26 16:27 ` Junio C Hamano
2020-08-26 1:17 ` [PATCH v1 2/3] cvsexportcommit: do not run git programs " Junio C Hamano
2020-08-26 1:28 ` Eric Sunshine
2020-08-26 1:42 ` Junio C Hamano
2020-08-26 16:08 ` Junio C Hamano
2020-08-26 16:28 ` Junio C Hamano
2020-08-26 8:02 ` Johannes Schindelin
2020-08-26 1:17 ` [PATCH v1 3/3] git: catch an attempt to run "git-foo" Junio C Hamano
2020-08-26 1:19 ` Junio C Hamano
2020-08-26 8:06 ` Johannes Schindelin
2020-08-26 16:30 ` Junio C Hamano
2020-08-28 2:13 ` Johannes Schindelin
2020-08-28 22:03 ` Junio C Hamano
2020-08-31 9:59 ` Johannes Schindelin
2020-08-31 17:45 ` Junio C Hamano
2020-12-20 15:25 ` Johannes Schindelin
2020-12-21 22:24 ` Junio C Hamano
2020-12-30 5:30 ` Johannes Schindelin
2020-08-26 8:09 ` [PATCH v1 0/3] War on dashed-git Johannes Schindelin
2020-08-26 16:45 ` Junio C Hamano
2020-08-26 19:46 ` [PATCH v2 0/2] avoid running "git-subcmd" in the dashed form Junio C Hamano
2020-08-26 19:46 ` [PATCH v2 1/2] transport-helper: do not run git-remote-ext etc. in " Junio C Hamano
2020-08-26 19:46 ` [PATCH v2 2/2] cvsexportcommit: do not run git programs " Junio C Hamano
2020-08-26 21:37 ` [PATCH v2 3/2] credential-cache: use child_process.args Junio C Hamano
2020-08-26 22:25 ` [PATCH] run_command: teach API users to use embedded 'args' more Junio C Hamano
2020-08-27 4:21 ` Jeff King
2020-08-27 4:30 ` Junio C Hamano
2020-08-27 4:31 ` Eric Sunshine
2020-08-27 4:44 ` Jeff King
2020-08-27 5:03 ` Eric Sunshine
2020-08-27 5:25 ` [PATCH] worktree: fix leak in check_clean_worktree() Jeff King
2020-08-27 5:56 ` Eric Sunshine
2020-08-27 15:31 ` Junio C Hamano
2020-08-27 4:13 ` [PATCH v2 3/2] credential-cache: use child_process.args Jeff King
2020-08-27 4:22 ` Jeff King
2020-08-27 4:31 ` Junio C Hamano
2020-08-27 4:14 ` Jeff King
2020-08-27 15:34 ` Junio C Hamano
2020-08-31 22:56 ` Junio C Hamano
2020-09-01 4:49 ` Jeff King
2020-09-01 16:11 ` Junio C Hamano
2020-08-27 0:57 ` [PATCH v2 0/2] avoid running "git-subcmd" in the dashed form Derrick Stolee
2020-08-27 1:22 ` Junio C Hamano
2020-08-28 9:14 ` [PATCH] pack-redundant: gauge the usage before proposing its removal Jeff King
2020-08-28 22:45 ` Junio C Hamano
2020-08-25 7:55 ` [PATCH] builtin/repack.c: invalidate MIDX only when necessary Son Luong Ngoc
2020-08-25 12:45 ` Derrick Stolee
2020-08-25 14:45 ` Taylor Blau
2020-08-25 16:04 ` Taylor Blau [this message]
2020-08-26 20:51 ` [PATCH v2] " Derrick Stolee
2020-08-26 20:54 ` Junio C Hamano
2020-08-25 16:47 ` [PATCH] " Jeff King
2020-08-25 17:10 ` Derrick Stolee
2020-08-25 17:29 ` Jeff King
2020-08-25 17:34 ` Taylor Blau
2020-08-25 17:42 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87a3b7a5a2f091e2a23c163a7d86effbbbedfa3a.1598371475.git.me@ttaylorr.com \
--to=me@ttaylorr.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=peff@peff.net \
--cc=sluongng@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).