From: Kousik Sanagavarapu <five231003@gmail.com>
To: git@vger.kernel.org
Cc: jonathanmy@google.com, gitster@pobox.com, fve231003@gmail.com,
Kousik Sanagavarapu <five231003@gmail.com>,
Jonathan Tan <jonathantanmy@google.com>
Subject: [PATCH v3] index-pack: remove fetch_if_missing=0
Date: Mon, 13 Mar 2023 23:45:18 +0530 [thread overview]
Message-ID: <20230313181518.6322-1-five231003@gmail.com> (raw)
In-Reply-To: <20230310183029.19429-1-five231003@gmail.com>
A collision test is triggered in sha1_object(), whenever there is an
object file in our repo. If our repo is a partial clone, then checking
for this file existence does not lazy-fetch the object (if the object
is missing and if there are one or more promisor remotes) when
fetch_if_missing is set to 0.
This global was added as a temporary measure to suppress the fetching
of missing objects [1] and can be removed once the remaining commands:
- fetch-pack
- fsck
- pack-objects
- prune
- rev-list
can handle lazy-fetching without fetch_if_missing.
Hence, use has_object() to check for the existence of an object, which
has the default behavior of not lazy-fetching in a partial clone. It is
worth mentioning that this is the only place where there is potential for
lazy-fetching and all other cases [2] are properly handled, making it safe
to remove this global here.
[1] See 8b4c0103a9 (sha1_file: support lazily fetching missing objects,
2017-12-08)
[2] These cases are:
- When we check objects, but we return with 0 early if the object
doesn't exist.
- We prefetch delta bases in a partial clone, if we don't have them
(as the comment outlines).
- There are some cases where we fsck objects, but lazy-fetching is
already handled in fsck.
Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Kousik Sanagavarapu <five231003@gmail.com>
---
builtin/index-pack.c | 11 +----------
t/t5616-partial-clone.sh | 28 ++++++++++++++++++++++++++++
2 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 6648f2daef..8c0f36a49e 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -800,8 +800,7 @@ static void sha1_object(const void *data, struct object_entry *obj_entry,
if (startup_info->have_repository) {
read_lock();
- collision_test_needed =
- has_object_file_with_flags(oid, OBJECT_INFO_QUICK);
+ collision_test_needed = has_object(the_repository, oid, 0);
read_unlock();
}
@@ -1728,14 +1727,6 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
int report_end_of_input = 0;
int hash_algo = 0;
- /*
- * index-pack never needs to fetch missing objects except when
- * REF_DELTA bases are missing (which are explicitly handled). It only
- * accesses the repo to do hash collision checks and to check which
- * REF_DELTA bases need to be fetched.
- */
- fetch_if_missing = 0;
-
if (argc == 2 && !strcmp(argv[1], "-h"))
usage(index_pack_usage);
diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index f519d2a87a..fdb34a0b50 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -644,6 +644,34 @@ test_expect_success 'repack does not loosen promisor objects' '
grep "loosen_unused_packed_objects/loosened:0" trace
'
+test_expect_success 'index-pack does not lazy-fetch when checking for sha1 collsions' '
+ rm -rf server client another-remote &&
+
+ git init server &&
+ echo "line" >server/file &&
+ git -C server add file &&
+ git -C server commit -am "file" &&
+ git -C server config --local uploadpack.allowFilter 1 &&
+ git -C server config --local uploadpack.allowAnySha1InWant 1 &&
+
+ git clone --no-checkout --filter=blob:none "file://$(pwd)/server" client &&
+ git -C client config extensions.partialClone 1 &&
+ git -C client config remote.origin.promisor 1 &&
+
+ git clone "file://$(pwd)/server" another-remote &&
+
+ echo "new line" >server/new-file &&
+ git -C server add new-file &&
+ git -C server commit -am "new-file" &&
+
+ git -C another-remote pull &&
+
+ # Try to fetch so that "client" will have to do a collision-check.
+ # This should, however, not fetch "new-file" because "client" is a
+ # partial clone.
+ git -C client fetch "file://$(pwd)/another-remote" main
+'
+
test_expect_success 'lazy-fetch in submodule succeeds' '
# setup
test_config_global protocol.file.allow always &&
--
2.25.1
next prev parent reply other threads:[~2023-03-13 18:15 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-25 5:24 [PATCH] index-pack: remove fetch_if_missing=0 Kousik Sanagavarapu
2023-02-27 16:56 ` Kousik Sanagavarapu
2023-02-27 22:14 ` Jonathan Tan
2023-02-28 3:54 ` Kousik Sanagavarapu
2023-03-10 18:30 ` [PATCH v2] " Kousik Sanagavarapu
2023-03-10 20:30 ` Junio C Hamano
2023-03-10 21:13 ` Jonathan Tan
2023-03-10 21:41 ` Junio C Hamano
2023-03-11 2:59 ` Jonathan Tan
2023-03-12 17:16 ` Kousik Sanagavarapu
2023-03-11 6:22 ` [PATCH] " Kousik Sanagavarapu
2023-03-11 6:00 ` Kousik Sanagavarapu
2023-03-13 18:15 ` Kousik Sanagavarapu [this message]
2023-03-13 19:17 ` [PATCH v3] " Junio C Hamano
2023-03-13 19:18 ` Junio C Hamano
2023-03-17 17:56 ` [PATCH v4] " Kousik Sanagavarapu
2023-03-17 22:58 ` Junio C Hamano
2023-03-19 6:17 ` Kousik Sanagavarapu
2023-03-11 20:01 ` [PATCH] " Sean Allred
2023-03-11 20:37 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230313181518.6322-1-five231003@gmail.com \
--to=five231003@gmail.com \
--cc=fve231003@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jonathanmy@google.com \
--cc=jonathantanmy@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).