git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, me@ttaylorr.com, newren@gmail.com,
	avarab@gmail.com, mjcheetham@outlook.com, steadmon@google.com,
	Glen Choo <chooglen@google.com>,
	Jonathan Tan <jonathantanmy@google.com>,
	Teng Long <dyroneteng@gmail.com>,
	Derrick Stolee <derrickstolee@github.com>,
	Derrick Stolee <derrickstolee@github.com>
Subject: [PATCH v4 08/11] bundle: add flags to verify_bundle(), skip walk
Date: Mon, 10 Oct 2022 16:04:16 +0000	[thread overview]
Message-ID: <83f2cd893a4c47c947a93fe99d202d67f540e963.1665417859.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1333.v4.git.1665417859.gitgitgadget@gmail.com>

From: Derrick Stolee <derrickstolee@github.com>

The verify_bundle() method checks if a bundle can be applied to a given
repository. This not only verifies that certain commits exist in the
repository, but Git also checks that these commits are reachable.

This behavior dates back to the original git-bundle builtin written in
2e0afafebd8 (Add git-bundle: move objects and references by archive,
2007-02-22), but the message does not go into detail why the
reachability check is important.

Since verify_bundle() is called from unbundle(), we need to add an
option to pipe the flags through that method.

When unbundling from a list of bundles, Git will create refs that point
to the tips of the latest bundle, which makes this reachability walk
succeed, in theory. However, the loose refs cache does not get
invalidated and hence the reachability walk fails. By disabling the
reachability walk in the bundle URI code, we can get around this
reachability check.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 builtin/bundle.c |  5 +++--
 bundle-uri.c     |  8 +++++++-
 bundle.c         | 12 +++++++-----
 bundle.h         | 15 +++++++++++++--
 transport.c      |  2 +-
 5 files changed, 31 insertions(+), 11 deletions(-)

diff --git a/builtin/bundle.c b/builtin/bundle.c
index 2adad545a2e..7d983a238f0 100644
--- a/builtin/bundle.c
+++ b/builtin/bundle.c
@@ -119,7 +119,8 @@ static int cmd_bundle_verify(int argc, const char **argv, const char *prefix) {
 		goto cleanup;
 	}
 	close(bundle_fd);
-	if (verify_bundle(the_repository, &header, !quiet)) {
+	if (verify_bundle(the_repository, &header,
+			  quiet ? 0 : VERIFY_BUNDLE_VERBOSE)) {
 		ret = 1;
 		goto cleanup;
 	}
@@ -185,7 +186,7 @@ static int cmd_bundle_unbundle(int argc, const char **argv, const char *prefix)
 		strvec_pushl(&extra_index_pack_args, "-v", "--progress-title",
 			     _("Unbundling objects"), NULL);
 	ret = !!unbundle(the_repository, &header, bundle_fd,
-			 &extra_index_pack_args) ||
+			 &extra_index_pack_args, 0) ||
 		list_bundle_refs(&header, argc, argv);
 	bundle_header_release(&header);
 cleanup:
diff --git a/bundle-uri.c b/bundle-uri.c
index 8a7c11c6393..ad5baabdd94 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -301,7 +301,13 @@ static int unbundle_from_file(struct repository *r, const char *file)
 	if ((bundle_fd = read_bundle_header(file, &header)) < 0)
 		return 1;
 
-	if ((result = unbundle(r, &header, bundle_fd, NULL)))
+	/*
+	 * Skip the reachability walk here, since we will be adding
+	 * a reachable ref pointing to the new tips, which will reach
+	 * the prerequisite commits.
+	 */
+	if ((result = unbundle(r, &header, bundle_fd, NULL,
+			       VERIFY_BUNDLE_SKIP_REACHABLE)))
 		return 1;
 
 	/*
diff --git a/bundle.c b/bundle.c
index 0208e6d90d3..36ffeb1e0eb 100644
--- a/bundle.c
+++ b/bundle.c
@@ -189,7 +189,7 @@ static int list_refs(struct string_list *r, int argc, const char **argv)
 
 int verify_bundle(struct repository *r,
 		  struct bundle_header *header,
-		  int verbose)
+		  enum verify_bundle_flags flags)
 {
 	/*
 	 * Do fast check, then if any prereqs are missing then go line by line
@@ -222,7 +222,8 @@ int verify_bundle(struct repository *r,
 			error("%s", message);
 		error("%s %s", oid_to_hex(oid), name);
 	}
-	if (revs.pending.nr != p->nr)
+	if (revs.pending.nr != p->nr ||
+	    (flags & VERIFY_BUNDLE_SKIP_REACHABLE))
 		goto cleanup;
 	req_nr = revs.pending.nr;
 	setup_revisions(2, argv, &revs, NULL);
@@ -259,7 +260,7 @@ int verify_bundle(struct repository *r,
 			clear_commit_marks(commit, ALL_REV_FLAGS);
 	}
 
-	if (verbose) {
+	if (flags & VERIFY_BUNDLE_VERBOSE) {
 		struct string_list *r;
 
 		r = &header->references;
@@ -620,7 +621,8 @@ err:
 }
 
 int unbundle(struct repository *r, struct bundle_header *header,
-	     int bundle_fd, struct strvec *extra_index_pack_args)
+	     int bundle_fd, struct strvec *extra_index_pack_args,
+	     enum verify_bundle_flags flags)
 {
 	struct child_process ip = CHILD_PROCESS_INIT;
 	strvec_pushl(&ip.args, "index-pack", "--fix-thin", "--stdin", NULL);
@@ -634,7 +636,7 @@ int unbundle(struct repository *r, struct bundle_header *header,
 		strvec_clear(extra_index_pack_args);
 	}
 
-	if (verify_bundle(r, header, 0))
+	if (verify_bundle(r, header, flags))
 		return -1;
 	ip.in = bundle_fd;
 	ip.no_stdout = 1;
diff --git a/bundle.h b/bundle.h
index 0c052f54964..9f798c00d93 100644
--- a/bundle.h
+++ b/bundle.h
@@ -29,7 +29,14 @@ int read_bundle_header_fd(int fd, struct bundle_header *header,
 int create_bundle(struct repository *r, const char *path,
 		  int argc, const char **argv, struct strvec *pack_options,
 		  int version);
-int verify_bundle(struct repository *r, struct bundle_header *header, int verbose);
+
+enum verify_bundle_flags {
+	VERIFY_BUNDLE_VERBOSE = (1 << 0),
+	VERIFY_BUNDLE_SKIP_REACHABLE = (1 << 1)
+};
+
+int verify_bundle(struct repository *r, struct bundle_header *header,
+		  enum verify_bundle_flags flags);
 
 /**
  * Unbundle after reading the header with read_bundle_header().
@@ -40,9 +47,13 @@ int verify_bundle(struct repository *r, struct bundle_header *header, int verbos
  * Provide "extra_index_pack_args" to pass any extra arguments
  * (e.g. "-v" for verbose/progress), NULL otherwise. The provided
  * "extra_index_pack_args" (if any) will be strvec_clear()'d for you.
+ *
+ * Before unbundling, this method will call verify_bundle() with the
+ * given 'flags'.
  */
 int unbundle(struct repository *r, struct bundle_header *header,
-	     int bundle_fd, struct strvec *extra_index_pack_args);
+	     int bundle_fd, struct strvec *extra_index_pack_args,
+	     enum verify_bundle_flags flags);
 int list_bundle_refs(struct bundle_header *header,
 		int argc, const char **argv);
 
diff --git a/transport.c b/transport.c
index 52db7a3cb09..c5d3042731a 100644
--- a/transport.c
+++ b/transport.c
@@ -178,7 +178,7 @@ static int fetch_refs_from_bundle(struct transport *transport,
 	if (!data->get_refs_from_bundle_called)
 		get_refs_from_bundle_inner(transport);
 	ret = unbundle(the_repository, &data->header, data->fd,
-		       &extra_index_pack_args);
+		       &extra_index_pack_args, 0);
 	transport->hash_algo = data->header.hash_algo;
 	return ret;
 }
-- 
gitgitgadget


  parent reply	other threads:[~2022-10-10 16:05 UTC|newest]

Thread overview: 94+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-22 15:12 [PATCH 0/7] Bundle URIs III: Parse and download from bundle lists Derrick Stolee via GitGitGadget
2022-08-22 15:12 ` [PATCH 1/7] bundle-uri: create bundle_list struct and helpers Derrick Stolee via GitGitGadget
2022-08-22 17:57   ` Junio C Hamano
2022-08-22 15:12 ` [PATCH 2/7] bundle-uri: create base key-value pair parsing Derrick Stolee via GitGitGadget
2022-08-22 18:20   ` Junio C Hamano
2022-08-23 16:29     ` Derrick Stolee
2022-08-31 22:10       ` Jonathan Tan
2022-08-31 22:02   ` Glen Choo
2022-09-01  2:38   ` [PATCH 4/7] bundle-uri: unit test "key=value" parsing Teng Long
2022-08-22 15:12 ` [PATCH 3/7] bundle-uri: create "key=value" line parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-08-22 19:17   ` Junio C Hamano
2022-08-23 16:31     ` Derrick Stolee
2022-09-02 23:41   ` Josh Steadmon
2022-08-22 15:12 ` [PATCH 4/7] bundle-uri: unit test "key=value" parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-09-01  2:56   ` Teng Long
2022-08-22 15:12 ` [PATCH 5/7] bundle-uri: parse bundle list in config format Derrick Stolee via GitGitGadget
2022-08-22 19:25   ` Junio C Hamano
2022-08-23 16:43     ` Derrick Stolee
2022-08-31 22:18     ` Jonathan Tan
2022-09-01  8:05   ` Teng Long
2022-08-22 15:12 ` [PATCH 6/7] bundle-uri: limit recursion depth for bundle lists Derrick Stolee via GitGitGadget
2022-08-22 15:12 ` [PATCH 7/7] bundle-uri: fetch a list of bundles Derrick Stolee via GitGitGadget
2022-09-02 23:51   ` Josh Steadmon
2022-09-05 12:50   ` Teng Long
2022-09-08 17:10     ` Derrick Stolee
2022-09-09 14:33 ` [PATCH v2 0/9] Bundle URIs III: Parse and download from bundle lists Derrick Stolee via GitGitGadget
2022-09-09 14:33   ` [PATCH v2 1/9] bundle-uri: short-circuit capability parsing Derrick Stolee via GitGitGadget
2022-09-09 17:24     ` Junio C Hamano
2022-09-19 17:55       ` Derrick Stolee
2022-09-09 14:33   ` [PATCH v2 2/9] bundle-uri: use plain string in find_temp_filename() Derrick Stolee via GitGitGadget
2022-09-09 17:56     ` Junio C Hamano
2022-09-19 17:54       ` Derrick Stolee
2022-09-19 18:16         ` Junio C Hamano
2022-09-09 14:33   ` [PATCH v2 3/9] bundle-uri: create bundle_list struct and helpers Derrick Stolee via GitGitGadget
2022-09-09 14:33   ` [PATCH v2 4/9] bundle-uri: create base key-value pair parsing Derrick Stolee via GitGitGadget
2022-09-29 21:49     ` Jonathan Tan
2022-09-09 14:33   ` [PATCH v2 5/9] bundle-uri: create "key=value" line parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-09-09 14:33   ` [PATCH v2 6/9] bundle-uri: unit test "key=value" parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-09-09 14:33   ` [PATCH v2 7/9] bundle-uri: parse bundle list in config format Derrick Stolee via GitGitGadget
2022-09-09 14:33   ` [PATCH v2 8/9] bundle-uri: limit recursion depth for bundle lists Derrick Stolee via GitGitGadget
2022-09-09 14:33   ` [PATCH v2 9/9] bundle-uri: fetch a list of bundles Derrick Stolee via GitGitGadget
2022-09-29 21:58     ` Jonathan Tan
2022-09-30 12:49       ` Derrick Stolee
2022-09-26 13:19   ` [PATCH v2 0/9] Bundle URIs III: Parse and download from bundle lists Derrick Stolee
2022-09-26 19:10     ` Junio C Hamano
2022-09-29 22:00       ` Jonathan Tan
2022-09-30 13:21         ` Derrick Stolee
2022-10-04 12:34   ` [PATCH v3 " Derrick Stolee via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 1/9] bundle-uri: use plain string in find_temp_filename() Derrick Stolee via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 2/9] bundle-uri: create bundle_list struct and helpers Derrick Stolee via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 3/9] bundle-uri: create base key-value pair parsing Derrick Stolee via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 4/9] bundle-uri: create "key=value" line parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 5/9] bundle-uri: unit test "key=value" parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 6/9] bundle-uri: parse bundle list in config format Derrick Stolee via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 7/9] bundle-uri: limit recursion depth for bundle lists Derrick Stolee via GitGitGadget
2022-10-04 12:34     ` [PATCH v3 8/9] bundle-uri: fetch a list of bundles Derrick Stolee via GitGitGadget
2022-10-04 21:44       ` Jonathan Tan
2022-10-07 13:29         ` Derrick Stolee
2022-10-04 12:34     ` [PATCH v3 9/9] bundle-uri: suppress stderr from remote-https Derrick Stolee via GitGitGadget
2022-10-10 16:04     ` [PATCH v4 00/11] Bundle URIs III: Parse and download from bundle lists Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 01/11] bundle-uri: use plain string in find_temp_filename() Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 02/11] bundle-uri: create bundle_list struct and helpers Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 03/11] bundle-uri: create base key-value pair parsing Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 04/11] bundle-uri: create "key=value" line parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 05/11] bundle-uri: unit test "key=value" parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 06/11] bundle-uri: parse bundle list in config format Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 07/11] bundle-uri: limit recursion depth for bundle lists Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` Derrick Stolee via GitGitGadget [this message]
2022-10-10 17:27         ` [PATCH v4 08/11] bundle: add flags to verify_bundle(), skip walk Junio C Hamano
2022-10-10 18:13           ` Derrick Stolee
2022-10-10 18:40             ` Junio C Hamano
2022-10-11 19:04               ` Derrick Stolee
2022-10-10 16:04       ` [PATCH v4 09/11] bundle-uri: fetch a list of bundles Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 10/11] bundle-uri: quiet failed unbundlings Derrick Stolee via GitGitGadget
2022-10-10 16:04       ` [PATCH v4 11/11] bundle-uri: suppress stderr from remote-https Derrick Stolee via GitGitGadget
2022-10-12 12:52       ` [PATCH v5 00/12] Bundle URIs III: Parse and download from bundle lists Derrick Stolee via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 01/12] bundle-uri: use plain string in find_temp_filename() Derrick Stolee via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 02/12] bundle-uri: create bundle_list struct and helpers Derrick Stolee via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 03/12] bundle-uri: create base key-value pair parsing Derrick Stolee via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 04/12] bundle-uri: create "key=value" line parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 05/12] bundle-uri: unit test "key=value" parsing Ævar Arnfjörð Bjarmason via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 06/12] bundle-uri: parse bundle list in config format Derrick Stolee via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 07/12] bundle-uri: limit recursion depth for bundle lists Derrick Stolee via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 08/12] bundle: properly clear all revision flags Derrick Stolee via GitGitGadget
2022-10-12 16:17           ` Junio C Hamano
2022-10-12 12:52         ` [PATCH v5 09/12] bundle-uri: fetch a list of bundles Derrick Stolee via GitGitGadget
2022-10-26 19:06           ` Junio C Hamano
2022-10-12 12:52         ` [PATCH v5 10/12] bundle: add flags to verify_bundle() Derrick Stolee via GitGitGadget
2022-10-12 12:52         ` [PATCH v5 11/12] bundle-uri: quiet failed unbundlings Derrick Stolee via GitGitGadget
2022-10-12 16:32           ` Junio C Hamano
2022-10-12 12:52         ` [PATCH v5 12/12] bundle-uri: suppress stderr from remote-https Derrick Stolee via GitGitGadget
2022-10-26 18:54           ` Junio C Hamano
2022-10-26 14:34         ` [PATCH v5 00/12] Bundle URIs III: Parse and download from bundle lists Derrick Stolee
2022-10-26 16:06           ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=83f2cd893a4c47c947a93fe99d202d67f540e963.1665417859.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=avarab@gmail.com \
    --cc=chooglen@google.com \
    --cc=derrickstolee@github.com \
    --cc=dyroneteng@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@google.com \
    --cc=me@ttaylorr.com \
    --cc=mjcheetham@outlook.com \
    --cc=newren@gmail.com \
    --cc=steadmon@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).