git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Jonathan Tan <jonathantanmy@google.com>
To: git@vger.kernel.org
Cc: Jonathan Tan <jonathantanmy@google.com>
Subject: [RFC PATCH] Modify fetch-pack to no longer die on error?
Date: Fri, 24 Jul 2020 15:38:44 -0700	[thread overview]
Message-ID: <20200724223844.2723397-1-jonathantanmy@google.com> (raw)

We've had a few instances where a lazy fetch in a partial clone fails,
leading to a fatal error, when the calling code could have easily
recovered - in other words, the severity of the bug should have just a
wasted fetch instead of stopping the whole command. Part of the issue
(and possibly the whole issue - I haven't looked at this beyond
fetch-pack yet) is that fetch-pack dies whenever it encounters an
error, so I took a look at fixing that.

(Note that fetch-pack is sometimes run through a remote helper, meaning
that we could leave the die() invocations in and just make sure that we
handle failure in the separate process correctly. But when the promisor
remote is HTTP protocol v2 or SSH protocol v0/v2, this is not true -
fetch_pack() is run in-process.)

I think the best way for easy authorship and review is to convert each
possibly-dying function into a function that either returns a
possibly-null error string or returns success/failure and writes the
error string into an "out" parameter. In this way, the change is rather
mechanical and should be easy to review. In the patch below I chose the
former approach, and I modified 2 functions (one that returns no value
and one that returns a value) to demonstrate what it would look like.

We could also take this further and have a "struct error" for type
safety and macros - e.g. THROW() to return a "struct error", TRY() to
execute what's inside the parentheses and return the error if there is
one, and OR_DIE() to execute what's inside the parentheses and die if
there is an error.

Any opinions before I continue working on this?

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 fetch-pack.c | 78 +++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 53 insertions(+), 25 deletions(-)

diff --git a/fetch-pack.c b/fetch-pack.c
index 80fb3bd899..20a7e05ea8 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -24,6 +24,8 @@
 #include "fsck.h"
 #include "shallow.h"
 
+typedef char * error_string;
+
 static int transfer_unpack_limit = -1;
 static int fetch_unpack_limit = -1;
 static int unpack_limit = 100;
@@ -136,8 +138,8 @@ enum ack_type {
 	ACK_ready
 };
 
-static void consume_shallow_list(struct fetch_pack_args *args,
-				 struct packet_reader *reader)
+static error_string consume_shallow_list(struct fetch_pack_args *args,
+					 struct packet_reader *reader)
 {
 	if (args->stateless_rpc && args->deepen) {
 		/* If we sent a depth we will get back "duplicate"
@@ -149,41 +151,54 @@ static void consume_shallow_list(struct fetch_pack_args *args,
 				continue;
 			if (starts_with(reader->line, "unshallow "))
 				continue;
-			die(_("git fetch-pack: expected shallow list"));
+			return xstrdup(_("git fetch-pack: expected shallow list"));
 		}
 		if (reader->status != PACKET_READ_FLUSH)
-			die(_("git fetch-pack: expected a flush packet after shallow list"));
+			return xstrdup(_("git fetch-pack: expected a flush packet after shallow list"));
 	}
+	return NULL;
 }
 
-static enum ack_type get_ack(struct packet_reader *reader,
-			     struct object_id *result_oid)
+static error_string get_ack(struct packet_reader *reader,
+			    enum ack_type *result_ack,
+			    struct object_id *result_oid)
 {
 	int len;
 	const char *arg;
 
 	if (packet_reader_read(reader) != PACKET_READ_NORMAL)
-		die(_("git fetch-pack: expected ACK/NAK, got a flush packet"));
+		return xstrdup(_("git fetch-pack: expected ACK/NAK, got a flush packet"));
 	len = reader->pktlen;
 
-	if (!strcmp(reader->line, "NAK"))
-		return NAK;
+	if (!strcmp(reader->line, "NAK")) {
+		*result_ack = NAK;
+		return NULL;
+	}
 	if (skip_prefix(reader->line, "ACK ", &arg)) {
 		const char *p;
 		if (!parse_oid_hex(arg, result_oid, &p)) {
 			len -= p - reader->line;
-			if (len < 1)
-				return ACK;
-			if (strstr(p, "continue"))
-				return ACK_continue;
-			if (strstr(p, "common"))
-				return ACK_common;
-			if (strstr(p, "ready"))
-				return ACK_ready;
-			return ACK;
+			if (len < 1) {
+				*result_ack = ACK;
+				return NULL;
+			}
+			if (strstr(p, "continue")) {
+				*result_ack = ACK_continue;
+				return NULL;
+			}
+			if (strstr(p, "common")) {
+				*result_ack = ACK_common;
+				return NULL;
+			}
+			if (strstr(p, "ready")) {
+				*result_ack = ACK_ready;
+				return NULL;
+			}
+			*result_ack = ACK;
+			return NULL;
 		}
 	}
-	die(_("git fetch-pack: expected ACK/NAK, got '%s'"), reader->line);
+	return xstrfmt(_("git fetch-pack: expected ACK/NAK, got '%s'"), reader->line);
 }
 
 static void send_request(struct fetch_pack_args *args,
@@ -394,7 +409,8 @@ static int find_common(struct fetch_negotiator *negotiator,
 		print_verbose(args, "have %s", oid_to_hex(oid));
 		in_vain++;
 		if (flush_at <= ++count) {
-			int ack;
+			enum ack_type ack;
+			error_string err;
 
 			packet_buf_flush(&req_buf);
 			send_request(args, fd[1], &req_buf);
@@ -409,9 +425,11 @@ static int find_common(struct fetch_negotiator *negotiator,
 			if (!args->stateless_rpc && count == INITIAL_FLUSH)
 				continue;
 
-			consume_shallow_list(args, &reader);
+			if ((err = consume_shallow_list(args, &reader)))
+				die("%s", err);
 			do {
-				ack = get_ack(&reader, result_oid);
+				if ((err = get_ack(&reader, &ack, result_oid)))
+					die("%s", err);
 				if (ack)
 					print_verbose(args, _("got %s %d %s"), "ack",
 						      ack, oid_to_hex(result_oid));
@@ -457,6 +475,9 @@ static int find_common(struct fetch_negotiator *negotiator,
 						got_ready = 1;
 					break;
 					}
+				case NAK:
+					/* nothing */
+					break;
 				}
 			} while (ack);
 			flushes--;
@@ -481,10 +502,17 @@ static int find_common(struct fetch_negotiator *negotiator,
 	}
 	strbuf_release(&req_buf);
 
-	if (!got_ready || !no_done)
-		consume_shallow_list(args, &reader);
+	if (!got_ready || !no_done) {
+		error_string err;
+		if ((err = consume_shallow_list(args, &reader)))
+			die("%s", err);
+	}
 	while (flushes || multi_ack) {
-		int ack = get_ack(&reader, result_oid);
+		error_string err;
+		enum ack_type ack;
+
+		if ((err = get_ack(&reader, &ack, result_oid)))
+			die("%s", err);
 		if (ack) {
 			print_verbose(args, _("got %s (%d) %s"), "ack",
 				      ack, oid_to_hex(result_oid));
-- 
2.28.0.rc0.142.g3c755180ce-goog


             reply	other threads:[~2020-07-24 22:38 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-24 22:38 Jonathan Tan [this message]
2020-07-24 23:07 ` Junio C Hamano
2020-07-24 23:11 ` Junio C Hamano
2020-07-25 21:41   ` Jeff King
2020-07-25 23:01     ` Junio C Hamano
2020-07-27 17:11       ` Jeff King
2020-07-28 19:23       ` Jonathan Tan
2020-07-28 20:08         ` Jeff King
2020-07-29 18:53           ` Jonathan Tan
2020-07-29 19:29             ` Jeff King
2020-07-29 19:02           ` Junio C Hamano
2020-07-29 22:55             ` Jonathan Tan
2020-08-05  1:20 ` [RFC PATCH 0/7] Lazy fetch with subprocess Jonathan Tan
2020-08-05  1:20   ` [RFC PATCH 1/7] fetch-pack: allow NULL negotiator->add_tip Jonathan Tan
2020-08-05 19:53     ` Junio C Hamano
2020-08-07 20:53       ` Jonathan Tan
2020-08-05  1:20   ` [RFC PATCH 2/7] fetch-pack: allow NULL negotiator->known_common Jonathan Tan
2020-08-05 20:08     ` Junio C Hamano
2020-08-05 22:11       ` Junio C Hamano
2020-08-07 20:59         ` Jonathan Tan
2020-08-05  1:20   ` [RFC PATCH 3/7] negotiator/null: add null fetch negotiator Jonathan Tan
2020-08-05  1:20   ` [RFC PATCH 4/7] fetch: --stdin Jonathan Tan
2020-08-05 20:33     ` Junio C Hamano
2020-08-07 21:10       ` Jonathan Tan
2020-08-07 21:58         ` Junio C Hamano
2020-08-07 21:10       ` Jonathan Tan
2020-08-05  1:20   ` [RFC PATCH 5/7] fetch: submodule config Jonathan Tan
2020-08-05  1:20   ` [RFC PATCH 6/7] fetch: only populate existing_refs if needed Jonathan Tan
2020-08-05  1:20   ` [RFC PATCH 7/7] promisor-remote: use subprocess to fetch Jonathan Tan
2020-08-11 22:52 ` [PATCH v2 0/7] Lazy fetch with subprocess Jonathan Tan
2020-08-11 22:52   ` [PATCH v2 1/7] negotiator/null: add null fetch negotiator Jonathan Tan
2020-08-12 12:55     ` Derrick Stolee
2020-08-12 16:44       ` Junio C Hamano
2020-08-12 17:29         ` Jonathan Tan
2020-08-11 22:52   ` [PATCH v2 2/7] fetch: allow refspecs specified through stdin Jonathan Tan
2020-08-11 22:52   ` [PATCH v2 3/7] fetch: avoid reading submodule config until needed Jonathan Tan
2020-08-12 17:34     ` Junio C Hamano
2020-08-11 22:52   ` [PATCH v2 4/7] fetch: only populate existing_refs if needed Jonathan Tan
2020-08-12 18:06     ` Junio C Hamano
2020-08-11 22:52   ` [PATCH v2 5/7] fetch-pack: do not lazy-fetch during ref iteration Jonathan Tan
2020-08-12 18:25     ` Junio C Hamano
2020-08-11 22:52   ` [PATCH v2 6/7] promisor-remote: lazy-fetch objects in subprocess Jonathan Tan
2020-08-12 18:28     ` Junio C Hamano
2020-08-11 22:52   ` [PATCH v2 7/7] fetch-pack: remove no_dependents code Jonathan Tan
2020-08-12 12:51   ` [PATCH v2 0/7] Lazy fetch with subprocess Derrick Stolee
2020-08-18  4:01 ` [PATCH v3 " Jonathan Tan
2020-08-18  4:01   ` [PATCH v3 1/7] negotiator/noop: add noop fetch negotiator Jonathan Tan
2020-08-18  4:01   ` [PATCH v3 2/7] fetch: allow refspecs specified through stdin Jonathan Tan
2020-08-18  4:01   ` [PATCH v3 3/7] fetch: avoid reading submodule config until needed Jonathan Tan
2020-08-18  4:01   ` [PATCH v3 4/7] fetch: only populate existing_refs if needed Jonathan Tan
2020-08-18  4:01   ` [PATCH v3 5/7] fetch-pack: do not lazy-fetch during ref iteration Jonathan Tan
2020-08-18  4:01   ` [PATCH v3 6/7] promisor-remote: lazy-fetch objects in subprocess Jonathan Tan
2020-08-18  4:01   ` [PATCH v3 7/7] fetch-pack: remove no_dependents code Jonathan Tan
2020-08-18 19:56   ` [PATCH v3 0/7] Lazy fetch with subprocess Junio C Hamano
2020-08-18 22:32     ` Junio C Hamano
2020-08-18 23:36       ` [PATCH] fixup! promisor-remote: lazy-fetch objects in subprocess Jonathan Tan
2020-08-18 23:57         ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200724223844.2723397-1-jonathantanmy@google.com \
    --to=jonathantanmy@google.com \
    --cc=git@vger.kernel.org \
    --subject='Re: [RFC PATCH] Modify fetch-pack to no longer die on error?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).