[PATCH 0/1] commit-graph: writing missing parents is a BUG

[PATCH 0/1] commit-graph: writing missing parents is a BUG

[Derrick Stolee via GitGitGadget]

A user complained that they had the following message in a git command:

fatal: invalid parent position 2147483647

In hex, this value is 0x7fffffff, corresponding to the GRAPH_MISSING_PARENT
constant. This constant was intended as a way to have the commit-graph store
commits with parents that are not also in the commit-graph. During
development, however, we chose to require the commit-graph to be closed
under reachability. Thus, this value should never be written, and we don't
fall back to parsing usual commits when we see the constant.

This actually happened, and the root cause is unknown. This either means the
reachable closure logic is broken somewhere, or something caused the binary
search to find the parent in our list of commits. This second problem is
more likely, as we have seen RAM issues cause corrupted memory before. I'm
still investigating the root cause, but for now we can hit a BUG() statement
instead of writing bad data.

Thanks, -Stolee

Derrick Stolee (1):
  commit-graph: writing missing parents is a BUG

 commit-graph.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)


base-commit: 0d0ac3826a3bbb9247e39e12623bbcfdd722f24c
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-102%2Fderrickstolee%2Fcommit-graph-bug-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-102/derrickstolee/commit-graph-bug-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/102
-- 
gitgitgadget

[PATCH 1/1] commit-graph: writing missing parents is a BUG

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

When writing a commit-graph, we write GRAPH_MISSING_PARENT if the
parent's object id does not appear in the list of commits to be
written into the commit-graph. This was done as the initial design
allowed commits to have missing parents, but the final version
requires the commit-graph to be closed under reachability. Thus,
this GRAPH_MISSING_PARENT value should never be written.

However, there are reasons why it could be written! These range
from a bug in the reachable-closure code to a memory error causing
the binary search into the list of object ids to fail. In either
case, we should fail fast and avoid writing the commit-graph file
with bad data.

Remove the GRAPH_MISSING_PARENT constant in favor of the constant
GRAPH_EDGE_LAST_MASK, which has the same value.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 commit-graph.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/commit-graph.c b/commit-graph.c
index 40c855f185..c14ada6918 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -34,7 +34,6 @@
 #define GRAPH_OID_LEN GRAPH_OID_LEN_SHA1
 
 #define GRAPH_OCTOPUS_EDGES_NEEDED 0x80000000
-#define GRAPH_PARENT_MISSING 0x7fffffff
 #define GRAPH_EDGE_LAST_MASK 0x7fffffff
 #define GRAPH_PARENT_NONE 0x70000000
 
@@ -496,7 +495,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,
 					      commit_to_sha1);
 
 			if (edge_value < 0)
-				edge_value = GRAPH_PARENT_MISSING;
+				BUG("missing parent %s for commit %s",
+				    oid_to_hex(&parent->item->object.oid),
+				    oid_to_hex(&(*list)->object.oid));
 		}
 
 		hashwrite_be32(f, edge_value);
@@ -514,7 +515,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,
 					      nr_commits,
 					      commit_to_sha1);
 			if (edge_value < 0)
-				edge_value = GRAPH_PARENT_MISSING;
+				BUG("missing parent %s for commit %s",
+				    oid_to_hex(&parent->item->object.oid),
+				    oid_to_hex(&(*list)->object.oid));
 		}
 
 		hashwrite_be32(f, edge_value);
@@ -567,7 +570,9 @@ static void write_graph_chunk_large_edges(struct hashfile *f,
 						  commit_to_sha1);
 
 			if (edge_value < 0)
-				edge_value = GRAPH_PARENT_MISSING;
+				BUG("missing parent %s for commit %s",
+				    oid_to_hex(&parent->item->object.oid),
+				    oid_to_hex(&(*list)->object.oid));
 			else if (!parent->next)
 				edge_value |= GRAPH_LAST_EDGE;
 
@@ -864,7 +869,7 @@ void write_commit_graph(const char *obj_dir,
 			count_distinct++;
 	}
 
-	if (count_distinct >= GRAPH_PARENT_MISSING)
+	if (count_distinct >= GRAPH_EDGE_LAST_MASK)
 		die(_("the commit graph format cannot write %d commits"), count_distinct);
 
 	commits.nr = 0;
@@ -891,7 +896,7 @@ void write_commit_graph(const char *obj_dir,
 	}
 	num_chunks = num_extra_edges ? 4 : 3;
 
-	if (commits.nr >= GRAPH_PARENT_MISSING)
+	if (commits.nr >= GRAPH_EDGE_LAST_MASK)
 		die(_("too many commits to write graph"));
 
 	compute_generation_numbers(&commits, report_progress);
-- 
gitgitgadget

Re: [PATCH 1/1] commit-graph: writing missing parents is a BUG

[Junio C Hamano]

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Derrick Stolee <dstolee@microsoft.com>
>
> When writing a commit-graph, we write GRAPH_MISSING_PARENT if the
> parent's object id does not appear in the list of commits to be
> written into the commit-graph. This was done as the initial design
> allowed commits to have missing parents, but the final version
> requires the commit-graph to be closed under reachability. Thus,
> this GRAPH_MISSING_PARENT value should never be written.
>
> However, there are reasons why it could be written! These range
> from a bug in the reachable-closure code to a memory error causing
> the binary search into the list of object ids to fail. In either
> case, we should fail fast and avoid writing the commit-graph file
> with bad data.
>
> Remove the GRAPH_MISSING_PARENT constant in favor of the constant
> GRAPH_EDGE_LAST_MASK, which has the same value.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---

Thanks, will queue.