git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* [PATCH 0/1] commit-graph: writing missing parents is a BUG
@ 2018-12-19 20:14 Derrick Stolee via GitGitGadget
  2018-12-19 20:14 ` [PATCH 1/1] " Derrick Stolee via GitGitGadget
  0 siblings, 1 reply; 3+ messages in thread
From: Derrick Stolee via GitGitGadget @ 2018-12-19 20:14 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano

A user complained that they had the following message in a git command:

fatal: invalid parent position 2147483647

In hex, this value is 0x7fffffff, corresponding to the GRAPH_MISSING_PARENT
constant. This constant was intended as a way to have the commit-graph store
commits with parents that are not also in the commit-graph. During
development, however, we chose to require the commit-graph to be closed
under reachability. Thus, this value should never be written, and we don't
fall back to parsing usual commits when we see the constant.

This actually happened, and the root cause is unknown. This either means the
reachable closure logic is broken somewhere, or something caused the binary
search to find the parent in our list of commits. This second problem is
more likely, as we have seen RAM issues cause corrupted memory before. I'm
still investigating the root cause, but for now we can hit a BUG() statement
instead of writing bad data.

Thanks, -Stolee

Derrick Stolee (1):
  commit-graph: writing missing parents is a BUG

 commit-graph.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)


base-commit: 0d0ac3826a3bbb9247e39e12623bbcfdd722f24c
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-102%2Fderrickstolee%2Fcommit-graph-bug-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-102/derrickstolee/commit-graph-bug-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/102
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH 1/1] commit-graph: writing missing parents is a BUG
  2018-12-19 20:14 [PATCH 0/1] commit-graph: writing missing parents is a BUG Derrick Stolee via GitGitGadget
@ 2018-12-19 20:14 ` Derrick Stolee via GitGitGadget
  2019-01-02 23:01   ` Junio C Hamano
  0 siblings, 1 reply; 3+ messages in thread
From: Derrick Stolee via GitGitGadget @ 2018-12-19 20:14 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Derrick Stolee

From: Derrick Stolee <dstolee@microsoft.com>

When writing a commit-graph, we write GRAPH_MISSING_PARENT if the
parent's object id does not appear in the list of commits to be
written into the commit-graph. This was done as the initial design
allowed commits to have missing parents, but the final version
requires the commit-graph to be closed under reachability. Thus,
this GRAPH_MISSING_PARENT value should never be written.

However, there are reasons why it could be written! These range
from a bug in the reachable-closure code to a memory error causing
the binary search into the list of object ids to fail. In either
case, we should fail fast and avoid writing the commit-graph file
with bad data.

Remove the GRAPH_MISSING_PARENT constant in favor of the constant
GRAPH_EDGE_LAST_MASK, which has the same value.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 commit-graph.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/commit-graph.c b/commit-graph.c
index 40c855f185..c14ada6918 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -34,7 +34,6 @@
 #define GRAPH_OID_LEN GRAPH_OID_LEN_SHA1
 
 #define GRAPH_OCTOPUS_EDGES_NEEDED 0x80000000
-#define GRAPH_PARENT_MISSING 0x7fffffff
 #define GRAPH_EDGE_LAST_MASK 0x7fffffff
 #define GRAPH_PARENT_NONE 0x70000000
 
@@ -496,7 +495,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,
 					      commit_to_sha1);
 
 			if (edge_value < 0)
-				edge_value = GRAPH_PARENT_MISSING;
+				BUG("missing parent %s for commit %s",
+				    oid_to_hex(&parent->item->object.oid),
+				    oid_to_hex(&(*list)->object.oid));
 		}
 
 		hashwrite_be32(f, edge_value);
@@ -514,7 +515,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,
 					      nr_commits,
 					      commit_to_sha1);
 			if (edge_value < 0)
-				edge_value = GRAPH_PARENT_MISSING;
+				BUG("missing parent %s for commit %s",
+				    oid_to_hex(&parent->item->object.oid),
+				    oid_to_hex(&(*list)->object.oid));
 		}
 
 		hashwrite_be32(f, edge_value);
@@ -567,7 +570,9 @@ static void write_graph_chunk_large_edges(struct hashfile *f,
 						  commit_to_sha1);
 
 			if (edge_value < 0)
-				edge_value = GRAPH_PARENT_MISSING;
+				BUG("missing parent %s for commit %s",
+				    oid_to_hex(&parent->item->object.oid),
+				    oid_to_hex(&(*list)->object.oid));
 			else if (!parent->next)
 				edge_value |= GRAPH_LAST_EDGE;
 
@@ -864,7 +869,7 @@ void write_commit_graph(const char *obj_dir,
 			count_distinct++;
 	}
 
-	if (count_distinct >= GRAPH_PARENT_MISSING)
+	if (count_distinct >= GRAPH_EDGE_LAST_MASK)
 		die(_("the commit graph format cannot write %d commits"), count_distinct);
 
 	commits.nr = 0;
@@ -891,7 +896,7 @@ void write_commit_graph(const char *obj_dir,
 	}
 	num_chunks = num_extra_edges ? 4 : 3;
 
-	if (commits.nr >= GRAPH_PARENT_MISSING)
+	if (commits.nr >= GRAPH_EDGE_LAST_MASK)
 		die(_("too many commits to write graph"));
 
 	compute_generation_numbers(&commits, report_progress);
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH 1/1] commit-graph: writing missing parents is a BUG
  2018-12-19 20:14 ` [PATCH 1/1] " Derrick Stolee via GitGitGadget
@ 2019-01-02 23:01   ` Junio C Hamano
  0 siblings, 0 replies; 3+ messages in thread
From: Junio C Hamano @ 2019-01-02 23:01 UTC (permalink / raw)
  To: Derrick Stolee via GitGitGadget; +Cc: git, Derrick Stolee

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Derrick Stolee <dstolee@microsoft.com>
>
> When writing a commit-graph, we write GRAPH_MISSING_PARENT if the
> parent's object id does not appear in the list of commits to be
> written into the commit-graph. This was done as the initial design
> allowed commits to have missing parents, but the final version
> requires the commit-graph to be closed under reachability. Thus,
> this GRAPH_MISSING_PARENT value should never be written.
>
> However, there are reasons why it could be written! These range
> from a bug in the reachable-closure code to a memory error causing
> the binary search into the list of object ids to fail. In either
> case, we should fail fast and avoid writing the commit-graph file
> with bad data.
>
> Remove the GRAPH_MISSING_PARENT constant in favor of the constant
> GRAPH_EDGE_LAST_MASK, which has the same value.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---

Thanks, will queue.

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2019-01-02 23:02 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-12-19 20:14 [PATCH 0/1] commit-graph: writing missing parents is a BUG Derrick Stolee via GitGitGadget
2018-12-19 20:14 ` [PATCH 1/1] " Derrick Stolee via GitGitGadget
2019-01-02 23:01   ` Junio C Hamano

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.version-control.git
	nntp://ie5yzdi7fg72h7s4sdcztq5evakq23rdt33mfyfcddc5u3ndnw24ogqd.onion/inbox.comp.version-control.git
	nntp://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git