Re: 2.29.0.rc0.windows.1: Duplicate commit id error message when fetching

[Derrick Stolee <stolee@gmail.com>]

On 10/9/2020 11:29 AM, Thomas Braun wrote:
> I think I have a starting point for what goes wrong. I found a local
> repo with another broken commit graph. And after some fiddling the
> following script can reproduce it. I tried with git/git first but that
> seems not to trigger that.

I'm glad you're able to trigger bad commit-graph data somehow.
Let's see what's going on:

> # rm -rf dummy
> mkdir dummy
> cd dummy
> 
> git init
> 
> git remote add origin https://github.com/tango-controls/cppTango
> git remote add fork1 https://github.com/bourtemb/cppTango
> git remote add fork2 https://github.com/t-b/cppTango
> git fetch --all --jobs 12

My gut reaction is that this parallel fetching is causing
an issue, but we will see. Do you get a repro if you drop
the "--jobs 12"?

> git commit-graph verify
> rm -rf .git/objects/info/commit-graphs/
> git commit-graph verify
> git fetch --jobs 12
> git remote add fork3 git@github.com:t-b/cppTango.git
> git commit-graph verify
> git remote add fork4 git@github.com:t-b/cppTango.git
> git fetch --jobs 12
> git commit-graph verify
> 
> The last verify outputs
> 
> commit-graph generation for commit
> 029341567c24582030592585b395f4438273263f is 1054 != 1
> commit-graph generation for commit
> 1e8d10aec7ca6075f622c447d416071390698124 is 4294967295 != 1171
> commit-graph generation for commit
> 296e93516189c0134843fd56ac4f10d36ccf284f is 1054 != 1
> commit-graph generation for commit
> 4c0a7a3cd369d06b99d867be6b47a96c519efd7f is 1054 != 1
> commit-graph has non-zero generation number for commit
> 4d39849950d3dc02b7426c780ac7991ec7221176, but zero elsewhere
> commit-graph has non-zero generation number for commit 4
> [....]

This looks more troubling than just duplicate rows, but
perhaps those duplicate rows are causing sufficient
confusion when reading the commit-graph during the
'verify' command?

I tried incorporating this into the Git test suite so I
could test it on v2.29.0-rc0 and the current merge-check,
but I'm failing to reproduce the failure with this script:

diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh
index c334ee9155..2b3f3db593 100755
--- a/t/t5324-split-commit-graph.sh
+++ b/t/t5324-split-commit-graph.sh
@@ -440,4 +440,26 @@ test_expect_success '--split=replace with partial Bloom data' '
 	verify_chain_files_exist $graphdir
 '
 
+test_expect_success 'test' '
+	git init dummy &&
+	(
+		cd dummy &&
+		export GIT_TRACE2_EVENT="$TRASH_DIRECTORY/../trace.txt" &&
+		git config fetch.writeCommitGraph true &&
+		git remote add origin https://github.com/tango-controls/cppTango &&
+		git remote add fork1 https://github.com/bourtemb/cppTango &&
+		git remote add fork2 https://github.com/t-b/cppTango &&
+		git fetch --all --jobs 12 &&
+		git commit-graph verify &&
+		rm -rf .git/objects/info/commit-graphs/ &&
+		git commit-graph verify &&
+		git fetch --jobs 12 &&
+		git remote add fork3 git@github.com:t-b/cppTango.git &&
+		git commit-graph verify &&
+		git remote add fork4 git@github.com:t-b/cppTango.git &&
+		git fetch --jobs 12 &&
+		git commit-graph verify
+	)
+'
+
 test_done

I tried this on Linux and Windows, and under "--stress" but never
saw a failure.

Thomas: some things that could possibly help is if you repro this
situation but also do something like

	export GIT_TRACE2_EVENT="$(pwd)/trace.txt"

so we can read the details of everything Git is tracing during
these parallel jobs. We might be able to stitch together a
sequence of events that lead to these failures.

Thanks,
Stolee