Re: Help with "fatal: unable to read ...." error during GC?

[Jeff King <peff@peff.net>]

On Wed, Aug 08, 2018 at 10:30:11AM -0400, Paul Smith wrote:

> I recently upgraded from Git 2.9.2 to 2.18.0 (note, I have no
> particular reason to believe this is related just passing info).  I'm
> running on Linux (64bit Ubuntu 18.04.1 but I've compiled Git myself
> from source, I'm not using the distro version).
> 
> I have a local repository I've been using for about two years (the
> .git/description file, which I don't use, has a TLM of July 31, 2016),
> with lots of worktrees being created/pruned/etc. during that time.
> 
> Note I'm doing all these operations in the 'main' repository, not in
> any of the worktrees.

Hrm, there was a pretty serious corruption bug in early versions of the
worktree code (IIRC, pruning would not consider detached HEADs from
other worktrees, and could drop that object).

> Yesterday, when I tried to fetch from my upstream I got a notification
> about GC needed.  Then GC failed with these errors (HEAD is set to
> master which is the same as origin/master):
> 
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   warning: reflog of 'HEAD' references pruned commits
>   fatal: unable to read c104b8fb3631b5c54695206b2f73310c023c9963
>   error: failed to run repack

So that definitely looks like the corruption I'd expect from the
worktree bug, but...

> I ran a git fsck --full which showed me a lot of dangling commits and
> blobs, but no errors, no broken link messages, etc.

I'd have expected fsck to find it, too. However, looking at the code,
I'm not convinced that fsck is actually considering detached worktree
heads properly, either. Try:

  git rev-list --all --reflog --objects >/dev/null

which I know checks worktrees correctly. I'd expect that to fail.

If it does, then we need to narrow down which worktree is corrupt.
Perhaps something like:

  git worktree list |
  while read worktree head junk; do
	git rev-list --objects $head >/dev/null ||
	echo "$worktree seems corrupt"
  done

> I can't find that SHA anywhere: I looked in .git/objects, etc.  I also
> can't find any problems with my repo; obviously I haven't checked
> everything but I can show the git log back to the initial commit, all
> my stashes look fine, all my worktrees seem to be OK (git status etc.
> work fine in all of them).

"git status" might succeed if the corruption is further back in the
history.

> I would hate to have to throw this setup away since it has 23 stashes
> and 25 worktrees in various states that would be annoying to have to
> recreate... 

Definitely don't throw it away. I suspect you have a single corrupt
worktree, and everything else is fine.

-Peff