fatal: unable to read after commit

fatal: unable to read after commit

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi folks,

I am a heavy user of git now at version 2.20.1 on debian.

Since some weeks I have the problem that I get often "fatal: unable to
read ..." and a unclear repository after a git commit. The commit itself
is correct and so a git reset --hard helps to fix the issue.

Any Idea what could be the reason for that problem. I encounter it on
different repositories so not limited to one.

Any hint how to find out the reason?

Regards
   Klaus

Ps. I am not subscribet to the list, so please keep me in the response.
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlyvbvEACgkQpnwKsYAZ
9qyeggwAjLjfP8M2+Bybc6WFCnPjT2tAoU2Rpt1w7lPfycbqOykwa8YHML9Hy4OW
G+ODpR3SXnRgKDzQ8xpz7i5/6pNC3ZlAYrvnOQFPHpL6zlOWr9aeDo8wai8FzVKk
4ei32u3Gw/bu0tH6D/SKxCVjC8sZzHWx+NjUDJh8ld3ln5RmF5XpK5+UvVak/nal
rj+MxUyB1Aklho5cnwlFbdW0aUOM6Iom4G5uZz3dfh3p2VuPNzbbU/RKYXHGrJIB
01/Wh67r9XsML8z2kwYrpmT6Hcpuz/uuuIw81fHp55419YcbtS9bCJsYwYy0BGo0
LWJX5PzyMbWbbJE6oS1YDZOfnY/W/iIdq9JysMsyt1+nURcWNkHdOQ4+vY5DeBDa
/kEDH9BLyzGddLAShYvspMpZg2J5c0v1FtX9N7q4A/vUyemevpQKJq+qJPV1WelI
7Hs45gvWj7oTOxVE4yJtMYkdO1jZLznVSLG36XT06+ciXiBl2FmVkTZ9yQaRse9L
bIWDVDXp
=4x1q
-----END PGP SIGNATURE-----

Re: fatal: unable to read after commit

[Christian Couder]

Hi,

On Thu, Apr 11, 2019 at 7:24 PM Klaus Ethgen <Klaus@ethgen.ch> wrote:

> I am a heavy user of git now at version 2.20.1 on debian.
>
> Since some weeks I have the problem that I get often "fatal: unable to
> read ..." and a unclear repository after a git commit. The commit itself
> is correct and so a git reset --hard helps to fix the issue.

Could you tell us at least which Debian version and file system you use?

Would you be ok to bisect it or at least tell us if it happens with
2.19.2, 2.20.0 and 2.21.0?

> Any Idea what could be the reason for that problem. I encounter it on
> different repositories so not limited to one.

Is it easy to reproduce even on very small test repos? Could you send
us a small script that reproduces it?

Could you also run the Git test suite on your machine?

Thanks,
Christian.

Re: fatal: unable to read after commit

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi Christian,

Am Fr den 12. Apr 2019 um  9:39 schrieb Christian Couder:
> On Thu, Apr 11, 2019 at 7:24 PM Klaus Ethgen <Klaus@ethgen.ch> wrote:
> 
> > I am a heavy user of git now at version 2.20.1 on debian.
> >
> > Since some weeks I have the problem that I get often "fatal: unable to
> > read ..." and a unclear repository after a git commit. The commit itself
> > is correct and so a git reset --hard helps to fix the issue.
> 
> Could you tell us at least which Debian version and file system you use?

Sorry, it is debian unstable and the filesystem is a brfs.

> Would you be ok to bisect it or at least tell us if it happens with
> 2.19.2, 2.20.0 and 2.21.0?

I could try, but from the fact, that it happens not all the time, I am
not sure if it helps.

However, I might have found one repo where it allways happens. In fact,
I am able to reproduce it also in my local geeqie checkout.

> > Any Idea what could be the reason for that problem. I encounter it on
> > different repositories so not limited to one.
> 
> Is it easy to reproduce even on very small test repos? Could you send
> us a small script that reproduces it?

In any case it happens if I modify a file and add+commit or commit -a
it. What happens is that the index seems to be corrupted. The HEAD gets
the correct commit and the checked out version of the file is also
correct. Only the index seems to be corrupt (Until I do git reset).

When I do a git status, I get the following output:
   Auf Branch master
   Ihr Branch ist 1 Commit vor 'origin/master'.
     (benutzen Sie "git push", um lokale Commits zu publizieren)

   Zum Commit vorgemerkte Änderungen:
     (benutzen Sie "git reset HEAD <Datei>..." zum Entfernen aus der Staging-Area)

	   neue Datei:     pending.data

   Änderungen, die nicht zum Commit vorgemerkt sind:
     (benutzen Sie "git add/rm <Datei>...", um die Änderungen zum Commit vorzumerken)
     (benutzen Sie "git checkout -- <Datei>...", um die Änderungen im Arbeitsverzeichnis zu verwerfen)

	   gelöscht:       pending.data

And it seems, that the brocken object is always the same:
   > git diff
   fatal: unable to read 544f4ec5fe7c7b04c73b2c2fe9e3e7779e929819

independent from the repository.

I do not have a pending.data and did not touch this one.

> Could you also run the Git test suite on your machine?

Sure.

But at the moment, I have some more findings that points to another
component in the queue. When I do a "commit -a -m test", the error does
not happens. When I do a "commit -a" and use the editor (vim) to write
the commit message, it happens.

As I use fugitive, I cannot rule out, that the problem is in that vim
addon.

Regards
   Klaus
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlywVuYACgkQpnwKsYAZ
9qybEQv/bn3Bs+e/xgPfWK9yDbBlcmVywAiBty5RgYl5FQNHPb/6rRbZMk9wT6MJ
NkLGTj5sgrtRik+TU8C0j982XBJW6EY8jEIzOisBtm5aQ63KdymSI+q6MQd1C3/R
1dd4IMfM2GylQohhsptUdRwErvjoOKHpqAvJnKMnzUC5YxYZ4QL64LLnjoCAz3mK
1W4VceZVHluD/2nv9HP5aOgqihWJNYhIGP9308X4JXw6qn5wGWGljDsyk4NE+NWr
3O0QBYGJO7AYA1WH6eCrPx7DSrPpArdLIwhQCntIKJIrsf3mn4VBUAVB+uhTLeIa
oFyx9vSX/9KlruL/ni287RCRJBYh4DVDsXligFeUTRe0SOuVPgjku9R2oYKpoeKB
7AhI+rKY6vg3At/6iIcaeSw7+ln8ilSqLmbTAz0ygc16T7+CNK5En+nAM2otiNmU
Fs5upQsVGb6pCFTp6aC8P6v+eTqlvH+l6jMsR/7Lx95U9+yOSdSO5x9sbCasCs/g
Z/XxPGkv
=P22A
-----END PGP SIGNATURE-----

Re: fatal: unable to read after commit

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I seems to have found the bug.

My last debugging gave the hint. pending.data is a concept of
taskwarrior and have the .task directory on git control. And I also use
a vim addon for it[0].

It seems that it introduces that troubles. As the last commit is from
2016, it is very strange. Especially as I had that troubles not before.

Regards
   Klaus
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlywWjkACgkQpnwKsYAZ
9qwZDgv+Pcg834UWtW5VJ9gwEz7Won8Vdt9/yPUEKb8h8hoD0goI/5J4LlTNVyn0
mKcfvPr1QCZTy08fltrSJPtJwyhvkcJpzxZ8+1XtQsRea6V3IXEN6D8HTpzfPSfm
wAIyTqX31F66Almk2sxxit9aRqCbio+KhyN50twDAOq60SwVtw/1Yex/VqP1WxHb
8245u9bg88hgdpJROZsiPA320XOBrQqykDTwvWyKsLewAhhx5qmxsm3SXcmzp7NC
kjIsqtiFNB68uzcAdwq2RLe7OEexoNHNHpktzWhL25CdPIIeQJIYHKiLXgj/dl3J
G4RrbXHAw6GAoB3ce5IboDpU2jm2JpgXtKIhbJItV4Tt/ju2pmP3caTc/JMZF4Vz
jLfcg8KyD+pe/mI9h0Tj71lz6Q3FvVY9HNW6t38v/5KM2n8gjYqfwkuXIT5xy/Dk
42D1WDoN+9qo8reLlrA4yK37DG2ze2q/+kf8mf2Vp2gJZB0/wfcFsr8xYiVHr14O
zH+WboKx
=y0cC
-----END PGP SIGNATURE-----

Re: fatal: unable to read after commit

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Forgot the link:
[0] https://github.com/farseer90718/vim-taskwarrior
Regards
   Klaus
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlywWsUACgkQpnwKsYAZ
9qwJNAv8D+9xnl9/E18OBG0I2lY70Fgm+NqdPrF+4RZVJ86UBJHT+R+K3hzxfbX6
gZb3qytdyIDms58KG6iB80Hk9d6BBJK8JLHdRV4cVwL6cpwT+EXPKzJUPgpU1rbs
BJbISWSP97+8O+5blPpddxaFGlsV77kdTDQYEabxIZ2WhmsgWDjD5uN7l5XSmhvC
9qRiI0+eQFQRDWDZHBeMunES18ntIVZRhidvzAQo+uMW5YZXd+hyrZBTJznBbbs7
u+GOIqS3a4ShxrxIjlQZQfE8RXlgoj10rqtRgJ4TvuftQsFjJP/cXTy+NXxxghuR
2/Fk9fTxgwIF+tPqZwQHLfHoQGlu0vNZ0zEGK674fdJ4AQ4NGrxgn50MYAYv8Npo
zHW/jCh61slON2XbNASQpx9PV/WsuUFQNQEAtJ477NVthT2omfRNoyPxYNZW/+aL
XJMCYSPbe9iQ5oN/3hicP4/V7vHNlzA0197/DGCwsJMieEyfBj6MWIHZuT/Z9bos
HBLkJuIE
=QrfL
-----END PGP SIGNATURE-----

Re: fatal: unable to read after commit - deeper analysis

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi,

I did a deep analysis of the problem and found that the trouble finally
was caused of some change in git (I did not search, when this changed.
But maybe you can tell me.)

Finally, the error was a combination of 4 tools, git, vim, the mentioned
vim-addon and task with a task-hook for committing pending.data.

When I do a git commit which invokes vim, then the following variables
are set:
- - GIT_INDEX_FILE
- - GIT_AUTHOR_DATE
- - GIT_PREFIX
- - GIT_EXEC_PATH

And $GIT_INDEX_FILE is the source of trouble here. The task-hook clears
respective sets the variables GIT_DIR and GIT_WORK_TREE. However, the
GIT_INDEX_FILE environment is set (in some cases) to an absolute path
pointing to .git/index.lock or .git/index (I have no idea when it is
taking the .lock variant).

Now we have a mismatch of GIT_DIR and GIT_WORK_TREE on one hand and the
absolute path of GIT_INDEX_FILE on the other. So the trouble is set. The
following "git add pending.data" did break all. It does something to an
index that does not belong to the git repo.

Mystery is when and why this changed in git. It was definitively changed
in some recent version.

Regards
   Klaus
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlyxqh8ACgkQpnwKsYAZ
9qxXWAv/QI6XeQ4H+Y39K3nLi68JLZo2/VzcI1CfVw42PeckxzxZHg1YUTXrr5Qk
+bQ5drXcfzNMxWkCe1fh9CoHiyJyiAIPNfMjqiUaCB8881Ttr4SYd/lalvYVXPgt
m0g7XO51Kh5LqPP6h7KcjBM0c6OSyQznE8Q6L0FSnDP4gCkBTW75AmxBytUk+sDq
uOajtyOiOr1Fz1krn89VBaLJJPMVo+OInbNwetQUgOGIN7BsHsG68Ilwoimdlt+H
7sd5HJwVrQ/w9VdXzniLPznzkG4l/3YlU8IQWKR13dRFf68LrT1ZR+F4TJpjlm/Y
l/KKF0EcVKbhuJ47gLIUDf3faeLFxdF1iOviKkJ0A0cfg5Z8M4ds8hBoiqhMSr+X
5fgINyhQfMurSTAuspZLJ0xjLu+Wv+h9Xt1jQZpW3YlPgi3O6Dn2K1V9ACaozP8G
0EIIMOvTNvqVw9iLe4a6cBm1M+gXfWRRf7H6wqvDhDuuSVF2h1sGwMMQHiumlqm3
au+0hEGQ
=6ikc
-----END PGP SIGNATURE-----

Re: fatal: unable to read after commit - deeper analysis

[Jeff King]

On Sat, Apr 13, 2019 at 10:21:36AM +0100, Klaus Ethgen wrote:

> Finally, the error was a combination of 4 tools, git, vim, the mentioned
> vim-addon and task with a task-hook for committing pending.data.
> 
> When I do a git commit which invokes vim, then the following variables
> are set:
> - GIT_INDEX_FILE
> - GIT_AUTHOR_DATE
> - GIT_PREFIX
> - GIT_EXEC_PATH
> 
> And $GIT_INDEX_FILE is the source of trouble here. The task-hook clears
> respective sets the variables GIT_DIR and GIT_WORK_TREE. However, the
> GIT_INDEX_FILE environment is set (in some cases) to an absolute path
> pointing to .git/index.lock or .git/index (I have no idea when it is
> taking the .lock variant).
> 
> Now we have a mismatch of GIT_DIR and GIT_WORK_TREE on one hand and the
> absolute path of GIT_INDEX_FILE on the other. So the trouble is set. The
> following "git add pending.data" did break all. It does something to an
> index that does not belong to the git repo.
> 
> Mystery is when and why this changed in git. It was definitively changed
> in some recent version.

It was always the case that GIT_INDEX_FILE would sometimes be set. For a
partial commit like:

  echo changes >>one
  echo changes >>two
  git add .
  git commit two

we have to create a temporary index (so as not to commit the staged
changes from "one").

I was curious if we started setting it for the more vanilla case of
just "git commit" more recently. But trying to bisect on:

  GIT_EDITOR=set | grep ^GIT_INDEX; :' git commit

I couldn't find any version where we _didn't_ set $GIT_INDEX_FILE, going
back to v1.6.0. So I don't know of any recent changes in this area.

Presumably you've fixed it by unsetting $GIT_INDEX_FILE in your vim
task-hook when it switches to another repo. However, you may want to
switch to asking Git about the full list of repo-related variables you
should clear. In the shell that looks like:

  unset $(git rev-parse --local-env-vars)

That should be more thorough, and will future-proof you against new
variables being added.

-Peff