Files modified, even after: git reset --hard

Files modified, even after: git reset --hard

[Martin]

I have some files, that will show up modified. Always.

If I just switch to a commit, with clean worktree before, then those 
files will be modified.

I can not stash them.
git reset --hard does not change anything.

Diff shows the entire file is modified. I suspect its the line endings..

This happens on windows, and
   autocrlf=true

The checked out file (which is marked as modified) has the correct CrLf 
endings.

What I have not been able to find, is what line endings are stored in 
the blob.
Its an xml file, so they should be just Lf.
But I *suspect* that the blob contains either CrLf, or mixed line-endings.

Could that be? that if a file does have unexpected line endings in the 
commit's blob, that it shows as modified?

One the one hand, yes, If I commit, that file will change.
On the other hand, if I just check out (or do reset-hard) then I don't 
expect modified files....

So what should happen?

This also causes problems, because in order to for example rebase 
something, I first need to switch to some commit that can be checked out 
without modified files.
Or rebase will not work.

Also in that repro, I had problems that some rebases failed with (with 
and without --reapply-cherry-picks)
    error: add_cacheinfo failed
Those rebase where between an orphaned branch, and a "normal" branch.
And it is possible that the file only had the line ending issues in one 
of the 2 branches....
But that I was not able to further investigate.



//git for windows
git version 2.32.0.windows.2

Re: Files modified, even after: git reset --hard

[Martin]

On 25/07/2021 17:04, Martin wrote:
> I have some files, that will show up modified. Always.
>
> If I just switch to a commit, with clean worktree before, then those 
> files will be modified.
>
...
> But I *suspect* that the blob contains either CrLf, or mixed 
> line-endings.

Also, if that is the case, what can I do to resolve the issue?
The file has been fixed in the meantime. It is about the old commits....

filter branch is not an option. All commit hashes must be kept. No force 
pushes to the repro...

That also means I can not use .gitattribute, because I could not amend 
it for the old revisions.
I might be able to use my local $GIT_DIR/info/attributes.
But then I risk to make new commits to that file, and introduce new 
wrong line-endings.

Maybe git replace?
I figure if I do not replace the commit objects, but only the blobs to 
which they point. Then all commits should keep their hashes. (so 
references to any commit by hash would remain working)

But it's thousands of commits, between the introduction of the issue, 
and the commit where it's fixed.
So I guess I need to create as many new blobs, and replacement entries.
- Is that practical? How does git perform with such many replacements
- Is there an easy way to create them?

Are there other, better solutions?

Re: Files modified, even after: git reset --hard

[Martin]

On 25/07/2021 17:40, Martin wrote:
> On 25/07/2021 17:04, Martin wrote:
>> I have some files, that will show up modified. Always.
>>
>> If I just switch to a commit, with clean worktree before, then those 
>> files will be modified.
>>
> ...
>> But I *suspect* that the blob contains either CrLf, or mixed 
>> line-endings.
>
> Maybe git replace?
> I figure if I do not replace the commit objects, but only the blobs to 
> which they point. Then all commits should keep their hashes. (so 
> references to any commit by hash would remain working)
>
> But it's thousands of commits, between the introduction of the issue, 
> and the commit where it's fixed.


Actually, git replace has a weird effect.

so I have "commit A" introducing the problem. Then many commits until 
"commit Y" fixing it.

If I do
switch -f --detach A
git add --renormalize the_file
git commit -m foo
git show   // get the new blob for the file
git show A  // get the old blob for the file
git replace  old_blob  new_blob

switch -f --detach A

Then "the_file" is still shown as modified.
Only
   git diff the_file
now returns empty
(where as before it returned the entire file)

On the plus side
   switch -f --detach B
has the same change, still modified, but no diff.
So one replace would solve the entire 1000 commits that are currently 
affected

Any idea, what causes the "modified" after the replace?

Re: Files modified, even after: git reset --hard

[Martin]

On 25/07/2021 19:48, Martin wrote:
>
> Actually, git replace has a weird effect.
>
> so I have "commit A" introducing the problem. Then many commits until 
> "commit Y" fixing it.
>
> If I do
> switch -f --detach A
> git add --renormalize the_file
> git commit -m foo
> git show   // get the new blob for the file
> git show A  // get the old blob for the file
> git replace  old_blob  new_blob
>
> switch -f --detach A
>
> Then "the_file" is still shown as modified.
> Only
>   git diff the_file
> now returns empty
> (where as before it returned the entire file)
>
> On the plus side
>   switch -f --detach B
> has the same change, still modified, but no diff.
> So one replace would solve the entire 1000 commits that are currently 
> affected
>
> Any idea, what causes the "modified" after the replace?
>

Ok, it seem that
   git switch
simple did not update the file.

- After the commit I am on a dangling commit, which is a direct child of 
"A".
- I insert the reprace
- I switch from that dangling to "A"

-> diff is empty
-> but status is not ok

If I now switch to a far away commit. And then switch back to "A". Then 
git status is also ok.

Re: Files modified, even after: git reset --hard

[Chris Torek]

On Sun, Jul 25, 2021 at 11:43 AM Martin <git@mfriebe.de> wrote:
>>>> [Files show up as every-line-modified]
>> [and] git replace has a weird effect.
> Ok, it seem that
>    git switch
> simple did not update the file.

Here's what's going on.

(1) No *committed* object, including committed file data, can ever
    be changed.

(2) Someone actually committed some files with some sort of line
    endings.

(Points 1 and 2 are true regardless of anything else.)

If you never have Git mess with line endings at all, neither of
these two points make any difference.  But you chose to start
asking Git to mess with line endings.  Things now depend on
exactly *what* you ask Git to *do* about these line endings.

Let's suppose, just for convenience for now, that the files in the
repository right now actually do have CRLF line endings.

Let's suppose further that you ask that Git ensure that your
*working tree* copies of each file contain CRLF line endings.

This means that existing *committed* copies of files already
have the ending you prefer.  But now that you have started to
ask Git to *mess with your line endings* -- by asking that your
*working tree* copies have CRLF endings -- Git will now begin
making sure that *future committed copies of files* have LF-only
line endings.

What this means is that any existing commit you check out has
the "wrong" line endings!  You've just told Git that all the
*committed* files should have LF-only line endings.

You might ask why "ensure my working tree copies have CRLF line
endings" means "ensure all future committed files have LF-only
line endings".  That's a valid design question.  But it is the
case.

As for "git replace", you've figured the rest out already: if
you use git replace to make Git use new, LF-only line ending
objects (file data), Git is now happy about the internal storage.
It just takes some shuffling-about to cause these replaced objects
to wind up in Git's *index* AKA *staging area*.

I'll mostly stop here, because this explains the results you have
so far, but in general, setting `core.autocrlf` tends to be
inferior to making explicit decisions about particular files
via `.gitatttributes` files.

Chris

Re: Files modified, even after: git reset --hard

[Martin]

On 26/07/2021 02:33, Chris Torek wrote:
> On Sun, Jul 25, 2021 at 11:43 AM Martin <git@mfriebe.de> wrote:
>>>>> [Files show up as every-line-modified]
>>> [and] git replace has a weird effect.
>> Ok, it seem that
>>     git switch
>> simple did not update the file.
> 
...
> Let's suppose, just for convenience for now, that the files in the
> repository right now actually do have CRLF line endings.
> 
> Let's suppose further that you ask that Git ensure that your
> *working tree* copies of each file contain CRLF line endings.
> 
Yes, I accept that decision.
I figured that is the reason why they show modified.

Not a problem. Until I am in the middle of a rebase, and i cannot run 
(after a conflict)
   git rebase --continue

The modified files are not part of the original series of commits. they 
are just random files from somewhere else in the tree.
I can not reset/restore them.
So I must now "git add" files entirely unrelated to continue rebasing.
Well or apparently change my config for the duration of the rebase.


> As for "git replace", you've figured the rest out already: if
> you use git replace to make Git use new, LF-only line ending
> objects (file data), Git is now happy about the internal storage.
> It just takes some shuffling-about to cause these replaced objects
> to wind up in Git's *index* AKA *staging area*.

Actually there is something else.

If a file has line-endings that will change, then
    git add --renormalize .
    git commit -m foo
will commit those files.

But I am now also getting files, that show modified, but that can not be 
committed renormalized (0 lines changed).

And that happens with or without refs/replaces.

Any idea how to find out why git thinks they are modified?
    git status --porcelain=v2
shows that the file mode is not modified. Only the file in the working tree.
But "git diff" shows nothing (no summary neither). And renormalizing has 
no effect.



In fact I started running the following
   git rev-list --reverse main | xargs -L 1 sh -c 'git switch --detach 
-q -f $0 ; a=$( git status -uno --porcelain=v1 ) ; if [ "$a" != "" ]; 
then git log --oneline -n1 $0 ; echo $a; fi '

That is, switch to each revision in main (or master). And check if any 
file is reported modified.

I just tried that on the gdb git. Plenty of files.
Also other repros have shown modified files.
(I have not yet tried the "git sources" git...

If I then manually switch to some of the commits that had modified files 
shown, and I do not switch to all the commits before, then sometimes 
there are no modified files.

git fsck has a few dangling commits
git gc made no difference.

Re: Files modified, even after: git reset --hard

[Chris Torek]

On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:
> Actually there is something else.
>
> If a file has line-endings that will change, then
>     git add --renormalize .
>     git commit -m foo
> will commit those files.
>
> But I am now also getting files, that show modified, but that can not be
> committed renormalized (0 lines changed).

I believe (but can't demonstrate) that this is a temporary condition.

Git has a number of cheats to make `git status` and other ops fast.
These cheats *assume* that the committed files, the index copies
of files, and the working tree copies of files all agree in terms of
line endings as coordinated through `core.autocrlf` and `.gitattributes`
settings.

When they *don't* agree, you get phantom differences.  Running
commands like `git diff` show no differences because of these
phantom states.  Eventually this clears up on its own when the
cheats really *do* agree with the settings.  Changing the settings
is what disturbs the cheats.

Git can't do much with `core.autocrlf`, but if it noticed that a
`.gitattributes` file was very recent, and turned off the shortcuts
and did the slower full status checks, updates via `.gitattributes`
would not show phantom changes.  The drawback is that updates
to `.gitattributes` could make `git status` very slow.

Overall this isn't normally a big problem.  It only affects one person
at a time, when they change these settings, and then it clears up
over time...

Chris

Re: Files modified, even after: git reset --hard

[Martin]

On 26/07/2021 04:59, Chris Torek wrote:
> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:
>> Actually there is something else.
>>
>> If a file has line-endings that will change, then
>>      git add --renormalize .
>>      git commit -m foo
>> will commit those files.
>>
>> But I am now also getting files, that show modified, but that can not be
>> committed renormalized (0 lines changed).
> 
> I believe (but can't demonstrate) that this is a temporary condition.

I now found one, that does not seem temporary at all...

git remote -v
gh-me   https://github.com/User4martin/binutils-gdb.git (fetch)
gh-me   https://github.com/User4martin/binutils-gdb.git (push)
gh-redprig      https://github.com/red-prig/binutils-gdb.git (fetch)
gh-redprig      https://github.com/red-prig/binutils-gdb.git (push)
origin  git://sourceware.org/git/binutils-gdb.git (fetch)
origin  git://sourceware.org/git/binutils-gdb.git (push)

fsck had some dangling commits, I did a "git gc --aggressive".
Now
  git fsck
Checking object directories: 100% (256/256), done.
Checking objects: 100% (1062929/1062929), done.
Checking connectivity: 1062929, done.

Switching to a far away commit
git switch -f --detach master
git rev-list master | wc -l
93917

git status shows no modified files


Then switching to  (this is on master branch)
git switch -f --detach a362ee23634
git rev-list a362ee23634 | wc -l
4164

git status --porcelain=v2
1 .M N... 100644 100644 100644 9e677a52ae690808165993a0f3f17ac49e3969df 
9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos
1 .M N... 100644 100644 100644 ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc 
ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc binutils/Makefile.dos
1 .M N... 100644 100644 100644 1d9541c2f896842d1bafe68ccf0a51e291d66688 
1d9541c2f896842d1bafe68ccf0a51e291d66688 gas/Makefile.dos
1 .M N... 100644 100644 100644 57fab985680ea151379069abe414bcb590cdd743 
57fab985680ea151379069abe414bcb590cdd743 ld/Makefile.dos


git reset --hard
makes no difference.

Only
     git diff
shows actual textual differences
e.g.

  TARGETLIB = libbfd.a
-CFLAGS = -g -O $(HDEFINES) $(TDEFINES) $(CSEARCH) $(CSWITCHES) # 
-DINTEL960VERSION
+CFLAGS = $(MINUS_G) $(HDEFINES) $(TDEFINES) $(CSEARCH) $(CSWITCHES) # 
-DINTEL960VERSION












> 
> Git has a number of cheats to make `git status` and other ops fast.
> These cheats *assume* that the committed files, the index copies
> of files, and the working tree copies of files all agree in terms of
> line endings as coordinated through `core.autocrlf` and `.gitattributes`
> settings.
> 
> When they *don't* agree, you get phantom differences.  Running
> commands like `git diff` show no differences because of these
> phantom states.  Eventually this clears up on its own when the
> cheats really *do* agree with the settings.  Changing the settings
> is what disturbs the cheats.
> 
> Git can't do much with `core.autocrlf`, but if it noticed that a
> `.gitattributes` file was very recent, and turned off the shortcuts
> and did the slower full status checks, updates via `.gitattributes`
> would not show phantom changes.  The drawback is that updates
> to `.gitattributes` could make `git status` very slow.

Ok, one of the repros has had its .gitattributes committed on the root 
commit, and never changed.
Neither do/did I change the config since it was created.

My question is are there any plumbing commands, that could allow me to 
look further at it.

Re: Files modified, even after: git reset --hard

[Philip Oakley]

On 26/07/2021 02:34, Martin wrote:
> I figured that is the reason why they show modified.
>
> Not a problem. Until I am in the middle of a rebase, and i cannot run
> (after a conflict)
>   git rebase --continue
>
> The modified files are not part of the original series of commits.
> they are just random files from somewhere else in the tree.
> I can not reset/restore them.
> So I must now "git add" files entirely unrelated to continue rebasing.
> Well or apparently change my config for the duration of the rebase.

Is this 'mid-rebase' the core case for the 'Files modified' problem? -
does it happen at other times (excepting maybe cherry-pick)

i.e. you are rebasing a series of commits where some files had 'old'
line endings in the repository, but your current line ending setting
wants the line endings in those un-related, un-changed files to change
their line endings, and the rebase command can't cope with these
incidental differences?

Philip

Re: Files modified, even after: git reset --hard

[Chris Torek]

I don't have Windows and hence cannot test your exact
situation (which I understand to be:

    `core.autocrlf` set to `true`

and of course "using Windows"). I will repeat that I think
setting `core.autocrlf` to `true` is a bad idea, though.

In the meantime:

On Mon, Jul 26, 2021 at 3:31 AM Martin <git@mfriebe.de> wrote:
> origin  git://sourceware.org/git/binutils-gdb.git (fetch)

I cloned this repository so as to be able to obtain its commits.

> fsck had some dangling commits, I did a "git gc --aggressive".

(Dangling commits are irrelevant and unimportant: there's no need
to do this.)

> Switching to a far away commit
> git switch -f --detach master
> git rev-list master | wc -l
> 93917

The actual hash ID of the commit would be potentially useful here
(whether it's really useful would depend on whether I have or can
obtain that commit).

I do see that commit 16788ca9fd7352b2b20f4b11111283a8cd4212e2
removed various `Makefile.dos` files, including one mentioned
below (`bfd/Makefile.dos`).  That's the only file name I looked at
specifically for this kind of operation initially (I checked another
just now and it too went away in that commit).

> ... Then switching to  (this is on master branch)
> git switch -f --detach a362ee23634

The hash ID is all that matters: branch names are irrelevant here.

This commit contains a `bfd/Makefile.dos`.  The actual line
endings in this file are LF-only.

> git status --porcelain=v2
> 1 .M N... 100644 100644 100644 9e677a52ae690808165993a0f3f17ac49e3969df
> 9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos

This is also useful:

 * The `.M` means that the committed copy of the file matches
   the index copy, while the index copy does not match the working
   tree copy (presumably because the working tree copy has had
   its line endings changed to CRLF).
 * The `N...` means that this is not a submodule.
 * The three `100644`s mean that the file is mode 100644 in
   the commit, the index, and the working tree.
 * The two hash IDs are the blob object hash IDs for the committed
   and index copies of the file data; these match.
 * Last, we have the file's name in the index.

The same pattern seems to repeat for the remaining files.

These "Makefile.dos" files seem to be hardly ever touched,
in any commit, so these four files are likely not to be swapped
out by `git checkout` or `git switch` operations.

> My question is are there any plumbing commands, that could allow me to
> look further at it.

The simplest is to get the raw hash ID of any existing Git-ized object,
and use `git cat-file -p <hash>` on that object.  Since `git cat-file` has
no idea whether the data are to be treated as text or binary (in this
case where there is no path name and no `--textconv` flag),it just
writes out the raw text.  You must, however, make sure that the output
from this command is not mangled by some other tool (e.g., some
Windows editor "helpfully" changing LF-only to CRLF).

Note that `git ls-tree`, with the `-r` (recursive) option, allows you
to inspect the raw hash IDs of committed files by their path names.

Chris

Re: Files modified, even after: git reset --hard

[Martin]

On 26/07/2021 12:39, Philip Oakley wrote:
> On 26/07/2021 02:34, Martin wrote:
>> I figured that is the reason why they show modified.
>>
>> Not a problem. Until I am in the middle of a rebase, and i cannot run
>> (after a conflict)
>>    git rebase --continue
>>
>> The modified files are not part of the original series of commits.
>> they are just random files from somewhere else in the tree.
>> I can not reset/restore them.
>> So I must now "git add" files entirely unrelated to continue rebasing.
>> Well or apparently change my config for the duration of the rebase.
> 
> Is this 'mid-rebase' the core case for the 'Files modified' problem? -
> does it happen at other times (excepting maybe cherry-pick)
> 

So far yes.
Potentially, not yet tested, "git bisect" could be affected. That would 
also be a problem.
And yes, when rebase failed, cherry pick would also fail.


I did not see a "force" option for either of them, to ignore a dirty 
worktree.

> i.e. you are rebasing a series of commits where some files had 'old'
> line endings in the repository, but your current line ending setting
> wants the line endings in those un-related, un-changed files to change
> their line endings, and the rebase command can't cope with these
> incidental differences?
> 

I had a series of errors. Some of the errors, are indeed that I can not 
continue after resolving conflicts, because I can not clean the worktree.
(I guess I could set up a .git/info/gitattribute to mark the files as 
binary, and remove it again after the rebase. But that is not a 
desirable solution)


As a once off I also got an
    error: add_cacheinfo failed
One branch really could not be rebase by any means. But it was a single 
commit, so I copied the files.



I did "git replace" now for lots of those files.
But some commits had "modified files" that appear to have the correct 
line ending in the repro (git add --renormalize did not add changes to 
be committed / the file is added, but commit says 0 lines changed, and 
the issue remains).
So for those I have no way to get rid of yet.

Those none-line ending issues existed already before I started to git 
replace.
I did not replace commits, but only the blobs for the file in question 
(had to find several such blobs, per commit series)

Re: Files modified, even after: git reset --hard

[Martin]

On 26/07/2021 12:31, Martin wrote:
> On 26/07/2021 04:59, Chris Torek wrote:
>> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:
>>> Actually there is something else.
>>>
>>> If a file has line-endings that will change, then
>>>      git add --renormalize .
>>>      git commit -m foo
>>> will commit those files.
>>>
>>> But I am now also getting files, that show modified, but that can 
>>> not be
>>> committed renormalized (0 lines changed).
>>
>> I believe (but can't demonstrate) that this is a temporary condition.
>
> I now found one, that does not seem temporary at all...
>
> git remote -v
> origin  git://sourceware.org/git/binutils-gdb.git (fetch)
> origin  git://sourceware.org/git/binutils-gdb.git (push)
>
> Switching to a far away commit
> git switch -f --detach master
> git rev-list master | wc -l
> 93917
>
> git status shows no modified files
>
>
> Then switching to  (this is on master branch)
> git switch -f --detach a362ee23634
> git rev-list a362ee23634 | wc -l
> 4164
>
> git status --porcelain=v2
> 1 .M N... 100644 100644 100644 
> 9e677a52ae690808165993a0f3f17ac49e3969df 
> 9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos
> 1 .M N... 100644 100644 100644 
> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc 
> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc binutils/Makefile.dos
> 1 .M N... 100644 100644 100644 
> 1d9541c2f896842d1bafe68ccf0a51e291d66688 
> 1d9541c2f896842d1bafe68ccf0a51e291d66688 gas/Makefile.dos
> 1 .M N... 100644 100644 100644 
> 57fab985680ea151379069abe414bcb590cdd743 
> 57fab985680ea151379069abe414bcb590cdd743 ld/Makefile.dos

This seems an issue with gitforwindows.

I checked on 2 linux distros (git 2.25 and 2.31.1) and both are fine.

On Windows I set
core.autocrlf false
core.fscache false

Made a new clone, some (applied config as above), issue still present.

Tested with gitforwindow 2.32, 2.31.1 and 2.30.2

Re: Files modified, even after: git reset --hard

[Philip Oakley]

On 26/07/2021 14:57, Martin wrote:
> On 26/07/2021 12:31, Martin wrote:
>> On 26/07/2021 04:59, Chris Torek wrote:
>>> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:
>>>> Actually there is something else.
>>>>
>>>> If a file has line-endings that will change, then
>>>>      git add --renormalize .
>>>>      git commit -m foo
>>>> will commit those files.
>>>>
>>>> But I am now also getting files, that show modified, but that can
>>>> not be
>>>> committed renormalized (0 lines changed).
>>>
>>> I believe (but can't demonstrate) that this is a temporary condition.
>>
>> I now found one, that does not seem temporary at all...
>>
>> git remote -v
>> origin  git://sourceware.org/git/binutils-gdb.git (fetch)
>> origin  git://sourceware.org/git/binutils-gdb.git (push)
>>
>> Switching to a far away commit
>> git switch -f --detach master
>> git rev-list master | wc -l
>> 93917
>>
>> git status shows no modified files
>>
>>
>> Then switching to  (this is on master branch)
>> git switch -f --detach a362ee23634
>> git rev-list a362ee23634 | wc -l
>> 4164
>>
>> git status --porcelain=v2
>> 1 .M N... 100644 100644 100644
>> 9e677a52ae690808165993a0f3f17ac49e3969df
>> 9e677a52ae690808165993a0f3f17ac49e3969df bfd/Makefile.dos
>> 1 .M N... 100644 100644 100644
>> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc
>> ff24f19c0b6e0c7fb713c79e8f1765bc22fe7adc binutils/Makefile.dos
>> 1 .M N... 100644 100644 100644
>> 1d9541c2f896842d1bafe68ccf0a51e291d66688
>> 1d9541c2f896842d1bafe68ccf0a51e291d66688 gas/Makefile.dos
>> 1 .M N... 100644 100644 100644
>> 57fab985680ea151379069abe414bcb590cdd743
>> 57fab985680ea151379069abe414bcb590cdd743 ld/Makefile.dos
>
> This seems an issue with gitforwindows.

I doubt that it is Git-for-Windows (the .exe), but could easily believe
that the whole scenario is predicated on the Windows line endings and
personal .gitconfigs (some of which may be GfW norms) and the
repository's upstream norms (such as what is committed in the repo).

Given the repos/remotes listed your email in
https://public-inbox.org/git/f454bf5b-c5ff-4140-02a8-b02dcd35c6b9@mfriebe.de/,
What are all your differing configs between Windows & Linux testing and
their location (i.e. use ` git config --list --show-origin --show-scope`).

The comparison you suggest, I believe, is between
master:1b348b6b67dcdc120f5ac9dc409142b8ec2b4a09 (17 Apr 2021) and the
detached https://github.com/User4martin/binutils-gdb/commit/a362ee23634
"recording file death" (8 Dec 1992) (is that the commit you really
wanted to detach and compare with?)

The Github compare wasn't helpful as there are too many changes
[https://github.com/red-prig/binutils-gdb/compare/1b348b6..a362ee23634]

Just thinking about an MVCE.
>
> I checked on 2 linux distros (git 2.25 and 2.31.1) and both are fine.
>
> On Windows I set
> core.autocrlf false
> core.fscache false
>
> Made a new clone, some (applied config as above), issue still present.
>
> Tested with gitforwindow 2.32, 2.31.1 and 2.30.2
>
Philip

Re: Files modified, even after: git reset --hard

[Martin]

On 26/07/2021 04:59, Chris Torek wrote:
> On Sun, Jul 25, 2021 at 6:34 PM Martin <git@mfriebe.de> wrote:
>> Actually there is something else.
>>
>> If a file has line-endings that will change, then
>>      git add --renormalize .
>>      git commit -m foo
>> will commit those files.
>>
>> But I am now also getting files, that show modified, but that can not be
>> committed renormalized (0 lines changed).
> 
> I believe (but can't demonstrate) that this is a temporary condition.
> 
> Git has a number of cheats to make `git status` and other ops fast.
> These cheats *assume* that the committed files, the index copies
> of files, and the working tree copies of files all agree in terms of
> line endings as coordinated through `core.autocrlf` and `.gitattributes`
> settings.
> 
> When they *don't* agree, you get phantom differences.  Running
> commands like `git diff` show no differences because of these
> phantom states.  Eventually this clears up on its own when the
> cheats really *do* agree with the settings.  Changing the settings
> is what disturbs the cheats.
> 

Is it possible that those cheats also look at the "replaced" (rather 
than the "replacement") commit(s)?

I am pretty sure I have "git replace"d all blobs for some of the files, 
and yet they do get phantoms.

Re: Files modified, even after: git reset --hard

[Chris Torek]

On Mon, Jul 26, 2021 at 12:57 PM Martin <git@mfriebe.de> wrote:
> Is it possible that those cheats also look at the "replaced" (rather
> than the "replacement") commit(s)?

They look at `stat` system call (well, `lstat` *call*) data, not the
actual files.  This works better on Unix systems, where `lstat` is
a real system call, than it does on Windows, where it's faked up
from whatever Windows really stores as information about files.

> I am pretty sure I have "git replace"d all blobs for some of the files,
> and yet they do get phantoms.

The stat data are stored in Git's index.  It's the rebuilding of various
index entries that updates what Git uses to do a fast check of file
status.  (Note that the stat data on the index itself count as part of
the cheating; this gets tricky.  See [1].)

There is an article at [2] on how Windows implements `stat`. It's
probably out of date (says "VS2005").  It's interesting to me what's
wrong here: `st_dev` is made to mirror `st_rdev` but `st_dev` should
probably always just be zero, and they chose to attempt to store a
file *creation* time in `st_ctime`, when that is in `st_birthtime` in a
modern Unix-like system: the `ctime` field is the *inode change time*,
not a file creation time.  (Fortunately Git uses neither field.)

Chris

[1]: https://git-scm.com/docs/racy-git/en
[2]: https://web.archive.org/web/20101214222704/http://msdn.microsoft.com/en-us/library/14h5k7ff(v=vs.80).aspx

Re: Files modified, even after: git reset --hard

[Martin]

On 27/07/2021 00:03, Chris Torek wrote:
> On Mon, Jul 26, 2021 at 12:57 PM Martin <git@mfriebe.de> wrote:
>> Is it possible that those cheats also look at the "replaced" (rather
>> than the "replacement") commit(s)?
> 
> [1]: https://git-scm.com/docs/racy-git/en
> [2]: https://web.archive.org/web/20101214222704/http://msdn.microsoft.com/en-us/library/14h5k7ff(v=vs.80).aspx
> 

Thanks for the info. I still think there is something wrong.
So I made a simple testcase

https://github.com/User4martin/testrep.git
git fetch origin 'refs/replace/*:refs/replace/*'


All tested on linux. (Though git version 2.25.1)

git init test

FILE .gitattributes
foo -text

FILE foo
AAAAA

git add .gitattributes foo
git commit -m A

modify foo (real content modify)
add / commit -m B
modify foo (real content modify)
add / commit -m C

switch -d <A>
modify foo to be the same as commited to "C"
add / commit -m A2

git show
- get the old and new hash for the blob containing foo
git replace old new

Now commit A has the same content in foo as commit C
(we are currently at the detached commit A2)

git switch <C>
git status  // all fine

git switch <A>
git status
=> foo is modified
git diff
=> no diff


So I would say, it is not autorclf, or line endings in this case.

I do not know, if the above can be caused by "raciness".
But it appears to only(?) happens if a replace is in place.

So at this point my guess would be, that when the switch is done, at 
some point something looks at the original blob, even though it is meant 
to look at the replacement.