Bug in lineendings handling that prevents resetting checking out, rebasing etc

Bug in lineendings handling that prevents resetting checking out, rebasing etc

[Mr&Mrs D]

Hi all,

I maintain a python project you can clone from:

git@github.com:wrye-bash/wrye-bash.git

For reasons unknown git sees a particular file as changed
(Mopy/Docs/Bash Readme Template.html, sometimes others too). This file
was probably committed to the svn repository this git repo was created
from with CRLF line endings. When we moved to git we added a
gitattributes file (
https://github.com/wrye-bash/wrye-bash/blob/dev/.gitattributes ) and
this file was edited to explicitly state htms are text - all to no
avail. From time to time - on windows - as in when checking out an old
commit - git would see that file as changed. The only workaround that
worked for me was

    git rm -r . --cached -q && git reset --hard

For more details and discussion see this SO question I posted almost
five years ago:

https://stackoverflow.com/questions/21122094/git-line-endings-cant-stash-reset-and-now-cant-rebase-over-spurious-line-en

I used to work in windows and the bug was tolerable as there was that
workaround. Now I moved to mac and no workaround works anymore - we
have a special page on our wiki  with workarounds for this one btw:

https://github.com/wrye-bash/wrye-bash/wiki/%5Bgit%5D-Issues-with-line-endings-preventing-checking,-merge,-etc

Well after 5 years and countless hours trying to solve this I reach
out to you guys and girls - _this is a full-time bug in git line
endings handling_. When someone issues a git reset --hard this should
work no matter what - well it does not. So this bug may be really a
can of worms.

Please someone clone this repo on linux or mac - probably just cloning
will have the files appear as changed (by the way hitting refresh on
git gui I have different sets of files appear as changed). If not then

git checkout utumno-wip
git rebase -i dev

and then select a commit to edit should be enough to trigger this bug

Needless to say I am  well aware of things like `git add --renormalize
.` - but renormalizing is not the issue. The issue is that _files show
as changed and even a git reset --hard won't convince git that
nothing's changed_.

$ git reset --hard
HEAD is now at e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
$ git status
interactive rebase in progress; onto 02ae6f26
Last commands done (4 commands done):
   pick 3a39a0c0 Monkey patch for undecodable inis:
   pick e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
  (see more in file .git/rebase-merge/done)
Next commands to do (19 remaining commands):
   edit a3a7b237 Amend last commit and linefixes:  ΕΕΕΕ
   edit 432fd314 fFF handle empty or malformed inis
  (use "git rebase --edit-todo" to view and edit)
You are currently editing a commit while rebasing branch 'utumno-wip'
on '02ae6f26'.
  (use "git commit --amend" to amend the current commit)
  (use "git rebase --continue" once you are satisfied with your changes)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

modified:   Mopy/Docs/Bash Readme Template.html

Untracked files:
  (use "git add <file>..." to include in what will be committed)

.DS_Store
.idea.7z

no changes added to commit (use "git add" and/or "git commit -a")
$

I really hope someone here can debug this
Thanks!

Re: Bug in lineendings handling that prevents resetting checking out, rebasing etc

[John Passaro]

On Fri, Dec 14, 2018 at 4:08 PM Mr&Mrs D <the.ubik@gmail.com> wrote:
>
> Hi all,
>
> I maintain a python project you can clone from:
>
> git@github.com:wrye-bash/wrye-bash.git
>
> For reasons unknown git sees a particular file as changed
> (Mopy/Docs/Bash Readme Template.html, sometimes others too). This file
> was probably committed to the svn repository this git repo was created
> from with CRLF line endings. When we moved to git we added a
> gitattributes file (
> https://github.com/wrye-bash/wrye-bash/blob/dev/.gitattributes ) and
> this file was edited to explicitly state htms are text - all to no
> avail. From time to time - on windows - as in when checking out an old
> commit - git would see that file as changed. The only workaround that
> worked for me was
>
>     git rm -r . --cached -q && git reset --hard
>
> For more details and discussion see this SO question I posted almost
> five years ago:
>
> https://stackoverflow.com/questions/21122094/git-line-endings-cant-stash-reset-and-now-cant-rebase-over-spurious-line-en
>
> I used to work in windows and the bug was tolerable as there was that
> workaround. Now I moved to mac and no workaround works anymore - we
> have a special page on our wiki  with workarounds for this one btw:
>
> https://github.com/wrye-bash/wrye-bash/wiki/%5Bgit%5D-Issues-with-line-endings-preventing-checking,-merge,-etc
>
> Well after 5 years and countless hours trying to solve this I reach
> out to you guys and girls - _this is a full-time bug in git line
> endings handling_. When someone issues a git reset --hard this should
> work no matter what - well it does not. So this bug may be really a
> can of worms.
>
> Please someone clone this repo on linux or mac - probably just cloning
> will have the files appear as changed (by the way hitting refresh on
> git gui I have different sets of files appear as changed). If not then
>
> git checkout utumno-wip
> git rebase -i dev
>
> and then select a commit to edit should be enough to trigger this bug

Does not reproduce on git 2.20.0 (mac high sierra fwiw). What version of git
are you using?
>
> Needless to say I am  well aware of things like `git add --renormalize
> .` - but renormalizing is not the issue. The issue is that _files show
> as changed and even a git reset --hard won't convince git that
> nothing's changed_.
>
> $ git reset --hard
> HEAD is now at e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> $ git status
> interactive rebase in progress; onto 02ae6f26
> Last commands done (4 commands done):
>    pick 3a39a0c0 Monkey patch for undecodable inis:
>    pick e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
>   (see more in file .git/rebase-merge/done)
> Next commands to do (19 remaining commands):
>    edit a3a7b237 Amend last commit and linefixes:  ΕΕΕΕ
>    edit 432fd314 fFF handle empty or malformed inis
>   (use "git rebase --edit-todo" to view and edit)
> You are currently editing a commit while rebasing branch 'utumno-wip'
> on '02ae6f26'.
>   (use "git commit --amend" to amend the current commit)
>   (use "git rebase --continue" once you are satisfied with your changes)
>
> Changes not staged for commit:
>   (use "git add <file>..." to update what will be committed)
>   (use "git checkout -- <file>..." to discard changes in working directory)
>
> modified:   Mopy/Docs/Bash Readme Template.html
>
> Untracked files:
>   (use "git add <file>..." to include in what will be committed)
>
> .DS_Store
> .idea.7z
>
> no changes added to commit (use "git add" and/or "git commit -a")
> $
>
> I really hope someone here can debug this
> Thanks!

Re: Bug in lineendings handling that prevents resetting checking out, rebasing etc

[Torsten Bögershausen]

On Fri, Dec 14, 2018 at 04:04:15PM -0500, Mr&Mrs D wrote:
> Hi all,
> 
> I maintain a python project you can clone from:
> 
> git@github.com:wrye-bash/wrye-bash.git
> 
> For reasons unknown git sees a particular file as changed
> (Mopy/Docs/Bash Readme Template.html, sometimes others too). This file
> was probably committed to the svn repository this git repo was created
> from with CRLF line endings. When we moved to git we added a
> gitattributes file (
> https://github.com/wrye-bash/wrye-bash/blob/dev/.gitattributes ) and
> this file was edited to explicitly state htms are text - all to no
> avail. From time to time - on windows - as in when checking out an old
> commit - git would see that file as changed. The only workaround that
> worked for me was
> 
>     git rm -r . --cached -q && git reset --hard
> 
> For more details and discussion see this SO question I posted almost
> five years ago:
> 
> https://stackoverflow.com/questions/21122094/git-line-endings-cant-stash-reset-and-now-cant-rebase-over-spurious-line-en
> 
> I used to work in windows and the bug was tolerable as there was that
> workaround. Now I moved to mac and no workaround works anymore - we
> have a special page on our wiki  with workarounds for this one btw:
> 
> https://github.com/wrye-bash/wrye-bash/wiki/%5Bgit%5D-Issues-with-line-endings-preventing-checking,-merge,-etc
> 
> Well after 5 years and countless hours trying to solve this I reach
> out to you guys and girls - _this is a full-time bug in git line
> endings handling_. When someone issues a git reset --hard this should
> work no matter what - well it does not. So this bug may be really a
> can of worms.
> 
> Please someone clone this repo on linux or mac - probably just cloning
> will have the files appear as changed (by the way hitting refresh on
> git gui I have different sets of files appear as changed). If not then
> 
> git checkout utumno-wip
Thet commit is -excuse me if that sounds too harsh- is messed up.
git status says
modified:   Mopy/Docs/Bash Readme Template.html

And if I dig into the EOL stuff, I run
git ls-files --eol | grep  Readme | less

And find a contradiction here:
i/crlf  w/crlf  attr/text               Mopy/Docs/Bash Readme Template.html

The attributes say "text" and the file has CRLF "in the repo",
(techically speaking in the index) and that is an "illegal" condition
in the repo, and not a bug in Git.
I didn't try the rebase as such, sice the first problem needs
to be fixed, before we try to move on.

So, the old commits are problematic/illegal and they are as they are.
Such a commit can not be fixed, whenever somebody checks it out,
there will be a problem (or two, or none, depending on the timing,
the file system...)

We can not fix commits like b1acc012878c9fdd8b4ad610ce7eae0dcbcbcab0.
We can make new commits, and fix them.

We can fix one branch, and other branches, and merge them together.
But rebase seems to be problamatic, at least to me.
What exactly do you want to do?

Can we agree to do a merge of 2 branches?
Then I can possibly help you out.





> git rebase -i dev
> 
> and then select a commit to edit should be enough to trigger this bug
> 
> Needless to say I am  well aware of things like `git add --renormalize
> .` - but renormalizing is not the issue. The issue is that _files show
> as changed and even a git reset --hard won't convince git that
> nothing's changed_.
> 
> $ git reset --hard
> HEAD is now at e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> $ git status
> interactive rebase in progress; onto 02ae6f26
> Last commands done (4 commands done):
>    pick 3a39a0c0 Monkey patch for undecodable inis:
>    pick e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
>   (see more in file .git/rebase-merge/done)
> Next commands to do (19 remaining commands):
>    edit a3a7b237 Amend last commit and linefixes:  ΕΕΕΕ
>    edit 432fd314 fFF handle empty or malformed inis
>   (use "git rebase --edit-todo" to view and edit)
> You are currently editing a commit while rebasing branch 'utumno-wip'
> on '02ae6f26'.
>   (use "git commit --amend" to amend the current commit)
>   (use "git rebase --continue" once you are satisfied with your changes)
> 
> Changes not staged for commit:
>   (use "git add <file>..." to update what will be committed)
>   (use "git checkout -- <file>..." to discard changes in working directory)
> 
> modified:   Mopy/Docs/Bash Readme Template.html
> 
> Untracked files:
>   (use "git add <file>..." to include in what will be committed)
> 
> .DS_Store
> .idea.7z
> 
> no changes added to commit (use "git add" and/or "git commit -a")
> $
> 
> I really hope someone here can debug this
> Thanks!

Re: Bug in lineendings handling that prevents resetting checking out, rebasing etc

[Mr&Mrs D]

$ git --version
git version 2.19.2

Mac os mojave

Hmm the latest version here: https://git-scm.com/download/mac seems to
be this one - where do I get 2.20?

Thanks!
On Fri, Dec 14, 2018 at 4:22 PM John Passaro <john.a.passaro@gmail.com> wrote:
>
> On Fri, Dec 14, 2018 at 4:08 PM Mr&Mrs D <the.ubik@gmail.com> wrote:
> >
> > Hi all,
> >
> > I maintain a python project you can clone from:
> >
> > git@github.com:wrye-bash/wrye-bash.git
> >
> > For reasons unknown git sees a particular file as changed
> > (Mopy/Docs/Bash Readme Template.html, sometimes others too). This file
> > was probably committed to the svn repository this git repo was created
> > from with CRLF line endings. When we moved to git we added a
> > gitattributes file (
> > https://github.com/wrye-bash/wrye-bash/blob/dev/.gitattributes ) and
> > this file was edited to explicitly state htms are text - all to no
> > avail. From time to time - on windows - as in when checking out an old
> > commit - git would see that file as changed. The only workaround that
> > worked for me was
> >
> >     git rm -r . --cached -q && git reset --hard
> >
> > For more details and discussion see this SO question I posted almost
> > five years ago:
> >
> > https://stackoverflow.com/questions/21122094/git-line-endings-cant-stash-reset-and-now-cant-rebase-over-spurious-line-en
> >
> > I used to work in windows and the bug was tolerable as there was that
> > workaround. Now I moved to mac and no workaround works anymore - we
> > have a special page on our wiki  with workarounds for this one btw:
> >
> > https://github.com/wrye-bash/wrye-bash/wiki/%5Bgit%5D-Issues-with-line-endings-preventing-checking,-merge,-etc
> >
> > Well after 5 years and countless hours trying to solve this I reach
> > out to you guys and girls - _this is a full-time bug in git line
> > endings handling_. When someone issues a git reset --hard this should
> > work no matter what - well it does not. So this bug may be really a
> > can of worms.
> >
> > Please someone clone this repo on linux or mac - probably just cloning
> > will have the files appear as changed (by the way hitting refresh on
> > git gui I have different sets of files appear as changed). If not then
> >
> > git checkout utumno-wip
> > git rebase -i dev
> >
> > and then select a commit to edit should be enough to trigger this bug
>
> Does not reproduce on git 2.20.0 (mac high sierra fwiw). What version of git
> are you using?
> >
> > Needless to say I am  well aware of things like `git add --renormalize
> > .` - but renormalizing is not the issue. The issue is that _files show
> > as changed and even a git reset --hard won't convince git that
> > nothing's changed_.
> >
> > $ git reset --hard
> > HEAD is now at e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> > $ git status
> > interactive rebase in progress; onto 02ae6f26
> > Last commands done (4 commands done):
> >    pick 3a39a0c0 Monkey patch for undecodable inis:
> >    pick e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> >   (see more in file .git/rebase-merge/done)
> > Next commands to do (19 remaining commands):
> >    edit a3a7b237 Amend last commit and linefixes:  ΕΕΕΕ
> >    edit 432fd314 fFF handle empty or malformed inis
> >   (use "git rebase --edit-todo" to view and edit)
> > You are currently editing a commit while rebasing branch 'utumno-wip'
> > on '02ae6f26'.
> >   (use "git commit --amend" to amend the current commit)
> >   (use "git rebase --continue" once you are satisfied with your changes)
> >
> > Changes not staged for commit:
> >   (use "git add <file>..." to update what will be committed)
> >   (use "git checkout -- <file>..." to discard changes in working directory)
> >
> > modified:   Mopy/Docs/Bash Readme Template.html
> >
> > Untracked files:
> >   (use "git add <file>..." to include in what will be committed)
> >
> > .DS_Store
> > .idea.7z
> >
> > no changes added to commit (use "git add" and/or "git commit -a")
> > $
> >
> > I really hope someone here can debug this
> > Thanks!

Re: Bug in lineendings handling that prevents resetting checking out, rebasing etc

[Mr&Mrs D]

Thanks for looking to it - git attributes was added in
4b0863a8b834c5804fd3a568ed56ff85b27acdeb

The file in question was added in 17ca75f0a8c25f321f2b63bc3b9c065ff91adc23

So you mean to say that because a gitattributes was added after the
fact this resulted in an illegal state?

But _shouldn't git reset --hard work anyway?_ That's the buggy part.

As for fixing it - not sure what is the best course of action here.
probably issuing `git add --renormalize .` and committing that to the
stable (dev) branch will fix this for future checkouts/rebases but
IIUC won't do nothing for older commits - so checking out a commit
before the fix one, ghit will see this file as changed and then
completely refuse to go back to another branch

This seems a bug - as illegal as the state of the file is, shouldn't
git reset always work?

Thanks! (will be out for a bit but really looking forward to your replies)
On Fri, Dec 14, 2018 at 4:32 PM Torsten Bögershausen <tboegi@web.de> wrote:
>
> On Fri, Dec 14, 2018 at 04:04:15PM -0500, Mr&Mrs D wrote:
> > Hi all,
> >
> > I maintain a python project you can clone from:
> >
> > git@github.com:wrye-bash/wrye-bash.git
> >
> > For reasons unknown git sees a particular file as changed
> > (Mopy/Docs/Bash Readme Template.html, sometimes others too). This file
> > was probably committed to the svn repository this git repo was created
> > from with CRLF line endings. When we moved to git we added a
> > gitattributes file (
> > https://github.com/wrye-bash/wrye-bash/blob/dev/.gitattributes ) and
> > this file was edited to explicitly state htms are text - all to no
> > avail. From time to time - on windows - as in when checking out an old
> > commit - git would see that file as changed. The only workaround that
> > worked for me was
> >
> >     git rm -r . --cached -q && git reset --hard
> >
> > For more details and discussion see this SO question I posted almost
> > five years ago:
> >
> > https://stackoverflow.com/questions/21122094/git-line-endings-cant-stash-reset-and-now-cant-rebase-over-spurious-line-en
> >
> > I used to work in windows and the bug was tolerable as there was that
> > workaround. Now I moved to mac and no workaround works anymore - we
> > have a special page on our wiki  with workarounds for this one btw:
> >
> > https://github.com/wrye-bash/wrye-bash/wiki/%5Bgit%5D-Issues-with-line-endings-preventing-checking,-merge,-etc
> >
> > Well after 5 years and countless hours trying to solve this I reach
> > out to you guys and girls - _this is a full-time bug in git line
> > endings handling_. When someone issues a git reset --hard this should
> > work no matter what - well it does not. So this bug may be really a
> > can of worms.
> >
> > Please someone clone this repo on linux or mac - probably just cloning
> > will have the files appear as changed (by the way hitting refresh on
> > git gui I have different sets of files appear as changed). If not then
> >
> > git checkout utumno-wip
> Thet commit is -excuse me if that sounds too harsh- is messed up.
> git status says
> modified:   Mopy/Docs/Bash Readme Template.html
>
> And if I dig into the EOL stuff, I run
> git ls-files --eol | grep  Readme | less
>
> And find a contradiction here:
> i/crlf  w/crlf  attr/text               Mopy/Docs/Bash Readme Template.html
>
> The attributes say "text" and the file has CRLF "in the repo",
> (techically speaking in the index) and that is an "illegal" condition
> in the repo, and not a bug in Git.
> I didn't try the rebase as such, sice the first problem needs
> to be fixed, before we try to move on.
>
> So, the old commits are problematic/illegal and they are as they are.
> Such a commit can not be fixed, whenever somebody checks it out,
> there will be a problem (or two, or none, depending on the timing,
> the file system...)
>
> We can not fix commits like b1acc012878c9fdd8b4ad610ce7eae0dcbcbcab0.
> We can make new commits, and fix them.
>
> We can fix one branch, and other branches, and merge them together.
> But rebase seems to be problamatic, at least to me.
> What exactly do you want to do?
>
> Can we agree to do a merge of 2 branches?
> Then I can possibly help you out.
>
>
>
>
>
> > git rebase -i dev
> >
> > and then select a commit to edit should be enough to trigger this bug
> >
> > Needless to say I am  well aware of things like `git add --renormalize
> > .` - but renormalizing is not the issue. The issue is that _files show
> > as changed and even a git reset --hard won't convince git that
> > nothing's changed_.
> >
> > $ git reset --hard
> > HEAD is now at e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> > $ git status
> > interactive rebase in progress; onto 02ae6f26
> > Last commands done (4 commands done):
> >    pick 3a39a0c0 Monkey patch for undecodable inis:
> >    pick e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> >   (see more in file .git/rebase-merge/done)
> > Next commands to do (19 remaining commands):
> >    edit a3a7b237 Amend last commit and linefixes:  ΕΕΕΕ
> >    edit 432fd314 fFF handle empty or malformed inis
> >   (use "git rebase --edit-todo" to view and edit)
> > You are currently editing a commit while rebasing branch 'utumno-wip'
> > on '02ae6f26'.
> >   (use "git commit --amend" to amend the current commit)
> >   (use "git rebase --continue" once you are satisfied with your changes)
> >
> > Changes not staged for commit:
> >   (use "git add <file>..." to update what will be committed)
> >   (use "git checkout -- <file>..." to discard changes in working directory)
> >
> > modified:   Mopy/Docs/Bash Readme Template.html
> >
> > Untracked files:
> >   (use "git add <file>..." to include in what will be committed)
> >
> > .DS_Store
> > .idea.7z
> >
> > no changes added to commit (use "git add" and/or "git commit -a")
> > $
> >
> > I really hope someone here can debug this
> > Thanks!

Re: Bug in lineendings handling that prevents resetting checking out, rebasing etc

[Torsten Bögershausen]

On Fri, Dec 14, 2018 at 04:51:54PM -0500, Mr&Mrs D wrote:
> Thanks for looking to it - git attributes was added in
> 4b0863a8b834c5804fd3a568ed56ff85b27acdeb
> 
> The file in question was added in 17ca75f0a8c25f321f2b63bc3b9c065ff91adc23
> 
> So you mean to say that because a gitattributes was added after the
> fact this resulted in an illegal state?
> 
> But _shouldn't git reset --hard work anyway?_ That's the buggy part.

Yes and no.
If I look at the dev branch:
commit 02ae6f264f340137b8b41ba6953e2a4f962c222e (HEAD, origin/dev, origin/HEAD, dev)

Now we can ask Git, why a file is modified:

git ls-files --eol | grep "Mopy/Docs/Bash eadme Template.html"
i/crlf  w/crlf  attr/text               Mopy/Docs/Bash Readme Template.html

Now we have 2 conflicting "messages" to Git:
a) "Mopy/Docs/Bash Readme Template.html" has the attribute "text"
b) "Mopy/Docs/Bash Readme Template.html" has been commited with CRLF.

Git itself can not resolve this conflict.
Either you normalize the repo (in this case only 1 file), other commits have 4 files
that needs to be normalized.
Or you change the attribute into "text=auto".

That decision is up to the user.

> 
> As for fixing it - not sure what is the best course of action here.
> probably issuing `git add --renormalize .` and committing that to the
> stable (dev) branch will fix this for future checkouts/rebases but
> IIUC won't do nothing for older commits - so checking out a commit
> before the fix one, ghit will see this file as changed and then
> completely refuse to go back to another branch
> 
> This seems a bug - as illegal as the state of the file is, shouldn't
> git reset always work?
> 
> Thanks! (will be out for a bit but really looking forward to your replies)
> On Fri, Dec 14, 2018 at 4:32 PM Torsten Bögershausen <tboegi@web.de> wrote:
> >
> > On Fri, Dec 14, 2018 at 04:04:15PM -0500, Mr&Mrs D wrote:
> > > Hi all,
> > >
> > > I maintain a python project you can clone from:
> > >
> > > git@github.com:wrye-bash/wrye-bash.git
> > >
> > > For reasons unknown git sees a particular file as changed
> > > (Mopy/Docs/Bash Readme Template.html, sometimes others too). This file
> > > was probably committed to the svn repository this git repo was created
> > > from with CRLF line endings. When we moved to git we added a
> > > gitattributes file (
> > > https://github.com/wrye-bash/wrye-bash/blob/dev/.gitattributes ) and
> > > this file was edited to explicitly state htms are text - all to no
> > > avail. From time to time - on windows - as in when checking out an old
> > > commit - git would see that file as changed. The only workaround that
> > > worked for me was
> > >
> > >     git rm -r . --cached -q && git reset --hard
> > >
> > > For more details and discussion see this SO question I posted almost
> > > five years ago:
> > >
> > > https://stackoverflow.com/questions/21122094/git-line-endings-cant-stash-reset-and-now-cant-rebase-over-spurious-line-en
> > >
> > > I used to work in windows and the bug was tolerable as there was that
> > > workaround. Now I moved to mac and no workaround works anymore - we
> > > have a special page on our wiki  with workarounds for this one btw:
> > >
> > > https://github.com/wrye-bash/wrye-bash/wiki/%5Bgit%5D-Issues-with-line-endings-preventing-checking,-merge,-etc
> > >
> > > Well after 5 years and countless hours trying to solve this I reach
> > > out to you guys and girls - _this is a full-time bug in git line
> > > endings handling_. When someone issues a git reset --hard this should
> > > work no matter what - well it does not. So this bug may be really a
> > > can of worms.
> > >
> > > Please someone clone this repo on linux or mac - probably just cloning
> > > will have the files appear as changed (by the way hitting refresh on
> > > git gui I have different sets of files appear as changed). If not then
> > >
> > > git checkout utumno-wip
> > Thet commit is -excuse me if that sounds too harsh- is messed up.
> > git status says
> > modified:   Mopy/Docs/Bash Readme Template.html
> >
> > And if I dig into the EOL stuff, I run
> > git ls-files --eol | grep  Readme | less
> >
> > And find a contradiction here:
> > i/crlf  w/crlf  attr/text               Mopy/Docs/Bash Readme Template.html
> >
> > The attributes say "text" and the file has CRLF "in the repo",
> > (techically speaking in the index) and that is an "illegal" condition
> > in the repo, and not a bug in Git.
> > I didn't try the rebase as such, sice the first problem needs
> > to be fixed, before we try to move on.
> >
> > So, the old commits are problematic/illegal and they are as they are.
> > Such a commit can not be fixed, whenever somebody checks it out,
> > there will be a problem (or two, or none, depending on the timing,
> > the file system...)
> >
> > We can not fix commits like b1acc012878c9fdd8b4ad610ce7eae0dcbcbcab0.
> > We can make new commits, and fix them.
> >
> > We can fix one branch, and other branches, and merge them together.
> > But rebase seems to be problamatic, at least to me.
> > What exactly do you want to do?
> >
> > Can we agree to do a merge of 2 branches?
> > Then I can possibly help you out.
> >
> >
> >
> >
> >
> > > git rebase -i dev
> > >
> > > and then select a commit to edit should be enough to trigger this bug
> > >
> > > Needless to say I am  well aware of things like `git add --renormalize
> > > .` - but renormalizing is not the issue. The issue is that _files show
> > > as changed and even a git reset --hard won't convince git that
> > > nothing's changed_.
> > >
> > > $ git reset --hard
> > > HEAD is now at e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> > > $ git status
> > > interactive rebase in progress; onto 02ae6f26
> > > Last commands done (4 commands done):
> > >    pick 3a39a0c0 Monkey patch for undecodable inis:
> > >    pick e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> > >   (see more in file .git/rebase-merge/done)
> > > Next commands to do (19 remaining commands):
> > >    edit a3a7b237 Amend last commit and linefixes:  ΕΕΕΕ
> > >    edit 432fd314 fFF handle empty or malformed inis
> > >   (use "git rebase --edit-todo" to view and edit)
> > > You are currently editing a commit while rebasing branch 'utumno-wip'
> > > on '02ae6f26'.
> > >   (use "git commit --amend" to amend the current commit)
> > >   (use "git rebase --continue" once you are satisfied with your changes)
> > >
> > > Changes not staged for commit:
> > >   (use "git add <file>..." to update what will be committed)
> > >   (use "git checkout -- <file>..." to discard changes in working directory)
> > >
> > > modified:   Mopy/Docs/Bash Readme Template.html
> > >
> > > Untracked files:
> > >   (use "git add <file>..." to include in what will be committed)
> > >
> > > .DS_Store
> > > .idea.7z
> > >
> > > no changes added to commit (use "git add" and/or "git commit -a")
> > > $
> > >
> > > I really hope someone here can debug this
> > > Thanks!

Re: Bug in lineendings handling that prevents resetting checking out, rebasing etc

[Mr&Mrs D]

Thanks again!

Whichever path I take all the intermediate commits between the fix
commit and the one that added the gitattributes file (so marked that
file as text) will be plagued by this - checking them out on linux or
macos will show that particular file as modified. Is my understanding
correct?
It still seems to me that at least the operating system should not
matter here - I did not encounter this on windows and when I did (as
described in the SO post) `git rm -r . --cached -q && git reset
--hard` would reset the branch.

Is there a way I can add this file to local .git/info/attributes so
that I overwrite the .gitattributes (see
https://stackoverflow.com/a/33715791/281545 ) ?

I tried adding the line:

$ cat .git/info/attributes
Mopy/Docs/Bash Readme Template.html -text

but does nothing. The idea would be to add the commit fixing the line
endings on the stable branch and also instruct collaborators to edit
their .git/info/attributes files so when checking out older commits
the file does not show as changed.

Speaking of line endings fix issuing:

(dev) $ git add --renormalize .
(dev) $ git status
On branch dev
Your branch is up to date with 'origin/dev'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

modified:   Mopy/Docs/Bash Readme Template.html
modified:   Mopy/Docs/Bash Readme Template.txt
modified:   Mopy/bash/db/Oblivion_ids.pkl
modified:   Mopy/bash/db/Skyrim_ids.pkl

Untracked files:
...

adds 4 files as seen (?)

Thanks again and sorry for the late reply
On Fri, Dec 14, 2018 at 11:43 PM Torsten Bögershausen <tboegi@web.de> wrote:
>
> On Fri, Dec 14, 2018 at 04:51:54PM -0500, Mr&Mrs D wrote:
> > Thanks for looking to it - git attributes was added in
> > 4b0863a8b834c5804fd3a568ed56ff85b27acdeb
> >
> > The file in question was added in 17ca75f0a8c25f321f2b63bc3b9c065ff91adc23
> >
> > So you mean to say that because a gitattributes was added after the
> > fact this resulted in an illegal state?
> >
> > But _shouldn't git reset --hard work anyway?_ That's the buggy part.
>
> Yes and no.
> If I look at the dev branch:
> commit 02ae6f264f340137b8b41ba6953e2a4f962c222e (HEAD, origin/dev, origin/HEAD, dev)
>
> Now we can ask Git, why a file is modified:
>
> git ls-files --eol | grep "Mopy/Docs/Bash eadme Template.html"
> i/crlf  w/crlf  attr/text               Mopy/Docs/Bash Readme Template.html
>
> Now we have 2 conflicting "messages" to Git:
> a) "Mopy/Docs/Bash Readme Template.html" has the attribute "text"
> b) "Mopy/Docs/Bash Readme Template.html" has been commited with CRLF.
>
> Git itself can not resolve this conflict.
> Either you normalize the repo (in this case only 1 file), other commits have 4 files
> that needs to be normalized.
> Or you change the attribute into "text=auto".
>
> That decision is up to the user.
>
> >
> > As for fixing it - not sure what is the best course of action here.
> > probably issuing `git add --renormalize .` and committing that to the
> > stable (dev) branch will fix this for future checkouts/rebases but
> > IIUC won't do nothing for older commits - so checking out a commit
> > before the fix one, ghit will see this file as changed and then
> > completely refuse to go back to another branch
> >
> > This seems a bug - as illegal as the state of the file is, shouldn't
> > git reset always work?
> >
> > Thanks! (will be out for a bit but really looking forward to your replies)
> > On Fri, Dec 14, 2018 at 4:32 PM Torsten Bögershausen <tboegi@web.de> wrote:
> > >
> > > On Fri, Dec 14, 2018 at 04:04:15PM -0500, Mr&Mrs D wrote:
> > > > Hi all,
> > > >
> > > > I maintain a python project you can clone from:
> > > >
> > > > git@github.com:wrye-bash/wrye-bash.git
> > > >
> > > > For reasons unknown git sees a particular file as changed
> > > > (Mopy/Docs/Bash Readme Template.html, sometimes others too). This file
> > > > was probably committed to the svn repository this git repo was created
> > > > from with CRLF line endings. When we moved to git we added a
> > > > gitattributes file (
> > > > https://github.com/wrye-bash/wrye-bash/blob/dev/.gitattributes ) and
> > > > this file was edited to explicitly state htms are text - all to no
> > > > avail. From time to time - on windows - as in when checking out an old
> > > > commit - git would see that file as changed. The only workaround that
> > > > worked for me was
> > > >
> > > >     git rm -r . --cached -q && git reset --hard
> > > >
> > > > For more details and discussion see this SO question I posted almost
> > > > five years ago:
> > > >
> > > > https://stackoverflow.com/questions/21122094/git-line-endings-cant-stash-reset-and-now-cant-rebase-over-spurious-line-en
> > > >
> > > > I used to work in windows and the bug was tolerable as there was that
> > > > workaround. Now I moved to mac and no workaround works anymore - we
> > > > have a special page on our wiki  with workarounds for this one btw:
> > > >
> > > > https://github.com/wrye-bash/wrye-bash/wiki/%5Bgit%5D-Issues-with-line-endings-preventing-checking,-merge,-etc
> > > >
> > > > Well after 5 years and countless hours trying to solve this I reach
> > > > out to you guys and girls - _this is a full-time bug in git line
> > > > endings handling_. When someone issues a git reset --hard this should
> > > > work no matter what - well it does not. So this bug may be really a
> > > > can of worms.
> > > >
> > > > Please someone clone this repo on linux or mac - probably just cloning
> > > > will have the files appear as changed (by the way hitting refresh on
> > > > git gui I have different sets of files appear as changed). If not then
> > > >
> > > > git checkout utumno-wip
> > > Thet commit is -excuse me if that sounds too harsh- is messed up.
> > > git status says
> > > modified:   Mopy/Docs/Bash Readme Template.html
> > >
> > > And if I dig into the EOL stuff, I run
> > > git ls-files --eol | grep  Readme | less
> > >
> > > And find a contradiction here:
> > > i/crlf  w/crlf  attr/text               Mopy/Docs/Bash Readme Template.html
> > >
> > > The attributes say "text" and the file has CRLF "in the repo",
> > > (techically speaking in the index) and that is an "illegal" condition
> > > in the repo, and not a bug in Git.
> > > I didn't try the rebase as such, sice the first problem needs
> > > to be fixed, before we try to move on.
> > >
> > > So, the old commits are problematic/illegal and they are as they are.
> > > Such a commit can not be fixed, whenever somebody checks it out,
> > > there will be a problem (or two, or none, depending on the timing,
> > > the file system...)
> > >
> > > We can not fix commits like b1acc012878c9fdd8b4ad610ce7eae0dcbcbcab0.
> > > We can make new commits, and fix them.
> > >
> > > We can fix one branch, and other branches, and merge them together.
> > > But rebase seems to be problamatic, at least to me.
> > > What exactly do you want to do?
> > >
> > > Can we agree to do a merge of 2 branches?
> > > Then I can possibly help you out.
> > >
> > >
> > >
> > >
> > >
> > > > git rebase -i dev
> > > >
> > > > and then select a commit to edit should be enough to trigger this bug
> > > >
> > > > Needless to say I am  well aware of things like `git add --renormalize
> > > > .` - but renormalizing is not the issue. The issue is that _files show
> > > > as changed and even a git reset --hard won't convince git that
> > > > nothing's changed_.
> > > >
> > > > $ git reset --hard
> > > > HEAD is now at e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> > > > $ git status
> > > > interactive rebase in progress; onto 02ae6f26
> > > > Last commands done (4 commands done):
> > > >    pick 3a39a0c0 Monkey patch for undecodable inis:
> > > >    pick e5c16790 Wip proper handling of ini tweaks encoding - TODOs:
> > > >   (see more in file .git/rebase-merge/done)
> > > > Next commands to do (19 remaining commands):
> > > >    edit a3a7b237 Amend last commit and linefixes:  ΕΕΕΕ
> > > >    edit 432fd314 fFF handle empty or malformed inis
> > > >   (use "git rebase --edit-todo" to view and edit)
> > > > You are currently editing a commit while rebasing branch 'utumno-wip'
> > > > on '02ae6f26'.
> > > >   (use "git commit --amend" to amend the current commit)
> > > >   (use "git rebase --continue" once you are satisfied with your changes)
> > > >
> > > > Changes not staged for commit:
> > > >   (use "git add <file>..." to update what will be committed)
> > > >   (use "git checkout -- <file>..." to discard changes in working directory)
> > > >
> > > > modified:   Mopy/Docs/Bash Readme Template.html
> > > >
> > > > Untracked files:
> > > >   (use "git add <file>..." to include in what will be committed)
> > > >
> > > > .DS_Store
> > > > .idea.7z
> > > >
> > > > no changes added to commit (use "git add" and/or "git commit -a")
> > > > $
> > > >
> > > > I really hope someone here can debug this
> > > > Thanks!