Merge conflicts in .gitattributes can cause trouble

Merge conflicts in .gitattributes can cause trouble

[Lars Schneider]

Hi,


If there is a conflict in the .gitattributes during a merge then it looks 
like as if the attributes are not applied (which kind of makes sense as Git 
would not know what to do). As a result Git can treat e.g. binary files 
as text and they can end up with changed line endings in the working tree. 
After resolving the conflict in .gitattributes all files would be marked 
as binary, again, and the user can easily commit the wrongly changed line 
endings.

Consider this script on Windows:

$ git init .
$ touch first.commit
$ git add .
$ git commit -m "first commit"

$ git checkout -b branch
$ printf "*.bin binary\n" >> .gitattributes
$ git add .
$ git commit -m "tracking *.bin files"

$ git checkout master
$ printf "binary\ndata\n" > file.dat # <-- Unix line ending!
$ printf "*.dat binary\n" >> .gitattributes # <-- Tell Git to keep Unix line ending!
$ git add .
$ git commit -m "tracking *.dat files"
$ git cat-file -p :file.dat | od -c
0000000   b   i   n   a   r   y  \n   d   a   t   a  \n 
                                ^^^^                ^^^^  <-- Correct!
$ git checkout branch
$ git merge master # <-- Causes merge conflict!
$ printf "*.bin binary\n*.dat binary\n" > .gitattributes # <-- Fix merge conflict!
$ git add .
$ git commit -m "merged"
$ git cat-file -p :file.dat | od -c
0000000   b   i   n   a   r   y  \r  \n   d   a   t   a  \r  \n
                                ^^^^^^^^                ^^^^^^^^  <-- Wrong!

Possible solutions:

1. We could print an appropriate warning if we detect a merge conflict 
   in .gitattributes

2. We could disable all line ending conversions in case of a merge conflict
   (I am not exactly sure about all the implications, though)

3. We could salvage what we could of the .gitattributes file, 
   perhaps by using the version from HEAD (or more likely, the ours stage of
   the index) -- suggested by Peff on the related GitHub issue mentioned below

Thoughts?

Thanks,
Lars


PS: I noticed that behavior while working with Git LFS and started a discussion
about it here: https://github.com/github/git-lfs/issues/1544 

Re: Merge conflicts in .gitattributes can cause trouble

[Duy Nguyen]

On Tue, Oct 4, 2016 at 5:19 PM, Lars Schneider <larsxschneider@gmail.com> wrote:
> Hi,
>
>
> If there is a conflict in the .gitattributes during a merge then it looks
> like as if the attributes are not applied (which kind of makes sense as Git
> would not know what to do). As a result Git can treat e.g. binary files
> as text and they can end up with changed line endings in the working tree.
> After resolving the conflict in .gitattributes all files would be marked
> as binary, again, and the user can easily commit the wrongly changed line
> endings.
>
> Consider this script on Windows:
>
> $ git init .
> $ touch first.commit
> $ git add .
> $ git commit -m "first commit"
>
> $ git checkout -b branch
> $ printf "*.bin binary\n" >> .gitattributes
> $ git add .
> $ git commit -m "tracking *.bin files"
>
> $ git checkout master
> $ printf "binary\ndata\n" > file.dat # <-- Unix line ending!
> $ printf "*.dat binary\n" >> .gitattributes # <-- Tell Git to keep Unix line ending!
> $ git add .
> $ git commit -m "tracking *.dat files"
> $ git cat-file -p :file.dat | od -c
> 0000000   b   i   n   a   r   y  \n   d   a   t   a  \n
>                                 ^^^^                ^^^^  <-- Correct!
> $ git checkout branch
> $ git merge master # <-- Causes merge conflict!
> $ printf "*.bin binary\n*.dat binary\n" > .gitattributes # <-- Fix merge conflict!
> $ git add .
> $ git commit -m "merged"
> $ git cat-file -p :file.dat | od -c
> 0000000   b   i   n   a   r   y  \r  \n   d   a   t   a  \r  \n
>                                 ^^^^^^^^                ^^^^^^^^  <-- Wrong!
>
> Possible solutions:
>
> 1. We could print an appropriate warning if we detect a merge conflict
>    in .gitattributes

This is good regardless, to encourage people to resolve conflicts in
.gitattributes first. A good place for this warning may be "git
status"?

> 2. We could disable all line ending conversions in case of a merge conflict
>    (I am not exactly sure about all the implications, though)
>
> 3. We could salvage what we could of the .gitattributes file,
>    perhaps by using the version from HEAD (or more likely, the ours stage of
>    the index) -- suggested by Peff on the related GitHub issue mentioned below

We already have code to fall back to index version in some cases,
adding "fall back on merge conflicts" (and probably updating the index
lookup code too because it looks for stage 0 now) sounds reasonable
(especially with the warning in #1).

BTW whoever fixes this probably should do the same for .gitignore files.
-- 
Duy

Re: Merge conflicts in .gitattributes can cause trouble

[Johannes Schindelin]

Hi Lars,

On Tue, 4 Oct 2016, Lars Schneider wrote:

> If there is a conflict in the .gitattributes during a merge then it looks 
> like as if the attributes are not applied

I tried to replicate this behavior, to the point where I wrote a patch
that demonstrates the breakage so I could single-step in a debugger to
find out where things go wrong, and fix them.

Alas, I found out that the .gitattributes are read *before* any merge
conflict arises in the case you demonstrated. Which kind of makes sense,
because the gitattributes decide over which merge driver to use, among
other things.

So in your example:

> Consider this script on Windows:
> 
> $ git init .
> $ touch first.commit
> $ git add .
> $ git commit -m "first commit"
> 
> $ git checkout -b branch
> $ printf "*.bin binary\n" >> .gitattributes
> $ git add .
> $ git commit -m "tracking *.bin files"
> 
> $ git checkout master
> $ printf "binary\ndata\n" > file.dat # <-- Unix line ending!
> $ printf "*.dat binary\n" >> .gitattributes # <-- Tell Git to keep Unix line ending!
> $ git add .
> $ git commit -m "tracking *.dat files"
> $ git cat-file -p :file.dat | od -c
> 0000000   b   i   n   a   r   y  \n   d   a   t   a  \n 
>                                 ^^^^                ^^^^  <-- Correct!
> $ git checkout branch

At this point, the .gitattributes list only .bin files as binary. That is
the revision of the .gitattributes used by this command:

> $ git merge master # <-- Causes merge conflict!

And as a consequence, the .gitattributes do not tell Git that it should
handle .dat files as binary. Which means that...

> $ printf "*.bin binary\n*.dat binary\n" > .gitattributes # <-- Fix merge conflict!
> $ git add .
> $ git commit -m "merged"
> $ git cat-file -p :file.dat | od -c
> 0000000   b   i   n   a   r   y  \r  \n   d   a   t   a  \r  \n
>                                 ^^^^^^^^                ^^^^^^^^  <-- Wrong!

... this is actually expected! Why? Because the .gitattributes that were
in effect when the user asked to perform a merge said so.

If you adjust .gitattributes *before* merging `master`, it works as you
would expect: the line endings are not changed.

The reason to do it this way: we want to respect the .gitattributes as per
the current worktree. We go even so far that we respect uncommitted
changes to said file...

> Possible solutions:
> 
> 1. We could print an appropriate warning if we detect a merge conflict 
>    in .gitattributes
> 
> 2. We could disable all line ending conversions in case of a merge conflict
>    (I am not exactly sure about all the implications, though)
> 
> 3. We could salvage what we could of the .gitattributes file, 
>    perhaps by using the version from HEAD (or more likely, the ours stage of
>    the index) -- suggested by Peff on the related GitHub issue mentioned below

I would vote for:

4. We keep letting Git read in the *current* version of .gitattributes
   *before* the merge, and apply those attributes while performing the
   merge.

Ciao,
Dscho

Re: Merge conflicts in .gitattributes can cause trouble

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> I would vote for:
>
> 4. We keep letting Git read in the *current* version of .gitattributes
>    *before* the merge, and apply those attributes while performing the
>    merge.

Even though this needs a major surgery to the way the attr subsystem
reads from these files, I think it is conceptually the cleanest.

Re: Merge conflicts in .gitattributes can cause trouble

[Johannes Schindelin]

Hi Junio,

On Mon, 17 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > I would vote for:
> >
> > 4. We keep letting Git read in the *current* version of .gitattributes
> >    *before* the merge, and apply those attributes while performing the
> >    merge.
> 
> Even though this needs a major surgery to the way the attr subsystem
> reads from these files, I think it is conceptually the cleanest.

To the contrary. As far as I can see, when calling `git merge`, Git
currently *does* read .gitattributes from the file, and if that fails,
falls back to reading that file from the index.

In other words, option 4. is the current behavior no change required.

Ciao,
Dscho

Re: Merge conflicts in .gitattributes can cause trouble

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> To the contrary. As far as I can see, when calling `git merge`, Git
> currently *does* read .gitattributes from the file, and if that fails,
> falls back to reading that file from the index.

Hmph.

Assuming that the merge always goes in the index order, I think you
are right.  When we need to merge path/to/dir/.gitattributes, we
would need to know all the .gitattributes files that may affect that
path, so .gitattributes, path/.gitattributes, path/to/.gitattributes
and the file being merged are all read into core before anything
happens and these are kept in attr_stack while merging anything
underneath path/to/dir/ hierarchy without being re-read from the
filesystem.  The original contents of path/to/dir/.gitattributes
cached in the attr_stack will be discarded when we start to merge
things outside path/to/dir (e.g. merging path/to/another/file), but
at that point the contents of path/to/dir/.gitattributes no longer
matters to the path being merged, so unless the merge somehow jumps
around it is OK.

It may be a fragile assumption in the longer term that the merge
always goes in the index order, but I think the assumption holds in
the current codebase, and the update planned immediately in the
future.