git diff --text does not work during rebase for binary files

git diff --text does not work during rebase for binary files

[Дилян Палаузов]

Hello,

I create a small text file ending with \0 and add it to git:

In .gitattributes I specify:

*.txt diff merge=text text

I create several commits by changing the small text file.

The result is at https://github.com/dilyanpalauzov/git-diff-biinary .

I do now

  git rebase -i HEAD~2
and before the “second commit” type E to edit that commit.

I change the small file.

git add a.txt && git rebase --continue

git says:

[detached HEAD bc00e34] second commit
 Date: Sun Mar 21 19:16:15 2021 +0200
 1 file changed, 2 insertions(+), 1 deletion(-)
warning: Cannot merge binary files: a.txt (HEAD vs. 04c77de (third
commit))
Auto-merging a.txt
CONFLICT (content): Merge conflict in a.txt
error: could not apply 04c77de... third commit
Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase
--abort".
Could not apply 04c77de... third commit


'git diff' and 'git diff --text' show:

diff --cc a.txt
index 7a61015,dc817ec..0000000
--- a/a.txt
+++ b/a.txt

The file does not contain <<HEAD===>>> markers, despite having
merge=text attribute.

HOW CAN I force git diff to show the differences and git merge to
include the <<<===>>> markers?

I have to say, that at different occassions git diff --text does work
ass expected (e.g. when I call git diff --cached --text).

I call now 
git rebase --abort
git log -p

git shows the differences in the small text file ending with \0.  So
apparently log reads the option, but diff does not.

Using git 2.30.2.

Greetings
  Дилян

Re: git diff --text does not work during rebase for binary files

[Bagas Sanjaya]

On 22/03/21 00.30, Дилян Палаузов wrote:
> Hello,
> 
> I create a small text file ending with \0 and add it to git:

Did you mean `echo -e "\0" >> a.txt`?

> In .gitattributes I specify:
> 
> *.txt diff merge=text text
> 
> I create several commits by changing the small text file.
> 
> The result is at https://github.com/dilyanpalauzov/git-diff-biinary .
> 
> I do now
> 
>    git rebase -i HEAD~2
> and before the “second commit” type E to edit that commit.
> 
> I change the small file.
> 
> git add a.txt && git rebase --continue
> 
> git says:
> 
> [detached HEAD bc00e34] second commit
>   Date: Sun Mar 21 19:16:15 2021 +0200
>   1 file changed, 2 insertions(+), 1 deletion(-)
> warning: Cannot merge binary files: a.txt (HEAD vs. 04c77de (third
> commit))
> Auto-merging a.txt
> CONFLICT (content): Merge conflict in a.txt
> error: could not apply 04c77de... third commit
> Resolve all conflicts manually, mark them as resolved with
> "git add/rm <conflicted_files>", then run "git rebase --continue".
> You can instead skip this commit: run "git rebase --skip".
> To abort and get back to the state before "git rebase", run "git rebase
> --abort".
> Could not apply 04c77de... third commit
> 
> 
> 'git diff' and 'git diff --text' show:
> 
> diff --cc a.txt
> index 7a61015,dc817ec..0000000
> --- a/a.txt
> +++ b/a.txt
> 
> The file does not contain <<HEAD===>>> markers, despite having
> merge=text attribute.
> 
> HOW CAN I force git diff to show the differences and git merge to
> include the <<<===>>> markers?
> 
> I have to say, that at different occassions git diff --text does work
> ass expected (e.g. when I call git diff --cached --text).
> 
> I call now
> git rebase --abort
> git log -p
> 
> git shows the differences in the small text file ending with \0.  So
> apparently log reads the option, but diff does not.
> 
> Using git 2.30.2.
> 
> Greetings
>    Дилян
> 

Does `file a.txt` shows it as being binary file? I do the similar and it just says "data".

-- 
An old man doll... just what I always wanted! - Clara

Re: git diff --text does not work during rebase for binary files

[Дилян Палаузов]

Hello,

I mean:

git init
echo -e "AB RA\nCADABRA\0" > a.txt && git add a.txt && git commit -
m"Initial"
echo -e "AB2RA\nCADABRA\0" > a.txt && git add a.txt && git commit -
m"Second"
echo -e "AB3RA\nCADABRA\0" > a.txt && git add a.txt && git commit -
m"Third"

git log -pa   show the diffs and handles the file as text.

git rebase -i HEAD~2

I EDIT the “Second” commit (the one before “Third”, not the second in
the editor).


master|REBASE 1/2$ echo -e "AB4RA\nCADABRA\0" > a.txt && git add a.txt
&& git rebase --continue	

[detached HEAD 78408d2] Second
 Date: Mon Mar 22 10:35:57 2021 +0200
 1 file changed, 0 insertions(+), 0 deletions(-)
warning: Cannot merge binary files: a.txt (HEAD vs. 149b67e (Third))
Auto-merging a.txt
CONFLICT (content): Merge conflict in a.txt
error: could not apply 149b67e... Third
Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase
--abort".
Could not apply 149b67e... Third
master|REBASE 2/2$

And now `git diff` and `git diff --text` both print:

diff --cc a.txt
index ee95164,d7db2f0..0000000
Binary files differ


But git diff --text should show how the files differ.

Greetings
  Дилян

On Mon, 2021-03-22 at 12:53 +0700, Bagas Sanjaya wrote:
> 
> 
> On 22/03/21 00.30, Дилян Палаузов wrote:
> > Hello,
> > 
> > I create a small text file ending with \0 and add it to git:
> 
> Did you mean `echo -e "\0" >> a.txt`?
> 
> > In .gitattributes I specify:
> > 
> > *.txt diff merge=text text
> > 
> > I create several commits by changing the small text file.
> > 
> > The result is at https://github.com/dilyanpalauzov/git-diff-biinary .
> > 
> > I do now
> > 
> >    git rebase -i HEAD~2
> > and before the “second commit” type E to edit that commit.
> > 
> > I change the small file.
> > 
> > git add a.txt && git rebase --continue
> > 
> > git says:
> > 
> > [detached HEAD bc00e34] second commit
> >   Date: Sun Mar 21 19:16:15 2021 +0200
> >   1 file changed, 2 insertions(+), 1 deletion(-)
> > warning: Cannot merge binary files: a.txt (HEAD vs. 04c77de (third
> > commit))
> > Auto-merging a.txt
> > CONFLICT (content): Merge conflict in a.txt
> > error: could not apply 04c77de... third commit
> > Resolve all conflicts manually, mark them as resolved with
> > "git add/rm <conflicted_files>", then run "git rebase --continue".
> > You can instead skip this commit: run "git rebase --skip".
> > To abort and get back to the state before "git rebase", run "git
> > rebase
> > --abort".
> > Could not apply 04c77de... third commit
> > 
> > 
> > 'git diff' and 'git diff --text' show:
> > 
> > diff --cc a.txt
> > index 7a61015,dc817ec..0000000
> > --- a/a.txt
> > +++ b/a.txt
> > 
> > The file does not contain <<HEAD===>>> markers, despite having
> > merge=text attribute.
> > 
> > HOW CAN I force git diff to show the differences and git merge to
> > include the <<<===>>> markers?
> > 
> > I have to say, that at different occassions git diff --text does work
> > ass expected (e.g. when I call git diff --cached --text).
> > 
> > I call now
> > git rebase --abort
> > git log -p
> > 
> > git shows the differences in the small text file ending with \0.  So
> > apparently log reads the option, but diff does not.
> > 
> > Using git 2.30.2.
> > 
> > Greetings
> >    Дилян
> > 
> 
> Does `file a.txt` shows it as being binary file? I do the similar and
> it just says "data".
> 

Re: git diff --text does not work during rebase for binary files

[Jeff King]

On Mon, Mar 22, 2021 at 10:45:39AM +0200, Дилян Палаузов wrote:

> And now `git diff` and `git diff --text` both print:
> 
> diff --cc a.txt
> index ee95164,d7db2f0..0000000
> Binary files differ

I think the "combined diff" code isn't ready to handle the "--text"
flag.

An easier reproduction than rebase is to just do a merge with a
conflict:

  git init
  printf 'base\0' >file && git add file && git commit -m base
  printf 'one\0' >file && git commit -am one
  git checkout -b side HEAD^
  printf 'two\0' >file && git commit -am two
  git merge master

Now "git diff --text" will try to show a combined diff, but it doesn't
work. Likewise if you complete the merge:

  printf 'new\0' >file && git commit -a --no-edit

then "git show --text" will not show a binary combined diff, even though
"git show --text HEAD^" will happily show a normal two-endpoint diff.

Curiously old versions of Git would show broken binary combined diffs,
until 4d5f347199 (combine-diff: handle binary files as binary,
2011-05-23). So the binary detection there looks mostly sane, but it's
just not respecting the "--text" flag.

Something like this helps a bit:

diff --git a/combine-diff.c b/combine-diff.c
index 06635f91bc..940c5c43ff 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1128,6 +1128,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
 
 	if (textconv)
 		is_binary = 0;
+	else if (opt->flags.text)
+		is_binary = 0;
 	else if (userdiff->binary != -1)
 		is_binary = userdiff->binary;
 	else {

but I don't think that's the full story. It looks like the combined-diff
code is too eager to treat the content as NUL-terminated strings. A
regular diff shows the NULs:

  $ git show --format= --text HEAD^  | cat -A
  diff --git a/file b/file$
  index d249428..a9382c3 100644$
  --- a/file$
  +++ b/file$
  @@ -1 +1 @@$
  -base^@$
  \ No newline at end of file$
  +two^@$
  \ No newline at end of file$

but the combined diff doesn't:

  $ git show --format= --text HEAD | cat -A
  diff --cc file$
  index a9382c3,f33a432..c984a04$
  --- a/file$
  +++ b/file$
  @@@ -1,1 -1,1 +1,1 @@@$
  - two$
   -one$
  ++new$

So I suspect the code is simply not read to handle the NULs, and the
current behavior of ignoring --text is better than producing truncated
or bogus output until those deeper problems are fixed. We could issue a
warning that --text was ignored, I suppose, but that doesn't really
solve your problem. :)

-Peff