bugreport: git apply -3 confusing "lacks the necessary blob"

bugreport: git apply -3 confusing "lacks the necessary blob"

[Jan Kratochvil]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)
git apply -3 b.diff

What did you expect to happen? (Expected behavior)
<<<<<<< ours
c
||||||| base
a
=======
b
>>>>>>> theirs

What happened instead? (Actual behavior)
error: patch failed: x:1
error: repository lacks the necessary blob to fall back on 3-way merge.
error: x: patch does not apply

What's different between what you expected and what actually happened?
The conflicting patch has not been applied to resolve it manually.

Anything else you want to add:
The problem is 'git apply -3' will print confusing "lacks the necessary blob"
message despite the real error message should be:
"b.diff is missing line starting with 'index', cannot apply by -3/--3way"

reproducer:
(set -ex;: rm -rf gitgit;mkdir gitgit;cd gitgit;git init;echo a >x;git add x;git commit -am.;git checkout -b b;echo b >x;git commit -am.;git checkout master;echo c >x;git commit -am.;git diff master^..b|grep -v ^index >b.diff;git apply -3 b.diff || cat b.diff)

Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.31.1
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 5.12.11-300.fc34.x86_64 #1 SMP Wed Jun 16 15:47:58 UTC 2021 x86_64
compiler info: gnuc: 11.0
libc info: glibc: 2.33
$SHELL (typically, interactive shell): /bin/bash


[Enabled Hooks]
not run from a git repository - no hooks to show

Re: bugreport: git apply -3 confusing "lacks the necessary blob"

[Bagas Sanjaya]

On 19/08/21 01.17, Jan Kratochvil wrote:
> reproducer:
> (set -ex;: rm -rf gitgit;mkdir gitgit;cd gitgit;git init;echo a >x;git add x;git commit -am.;git checkout -b b;echo b >x;git commit -am.;git checkout master;echo c >x;git commit -am.;git diff master^..b|grep -v ^index >b.diff;git apply -3 b.diff || cat b.diff)

I can reproduce your issue on latest Git (2.33.0).

It seems like you remove `index` line, which **may** contain blob hash 
information required for three-way merge with git apply -3.

But if you don't remove it when generating patch that way, you will get 
expected conflict when git applying.

-- 
An old man doll... just what I always wanted! - Clara

Re: bugreport: git apply -3 confusing "lacks the necessary blob"

[Jan Kratochvil]

On Fri, 20 Aug 2021 11:19:39 +0200, Bagas Sanjaya wrote:
> On 19/08/21 01.17, Jan Kratochvil wrote:
> > reproducer:
> > (set -ex;: rm -rf gitgit;mkdir gitgit;cd gitgit;git init;echo a >x;git add x;git commit -am.;git checkout -b b;echo b >x;git commit -am.;git checkout master;echo c >x;git commit -am.;git diff master^..b|grep -v ^index >b.diff;git apply -3 b.diff || cat b.diff)
> 
> I can reproduce your issue on latest Git (2.33.0).
> 
> It seems like you remove `index` line, which **may** contain blob hash
> information required for three-way merge with git apply -3.
> 
> But if you don't remove it when generating patch that way, you will get
> expected conflict when git applying.

Yes.

The problem is when the 'index' line is already removed (as it was always
being removed in my existing setup) it is difficult to find out that is the
reason of this confusing message. So the error message should be different
when the 'index' line is missing at all.


Jan

Re: bugreport: git apply -3 confusing "lacks the necessary blob"

[Junio C Hamano]

Bagas Sanjaya <bagasdotme@gmail.com> writes:

> On 19/08/21 01.17, Jan Kratochvil wrote:
>> reproducer:
>> (set -ex;: rm -rf gitgit;mkdir gitgit;cd gitgit;git init;echo a >x;git add x;git commit -am.;git checkout -b b;echo b >x;git commit -am.;git checkout master;echo c >x;git commit -am.;git diff master^..b|grep -v ^index >b.diff;git apply -3 b.diff || cat b.diff)
>
> I can reproduce your issue on latest Git (2.33.0).
>
> It seems like you remove `index` line, which **may** contain blob hash
> information required for three-way merge with git apply -3.
>
> But if you don't remove it when generating patch that way, you will
> get expected conflict when git applying.

You make it sound like an "it hurts when I do this, doctor. ---do
not do it, then" exchange ;-).

But I think Jan is talking about the case where users get a patch
that lacks the "index" information out of other people's "diff"
implementation and try to "apply -3" without realizing that it is
not Git's "diff" output.

Perhaps something like the attached patch would be a good start.

There is another place that relies on the index line (application of
binary patch), but the code rejects any patch that does not have the
full-length object name with "... without full index line".  This
message was originally meant to reject patches with abbreviated
object names on the index line, but it would equally apply to one
without the index line (perhaps by accident), so there is no need to
touch that one.


 apply.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git i/apply.c w/apply.c
index 44bc31d6eb..9972ada57e 100644
--- i/apply.c
+++ w/apply.c
@@ -3566,6 +3566,8 @@ static int try_threeway(struct apply_state *state,
 	/* Preimage the patch was prepared for */
 	if (patch->is_new)
 		write_object_file("", 0, blob_type, &pre_oid);
+	else if (!*patch->old_oid_prefix && !*patch->new_oid_prefix)
+		return error(_("cannot 'apply -3' a patch without the index line"));
 	else if (get_oid(patch->old_oid_prefix, &pre_oid) ||
 		 read_blob_object(&buf, &pre_oid, patch->old_mode))
 		return error(_("repository lacks the necessary blob to perform 3-way merge."));

Re: bugreport: git apply -3 confusing "lacks the necessary blob"

[Jan Kratochvil]

On Fri, 20 Aug 2021 19:40:12 +0200, Junio C Hamano wrote:
> But I think Jan is talking about the case where users get a patch
> that lacks the "index" information out of other people's "diff"
> implementation and try to "apply -3" without realizing that it is
> not Git's "diff" output.

Yes.


> Perhaps something like the attached patch would be a good start.

It does fix the problem for me, thanks!


Jan

Re: bugreport: git apply -3 confusing "lacks the necessary blob"

[Jan Kratochvil]

On Fri, 20 Aug 2021 19:40:12 +0200, Junio C Hamano wrote:
> But I think Jan is talking about the case where users get a patch
> that lacks the "index" information out of other people's "diff"
> implementation and try to "apply -3" without realizing that it is
> not Git's "diff" output.

For example from Phabricator:
	https://reviews.llvm.org/file/data/3ceoc32b3yv43vk3nw4q/PHID-FILE-lfeeh2qu4vrngdcwwudo/D107456.diff


> Perhaps something like the attached patch would be a good start.

Do you plan to check it in?


Thanks,
Jan Kratochvil


> diff --git i/apply.c w/apply.c
> index 44bc31d6eb..9972ada57e 100644
> --- i/apply.c
> +++ w/apply.c
> @@ -3566,6 +3566,8 @@ static int try_threeway(struct apply_state *state,
>  	/* Preimage the patch was prepared for */
>  	if (patch->is_new)
>  		write_object_file("", 0, blob_type, &pre_oid);
> +	else if (!*patch->old_oid_prefix && !*patch->new_oid_prefix)
> +		return error(_("cannot 'apply -3' a patch without the index line"));
>  	else if (get_oid(patch->old_oid_prefix, &pre_oid) ||
>  		 read_blob_object(&buf, &pre_oid, patch->old_mode))
>  		return error(_("repository lacks the necessary blob to perform 3-way merge."));

Re: bugreport: git apply -3 confusing "lacks the necessary blob"

[Junio C Hamano]

Jan Kratochvil <jan@jankratochvil.net> writes:

> On Fri, 20 Aug 2021 19:40:12 +0200, Junio C Hamano wrote:
>> But I think Jan is talking about the case where users get a patch
>> that lacks the "index" information out of other people's "diff"
>> implementation and try to "apply -3" without realizing that it is
>> not Git's "diff" output.
>
> For example from Phabricator:
> 	https://reviews.llvm.org/file/data/3ceoc32b3yv43vk3nw4q/PHID-FILE-lfeeh2qu4vrngdcwwudo/D107456.diff
>
>
>> Perhaps something like the attached patch would be a good start.
>
> Do you plan to check it in?

Maybe eventually but not yet.  When I say "a good start", it is
because it is known to be insufficient.  I haven't had a chance to
revisit the issue.