git format-patch lost the last part when branch merge

git format-patch lost the last part when branch merge

[Wang Yugui]

Hi,

git format-patch lost the last part when branch merge

Here is an example.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Merge tag 'for-5.12/block-2021-02-17' of git://git.kernel.dk/linux-block

1, from the web interface,
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/fs/btrfs?id=582cd91f69de8e44857cb610ebca661dac8656b7

the last part 'diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c' can be confirmed.

2, but from ' commit | 582cd91f69de8e44857cb610ebca661dac8656b7 (patch)' of this web page,
the last part 'diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c' is not in the patch file.

3, git format-patch 4f016a316f22.. fs/btrfs/ will not output 'diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c' too.

Best Regards


[System Info]
git version:
git version 2.30.1 and 2.27.0
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 5.10.17-3.el7.x86_64 #1 SMP Mon Feb 22 10:43:13 CST 2021 x86_64
compiler info: gnuc: 4.8
libc info: glibc: 2.17
$SHELL (typically, interactive shell): /bin/bash


[Enabled Hooks]
None


Best Regards
王玉贵
2021/02/22

--------------------------------------
北京京垓科技有限公司
王玉贵	wangyugui@e16-tech.com
电话：+86-136-71123776

Re: git format-patch lost the last part when branch merge

[Jeff King]

On Mon, Feb 22, 2021 at 09:16:21PM +0800, Wang Yugui wrote:

> git format-patch lost the last part when branch merge
> 
> Here is an example.
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> Merge tag 'for-5.12/block-2021-02-17' of git://git.kernel.dk/linux-block
> 
> 1, from the web interface,
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/fs/btrfs?id=582cd91f69de8e44857cb610ebca661dac8656b7
> 
> the last part 'diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c' can be confirmed.
> 
> 2, but from ' commit | 582cd91f69de8e44857cb610ebca661dac8656b7 (patch)' of this web page,
> the last part 'diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c' is not in the patch file.
> 
> 3, git format-patch 4f016a316f22.. fs/btrfs/ will not output 'diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c' too.

This is expected. Format-patch omits merge commits entirely, as they
can't be formatted as a simple diff that can be applied.

There are lots of ways to look at the diff of a merge. By default, `git
show` will show a combined diff, which omits hunks where one side was
taken verbatim, but otherwise shows what each side did.

The diff shown in the link above is a diff against the first-parent
(which you can also get locally with `git show --first-parent 582cd91`).
One _could_ apply that diff onto the first parent to achieve the same
tree as the merge plus all of the commits that got merged in. But it
wouldn't make any sense to apply that (aside from conflict resolution,
it would be redundant with all of the commits that format-patch just
output!).

You could imagine ways for format-patch to represent the conflict
resolution done in a merge, but it's not quite trivial, and nobody has
done it yet.

Depending on why you're using format-patch, the solution may be one of:

  - use "git log --cc" instead, which shows the merges using combined
    diffs

  - use "git bundle" instead, which reproduces the whole history graph,
    including merges (though obviously you cannot then re-apply the
    commits onto a different history)

-Peff

[PATCH] docs/format-patch: mention handling of merges

[Jeff King]

On Mon, Feb 22, 2021 at 05:57:50PM -0500, Jeff King wrote:

> This is expected. Format-patch omits merge commits entirely, as they
> can't be formatted as a simple diff that can be applied.

We don't seem to advertise this very well in the documentation.

I think we should at least do something like the patch below. I do have
a dream that one day we'll be able to represent conflict resolutions
over email, but we're not there yet.

Likewise, it might be reasonable format-patch to issue a warning() when
it is ignoring a merge. Or maybe some people would find that annoying
(e.g., because they don't care about back-merges from upstream that
happened during the development of a topic).

But I think this documentation change is easy and hopefully
uncontroversial.

-- >8 --
Subject: [PATCH] docs/format-patch: mention handling of merges

Format-patch doesn't have a way to format merges in a way that can be
applied by git-am (or any other tool), and so it just omits them.
However, this may be a surprising implication for users who are not well
versed in how the tool works. Let's add a note to the documentation
making this more clear.

Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/git-format-patch.txt | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index 3e49bf2210..33649098ce 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -36,7 +36,7 @@ SYNOPSIS
 DESCRIPTION
 -----------
 
-Prepare each commit with its patch in
+Prepare each non-merge commit with its patch in
 one file per commit, formatted to resemble UNIX mailbox format.
 The output of this command is convenient for e-mail submission or
 for use with 'git am'.
@@ -718,6 +718,13 @@ use it only when you know the recipient uses Git to apply your patch.
 $ git format-patch -3
 ------------
 
+CAVEATS
+-------
+
+Note that `format-patch` cannot represent commits with more than one
+parent (i.e., merges) and will silently omit them entirely from its
+output, even if they are part of the requested range.
+
 SEE ALSO
 --------
 linkgit:git-am[1], linkgit:git-send-email[1]
-- 
2.30.1.1033.gd525307ce1

Re: git format-patch lost the last part when branch merge

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> This is expected. Format-patch omits merge commits entirely, as they
> can't be formatted as a simple diff that can be applied.
>
> There are lots of ways to look at the diff of a merge. By default, `git
> show` will show a combined diff, which omits hunks where one side was
> taken verbatim, but otherwise shows what each side did.
>
> The diff shown in the link above is a diff against the first-parent
> (which you can also get locally with `git show --first-parent 582cd91`).
> One _could_ apply that diff onto the first parent to achieve the same
> tree as the merge plus all of the commits that got merged in. But it
> wouldn't make any sense to apply that (aside from conflict resolution,
> it would be redundant with all of the commits that format-patch just
> output!).

Yes, a first-parent diff is something you could call "a simple diff
that can be applied to represent a merge", and it is consistent with
the expectations of those who are used to do a squash (pseudo-)merge.

I agree with you that it does not make sense to apply such a patch
as a patch, of course.  In addition to be redundant, it would be an
equivalent of doing a squash (psuedo-)merge, and loses the "up to
this point the side branch has been merged, so future merges won't
have to look beyond this point in the past" (sort of going back to
the prehistoric subversion days that did not keep track of which
changes have been merged).

Re: [PATCH] docs/format-patch: mention handling of merges

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Subject: [PATCH] docs/format-patch: mention handling of merges
>
> Format-patch doesn't have a way to format merges in a way that can be
> applied by git-am (or any other tool), and so it just omits them.
> However, this may be a surprising implication for users who are not well
> versed in how the tool works. Let's add a note to the documentation
> making this more clear.
> ...
> +CAVEATS
> +-------
> +
> +Note that `format-patch` cannot represent commits with more than one
> +parent (i.e., merges) and will silently omit them entirely from its
> +output, even if they are part of the requested range.


I think "cannot represent" is a little bit misleading, unless we
expect the readers already know what we are trying to say (in which
case there is no point in documenting this).  Perhaps something like
this might clarify a bit, though.

    Note that `format-patch` omits merge commits from the output,
    because it is impossible to turn a merge commit into a simple
    "patch" in such a way that allows receiving end to reproduce the
    same merge commit.

Re: [PATCH] docs/format-patch: mention handling of merges

[Jeff King]

On Mon, Feb 22, 2021 at 03:31:53PM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Subject: [PATCH] docs/format-patch: mention handling of merges
> >
> > Format-patch doesn't have a way to format merges in a way that can be
> > applied by git-am (or any other tool), and so it just omits them.
> > However, this may be a surprising implication for users who are not well
> > versed in how the tool works. Let's add a note to the documentation
> > making this more clear.
> > ...
> > +CAVEATS
> > +-------
> > +
> > +Note that `format-patch` cannot represent commits with more than one
> > +parent (i.e., merges) and will silently omit them entirely from its
> > +output, even if they are part of the requested range.
> 
> 
> I think "cannot represent" is a little bit misleading, unless we
> expect the readers already know what we are trying to say (in which
> case there is no point in documenting this).  Perhaps something like
> this might clarify a bit, though.
> 
>     Note that `format-patch` omits merge commits from the output,
>     because it is impossible to turn a merge commit into a simple
>     "patch" in such a way that allows receiving end to reproduce the
>     same merge commit.

That seems worse to me, because "it is impossible" implies that this
can never be changed. But I don't think that's true. We might one day
output something useful for merges.

I think one could argue that any merge information (including conflict
resolution) works against the root notion of format-patch, which is a
set of changes that can be applied on a range of basesa. But even that I
would be hesitant to commit to (since --base exists now). And certainly
it's more subtlety than I'd want to get in to for this note. :)

I almost softened it to "cannot yet represent". Does that read better to
your (or worse)? Likewise, I considered adding a note at the end along
the lines of "this may change in the future", though I suspect we'd only
do so in combination with a command-line option.

-Peff

Re: [PATCH] docs/format-patch: mention handling of merges

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> That seems worse to me, because "it is impossible" implies that this
> can never be changed. But I don't think that's true. We might one day
> output something useful for merges.

Fair enough.  You are more optimistic than I am ;-)

> I think one could argue that any merge information (including conflict
> resolution) works against the root notion of format-patch, which is a
> set of changes that can be applied on a range of basesa.

That's true and it was the primary motive for omiting merges.

> But even that I
> would be hesitant to commit to (since --base exists now).

I am not quite sure what --base has to throw into the equation.  The
information --base gives is often useful when I want to learn where
the patches were taken from, but that does not restrict where the
patches are actually applied to in any meaningful way (iow, "on a
range of bases" part is not affected).

Re: [PATCH] docs/format-patch: mention handling of merges

[Jeff King]

On Mon, Feb 22, 2021 at 05:24:16PM -0800, Junio C Hamano wrote:

> > I think one could argue that any merge information (including conflict
> > resolution) works against the root notion of format-patch, which is a
> > set of changes that can be applied on a range of basesa.
> 
> That's true and it was the primary motive for omiting merges.
> 
> > But even that I
> > would be hesitant to commit to (since --base exists now).
> 
> I am not quite sure what --base has to throw into the equation.  The
> information --base gives is often useful when I want to learn where
> the patches were taken from, but that does not restrict where the
> patches are actually applied to in any meaningful way (iow, "on a
> range of bases" part is not affected).

What I meant is that without "--base", telling somebody "here is the
merge you should replay on top of these other patches" is virtually
meaningless. You cannot know what the merge base would be! So you might
be merging in other random crap, and you might or might not see the same
conflicts.

But in a world with --base, I can imagine some people recreating whole
sequences of the history graph by using "--base" along with some (to be
invented) format for representing a merge via email. That mode would
certainly not be the default, but at least at that point it is
conceivably useful. Sort of like a bundle, but more human-readable (it
would also need committer info to recreate the commit ids perfectly, of
course).

All of which meant only to argue that "it is not possible or not useful
to represent a merge in an email" is something that could change in the
future. :)

-Peff

Re: git format-patch lost the last part when branch merge

[Elijah Newren]

On Mon, Feb 22, 2021 at 3:03 PM Jeff King <peff@peff.net> wrote:
[...snip...]
>
> There are lots of ways to look at the diff of a merge. By default, `git
> show` will show a combined diff, which omits hunks where one side was
> taken verbatim, but otherwise shows what each side did.
>
> The diff shown in the link above is a diff against the first-parent
> (which you can also get locally with `git show --first-parent 582cd91`).
> One _could_ apply that diff onto the first parent to achieve the same
> tree as the merge plus all of the commits that got merged in. But it
> wouldn't make any sense to apply that (aside from conflict resolution,
> it would be redundant with all of the commits that format-patch just
> output!).
>
> You could imagine ways for format-patch to represent the conflict
> resolution done in a merge, but it's not quite trivial, and nobody has
> done it yet.

Are you referring to something like this (in the git.git repository)?

$ git show --oneline --remerge-diff 42342b3ee6
42342b3ee6 Merge branch 'ab/mailmap'
diff --git a/t/t4203-mailmap.sh b/t/t4203-mailmap.sh
index 1bce961e07..6fb18a34b0 100755
--- a/t/t4203-mailmap.sh
+++ b/t/t4203-mailmap.sh
@@ -257,16 +257,8 @@ test_expect_success 'No mailmap files, but configured' '

 test_expect_success 'setup mailmap blob tests' '
        git checkout -b map &&
-<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60ecad090d (Merge branch 'ps/fetch-atomic')
        test_when_finished "git checkout main" &&
-       cat >just-bugs <<- EOF &&
-|||||||||||||||||||||||||||||||| 72c4083ddf
-       test_when_finished "git checkout master" &&
-       cat >just-bugs <<- EOF &&
-================================
-       test_when_finished "git checkout master" &&
        cat >just-bugs <<-\EOF &&
->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4e168333a8 (shortlog: remove
unused(?) "repo-abbrev" feature)
        Blob Guy <bugs@company.xx>
        EOF
        cat >both <<-EOF &&


I agree that representing the conflict done in a merge is more
difficult than it sounds, but if you mean what I think you mean then I
disagree with the "nobody has done it yet" half of your statement.
:-)

That said, I haven't attempted to tie this into format-patch in any
way, and have absolutely no plans to.  (And --remerge-diff hasn't been
submitted upstream, because it depends on ort, and that still needs
reviews...)

Re: git format-patch lost the last part when branch merge

[Jeff King]

On Tue, Feb 23, 2021 at 08:24:24PM -0800, Elijah Newren wrote:

> > You could imagine ways for format-patch to represent the conflict
> > resolution done in a merge, but it's not quite trivial, and nobody has
> > done it yet.
> 
> Are you referring to something like this (in the git.git repository)?
> 
> $ git show --oneline --remerge-diff 42342b3ee6
> 42342b3ee6 Merge branch 'ab/mailmap'
> diff --git a/t/t4203-mailmap.sh b/t/t4203-mailmap.sh
> index 1bce961e07..6fb18a34b0 100755
> --- a/t/t4203-mailmap.sh
> +++ b/t/t4203-mailmap.sh
> @@ -257,16 +257,8 @@ test_expect_success 'No mailmap files, but configured' '
> 
>  test_expect_success 'setup mailmap blob tests' '
>         git checkout -b map &&
> -<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60ecad090d (Merge branch 'ps/fetch-atomic')
>         test_when_finished "git checkout main" &&
> -       cat >just-bugs <<- EOF &&
> -|||||||||||||||||||||||||||||||| 72c4083ddf
> -       test_when_finished "git checkout master" &&
> -       cat >just-bugs <<- EOF &&
> -================================
> -       test_when_finished "git checkout master" &&
>         cat >just-bugs <<-\EOF &&
> ->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 4e168333a8 (shortlog: remove
> unused(?) "repo-abbrev" feature)
>         Blob Guy <bugs@company.xx>
>         EOF
>         cat >both <<-EOF &&
> 
> 
> I agree that representing the conflict done in a merge is more
> difficult than it sounds, but if you mean what I think you mean then I
> disagree with the "nobody has done it yet" half of your statement.
> :-)
> 
> That said, I haven't attempted to tie this into format-patch in any
> way, and have absolutely no plans to.  (And --remerge-diff hasn't been
> submitted upstream, because it depends on ort, and that still needs
> reviews...)

I certainly was thinking of remerge-diff when I was writing all of that,
but didn't want to get too far into the details. I think it's an
absolutely wonderful way of showing off conflict resolution, and I'm
excited to see it as a feature in Git. But:

  - as you note, it isn't a public feature yet ;)

  - while format-patch will learn about it because it knows all of the
    diff formats, the fundamental point of format-patch is to produce
    output that can be applied with git-am. So we'd only be halfway
    there.

  - it does still get weird specifying a merge at all as part of a
    series. Merging _what_ exactly? The topic branch made of the patches
    that were just sent?  A back-merge from some other commit on master
    into the topic branch? And if the latter, wouldn't that depend
    heavily on the base commit that the patches are applied on top of?

So I left it as "you could imagine" and "it's not quite trivial". ;)

-Peff