* [PATCH 0/2] fix union merge with binary files @ 2021-06-10 12:51 Jeff King 2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King ` (2 more replies) 0 siblings, 3 replies; 7+ messages in thread From: Jeff King @ 2021-06-10 12:51 UTC (permalink / raw) To: git; +Cc: Junio C Hamano This started as an attempt to silence a "gcc -O3" warning. But I was curious if we could trigger the problem it complains about in practice (spoiler: we can), so I wrote a test. And it seems there was an even bigger bug lurking, where we'd generate bogus merge results. :) This fixes both bugs. [1/2]: ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION [2/2]: ll_union_merge(): pass name labels to ll_xdl_merge() ll-merge.c | 6 ++++-- t/t6406-merge-attr.sh | 18 ++++++++++++++++++ 2 files changed, 22 insertions(+), 2 deletions(-) -Peff ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION 2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King @ 2021-06-10 12:57 ` Jeff King 2021-06-11 3:36 ` Junio C Hamano 2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King 2021-06-10 15:19 ` [PATCH 0/2] fix union merge with binary files Elijah Newren 2 siblings, 1 reply; 7+ messages in thread From: Jeff King @ 2021-06-10 12:57 UTC (permalink / raw) To: git; +Cc: Junio C Hamano Prior to commit a944af1d86 (merge: teach -Xours/-Xtheirs to binary ll-merge driver, 2012-09-08), we always reported a conflict from ll_binary_merge() by returning "1" (in the xdl_merge and ll_merge code, this value is the number of conflict hunks). After that commit, we report zero conflicts if the "variant" flag is set, under the assumption that it is one of XDL_MERGE_FAVOR_OURS or XDL_MERGE_FAVOR_THEIRS. But this gets confused by XDL_MERGE_FAVOR_UNION. We do not know how to do a binary union merge, but erroneously report no conflicts anyway (and just blindly use the "ours" content as the result). Let's tighten our check to just the cases that a944af1d86 meant to cover. This fixes the union case (which existed already back when that commit was made), as well as future-proofing us against any other variants that get added later. Note that you can't trigger this from "git merge-file --union", as that bails on binary files before even calling into the ll-merge machinery. The test here uses the "union" merge attribute, which does erroneously report a successful merge. Signed-off-by: Jeff King <peff@peff.net> --- ll-merge.c | 4 +++- t/t6406-merge-attr.sh | 17 +++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/ll-merge.c b/ll-merge.c index 9a8a2c365c..145deb12fa 100644 --- a/ll-merge.c +++ b/ll-merge.c @@ -91,7 +91,9 @@ static int ll_binary_merge(const struct ll_merge_driver *drv_unused, * With -Xtheirs or -Xours, we have cleanly merged; * otherwise we got a conflict. */ - return (opts->variant ? 0 : 1); + return opts->variant == XDL_MERGE_FAVOR_OURS || + opts->variant == XDL_MERGE_FAVOR_THEIRS ? + 0 : 1; } static int ll_xdl_merge(const struct ll_merge_driver *drv_unused, diff --git a/t/t6406-merge-attr.sh b/t/t6406-merge-attr.sh index d5a4ac2d81..c1c458d933 100755 --- a/t/t6406-merge-attr.sh +++ b/t/t6406-merge-attr.sh @@ -207,4 +207,21 @@ test_expect_success 'custom merge does not lock index' ' git merge main ' +test_expect_success 'binary files with union attribute' ' + git checkout -b bin-main && + printf "base\0" >bin.txt && + echo "bin.txt merge=union" >.gitattributes && + git add bin.txt .gitattributes && + git commit -m base && + + printf "one\0" >bin.txt && + git commit -am one && + + git checkout -b bin-side HEAD^ && + printf "two\0" >bin.txt && + git commit -am two && + + test_must_fail git merge bin-main +' + test_done -- 2.32.0.529.g079a794268 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION 2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King @ 2021-06-11 3:36 ` Junio C Hamano 0 siblings, 0 replies; 7+ messages in thread From: Junio C Hamano @ 2021-06-11 3:36 UTC (permalink / raw) To: Jeff King; +Cc: git Jeff King <peff@peff.net> writes: > Prior to commit a944af1d86 (merge: teach -Xours/-Xtheirs to binary > ll-merge driver, 2012-09-08), we always reported a conflict from > ll_binary_merge() by returning "1" (in the xdl_merge and ll_merge code, > this value is the number of conflict hunks). After that commit, we > report zero conflicts if the "variant" flag is set, under the assumption > that it is one of XDL_MERGE_FAVOR_OURS or XDL_MERGE_FAVOR_THEIRS. > > But this gets confused by XDL_MERGE_FAVOR_UNION. We do not know how to > do a binary union merge, but erroneously report no conflicts anyway (and > just blindly use the "ours" content as the result). > > Let's tighten our check to just the cases that a944af1d86 meant to > cover. This fixes the union case (which existed already back when that > commit was made), as well as future-proofing us against any other > variants that get added later. Makes sense. > Note that you can't trigger this from "git merge-file --union", as that > bails on binary files before even calling into the ll-merge machinery. > The test here uses the "union" merge attribute, which does erroneously > report a successful merge. Nice discovery. Thanks. ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() 2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King 2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King @ 2021-06-10 12:58 ` Jeff King 2021-06-10 15:19 ` Elijah Newren 2021-06-10 15:19 ` [PATCH 0/2] fix union merge with binary files Elijah Newren 2 siblings, 1 reply; 7+ messages in thread From: Jeff King @ 2021-06-10 12:58 UTC (permalink / raw) To: git; +Cc: Junio C Hamano Since cd1d61c44f (make union merge an xdl merge favor, 2010-03-01), we pass NULL to ll_xdl_merge() for the "name" labels of the ancestor, ours and theirs buffers. We usually use these for annotating conflict markers left in a file. For a union merge, these shouldn't matter; the point of it is that we'd never leave conflict markers in the first place. But there is one code path where we may dereference them: if the file contents appear to be binary, ll_binary_merge() will give up and pass them to warning() to generate a message for the user (that was true even when cd1d61c44f was written, though the warning was in ll_xdl_merge() back then). That can result in a segfault, though on many systems (including glibc), the printf routines will helpfully just say "(null)" instead. We can extend our binary-union test in t6406 to check stderr, which catches the problem on all systems. This also fixes a warning from "gcc -O3". Unlike lower optimization levels, it inlines enough to see that the NULL can make it to warning() and complains: In function ‘ll_binary_merge’, inlined from ‘ll_xdl_merge’ at ll-merge.c:115:10, inlined from ‘ll_union_merge’ at ll-merge.c:151:9: ll-merge.c:74:4: warning: ‘%s’ directive argument is null [-Wformat-overflow=] 74 | warning("Cannot merge binary files: %s (%s vs. %s)", | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 75 | path, name1, name2); | ~~~~~~~~~~~~~~~~~~~ Signed-off-by: Jeff King <peff@peff.net> --- ll-merge.c | 2 +- t/t6406-merge-attr.sh | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/ll-merge.c b/ll-merge.c index 145deb12fa..0ee34d8a01 100644 --- a/ll-merge.c +++ b/ll-merge.c @@ -151,7 +151,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused, o = *opts; o.variant = XDL_MERGE_FAVOR_UNION; return ll_xdl_merge(drv_unused, result, path_unused, - orig, NULL, src1, NULL, src2, NULL, + orig, orig_name, src1, name1, src2, name2, &o, marker_size); } diff --git a/t/t6406-merge-attr.sh b/t/t6406-merge-attr.sh index c1c458d933..8494645837 100755 --- a/t/t6406-merge-attr.sh +++ b/t/t6406-merge-attr.sh @@ -221,7 +221,8 @@ test_expect_success 'binary files with union attribute' ' printf "two\0" >bin.txt && git commit -am two && - test_must_fail git merge bin-main + test_must_fail git merge bin-main 2>stderr && + grep -i "warning.*cannot merge.*HEAD vs. bin-main" stderr ' test_done -- 2.32.0.529.g079a794268 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() 2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King @ 2021-06-10 15:19 ` Elijah Newren 2021-06-10 16:14 ` [PATCH 3/2] ll_union_merge(): rename path_unused parameter Jeff King 0 siblings, 1 reply; 7+ messages in thread From: Elijah Newren @ 2021-06-10 15:19 UTC (permalink / raw) To: Jeff King; +Cc: Git Mailing List, Junio C Hamano On Thu, Jun 10, 2021 at 6:00 AM Jeff King <peff@peff.net> wrote: > > Since cd1d61c44f (make union merge an xdl merge favor, 2010-03-01), we > pass NULL to ll_xdl_merge() for the "name" labels of the ancestor, ours > and theirs buffers. We usually use these for annotating conflict markers > left in a file. For a union merge, these shouldn't matter; the point of > it is that we'd never leave conflict markers in the first place. > > But there is one code path where we may dereference them: if the file > contents appear to be binary, ll_binary_merge() will give up and pass > them to warning() to generate a message for the user (that was true even > when cd1d61c44f was written, though the warning was in ll_xdl_merge() > back then). > > That can result in a segfault, though on many systems (including glibc), > the printf routines will helpfully just say "(null)" instead. We can > extend our binary-union test in t6406 to check stderr, which catches the > problem on all systems. Nice catch (as is your 1/2 as well). > This also fixes a warning from "gcc -O3". Unlike lower optimization > levels, it inlines enough to see that the NULL can make it to warning() > and complains: > > In function ‘ll_binary_merge’, > inlined from ‘ll_xdl_merge’ at ll-merge.c:115:10, > inlined from ‘ll_union_merge’ at ll-merge.c:151:9: > ll-merge.c:74:4: warning: ‘%s’ directive argument is null [-Wformat-overflow=] > 74 | warning("Cannot merge binary files: %s (%s vs. %s)", > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > 75 | path, name1, name2); > | ~~~~~~~~~~~~~~~~~~~ So the warning uses path as well as name1 and name2... > > Signed-off-by: Jeff King <peff@peff.net> > --- > ll-merge.c | 2 +- > t/t6406-merge-attr.sh | 3 ++- > 2 files changed, 3 insertions(+), 2 deletions(-) > > diff --git a/ll-merge.c b/ll-merge.c > index 145deb12fa..0ee34d8a01 100644 > --- a/ll-merge.c > +++ b/ll-merge.c > @@ -151,7 +151,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused, > o = *opts; > o.variant = XDL_MERGE_FAVOR_UNION; > return ll_xdl_merge(drv_unused, result, path_unused, Should we also rename 'path_unused' to 'path', since it is actually used? > - orig, NULL, src1, NULL, src2, NULL, > + orig, orig_name, src1, name1, src2, name2, > &o, marker_size); > } > > diff --git a/t/t6406-merge-attr.sh b/t/t6406-merge-attr.sh > index c1c458d933..8494645837 100755 > --- a/t/t6406-merge-attr.sh > +++ b/t/t6406-merge-attr.sh > @@ -221,7 +221,8 @@ test_expect_success 'binary files with union attribute' ' > printf "two\0" >bin.txt && > git commit -am two && > > - test_must_fail git merge bin-main > + test_must_fail git merge bin-main 2>stderr && > + grep -i "warning.*cannot merge.*HEAD vs. bin-main" stderr > ' > > test_done > -- > 2.32.0.529.g079a794268 This patch has a minor textual conflict with my remerge-diff series, but since I haven't submitted it yet, that's my problem rather than yours...and it will be an easy fix anyway. Anyway, good catches. Other than maybe considering fixing the name of 'path_unused', this looks good to me. ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 3/2] ll_union_merge(): rename path_unused parameter 2021-06-10 15:19 ` Elijah Newren @ 2021-06-10 16:14 ` Jeff King 0 siblings, 0 replies; 7+ messages in thread From: Jeff King @ 2021-06-10 16:14 UTC (permalink / raw) To: Elijah Newren; +Cc: Git Mailing List, Junio C Hamano On Thu, Jun 10, 2021 at 08:19:02AM -0700, Elijah Newren wrote: > > Signed-off-by: Jeff King <peff@peff.net> > > --- > > ll-merge.c | 2 +- > > t/t6406-merge-attr.sh | 3 ++- > > 2 files changed, 3 insertions(+), 2 deletions(-) > > > > diff --git a/ll-merge.c b/ll-merge.c > > index 145deb12fa..0ee34d8a01 100644 > > --- a/ll-merge.c > > +++ b/ll-merge.c > > @@ -151,7 +151,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused, > > o = *opts; > > o.variant = XDL_MERGE_FAVOR_UNION; > > return ll_xdl_merge(drv_unused, result, path_unused, > > Should we also rename 'path_unused' to 'path', since it is actually used? That seems reasonable, but I'd prefer to do it as a separate patch. So here that is. Ironically, orig_path was unused both before this patch (where we threw it away and passed NULL instead) and after (where we pass it into the xdl merge, but the union merge should ignore it completely). But I prefer not to go too wild with these kind of annotations, as they can easily become inaccurate or out of date. If we're passing it, then we shouldn't make too many assumptions about what xdl_merge() is doing under the hood. So we could also rename drv_unused mentioned below, but I didn't here. -- >8 -- Subject: [PATCH] ll_union_merge(): rename path_unused parameter The "path" parameter to ll_union_merge() is named "path_unused", since we don't ourselves use it. But we do pass it to ll_xdl_merge(), which may look at it (it gets passed to ll_binary_merge(), which may pass it to warning()). Let's rename it to correct this inaccuracy (both of the other functions correctly do not call this "unused"). Note that we also pass drv_unused, but it truly is unused by the rest of the stack (it only exists at all to provide a generic interface that matches what ll_ext_merge() needs). Reported-by: Elijah Newren <newren@gmail.com> Signed-off-by: Jeff King <peff@peff.net> --- ll-merge.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ll-merge.c b/ll-merge.c index 0ee34d8a01..261657578c 100644 --- a/ll-merge.c +++ b/ll-merge.c @@ -138,7 +138,7 @@ static int ll_xdl_merge(const struct ll_merge_driver *drv_unused, static int ll_union_merge(const struct ll_merge_driver *drv_unused, mmbuffer_t *result, - const char *path_unused, + const char *path, mmfile_t *orig, const char *orig_name, mmfile_t *src1, const char *name1, mmfile_t *src2, const char *name2, @@ -150,7 +150,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused, assert(opts); o = *opts; o.variant = XDL_MERGE_FAVOR_UNION; - return ll_xdl_merge(drv_unused, result, path_unused, + return ll_xdl_merge(drv_unused, result, path, orig, orig_name, src1, name1, src2, name2, &o, marker_size); } -- 2.32.0.529.g079a794268 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 0/2] fix union merge with binary files 2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King 2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King 2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King @ 2021-06-10 15:19 ` Elijah Newren 2 siblings, 0 replies; 7+ messages in thread From: Elijah Newren @ 2021-06-10 15:19 UTC (permalink / raw) To: Jeff King; +Cc: Git Mailing List, Junio C Hamano On Thu, Jun 10, 2021 at 5:54 AM Jeff King <peff@peff.net> wrote: > > This started as an attempt to silence a "gcc -O3" warning. But I was > curious if we could trigger the problem it complains about in practice > (spoiler: we can), so I wrote a test. And it seems there was an even > bigger bug lurking, where we'd generate bogus merge results. :) > > This fixes both bugs. Nice catches, and fixes. I had a minor comment on 2/2, but with or without fixing up the 'path_unused' variable name both patches are: Reviewed-by: Elijah Newren <newren@gmail.com> ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-06-11 3:36 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King 2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King 2021-06-11 3:36 ` Junio C Hamano 2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King 2021-06-10 15:19 ` Elijah Newren 2021-06-10 16:14 ` [PATCH 3/2] ll_union_merge(): rename path_unused parameter Jeff King 2021-06-10 15:19 ` [PATCH 0/2] fix union merge with binary files Elijah Newren
Code repositories for project(s) associated with this public inbox https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).