RFC: rebase inconsistency in 2.18 -- course of action?

RFC: rebase inconsistency in 2.18 -- course of action?

[Elijah Newren]

Hi everyone,

We've got a small rebase problem; I'm curious for thoughts about the
best course of action.  I'll provide several options below.

In short, while interactive-based and merge-based rebases handle
directory rename detection fine, am-based rebases do not.  This
inconsistency may frustrate or surprise users.

=== Demonstration Example ===

Let's say we have a repo with three commits -- an original commit
('O'), and two branches based off of it, A and B with an extra commit
each:

  O: has files l, x/a, x/b, x/c
  A: renames l -> lt, x/ -> y/
  B: modifies l, adds x/d

If I checkout B and run

  git rebase --interactive A

and don't bother editing the list of commits at all but just save,
then I end up with lt, y/{a, b, c, d} as expected.  However, if I run

  git rebase A

then I end up with lt, y/{a, b, c} and x/d; d is in the wrong place.

=== Problem explanation ===

am-based rebases suffer from a reduced ability to detect directory
renames upstream, which is fundamental to the fact that it throws away
information about the history: in particular, it dispenses with the
original commits involved by turning them into patches, and without
the knowledge of the original commits we cannot determine a proper
merge base.

The am-based rebase will proceed after generating patches by
commit-wise first trying git-apply, and if that fails it will attempt
a 3-way merge.  Since am has no knowledge of original commits, it
instead reconstructs a provisional merge base using
build_fake_ancestor() which will only contain the original versions of
the files modified in the patch.  Without the full list of files in
the real merge base, renames of any missing files cannot be detected.
Directory rename detection works by looking at individual file renames
and deducing when a full directory has been renamed.

Trying to modify build_fake_ancestor() to instead create a merge base
that includes common file information by looking for a commit that
contained all the same blobs as the pre-image of the patch would
require a very expensive search through history, and may choose a
wrong commit from among the several it finds.  (Also, regular am
outside of rebase may not find any commit that matches, forcing it to
somehow choose a near-fit).  Further, this would only work when the
fallback to a 3-way merge is triggered (i.e. when git-apply fails),
which may not happen for some patches. (For example, if the l->lt
change were not in the example, then git-apply would succeed and
incorrectly place d in x/, so we'd never even hit the 3-way fallback.)

In short, the am machinery simply doesn't have the necessary
information to properly detect directory renames, except when other
files involved in the renames upstream were also modified on the side
with the new file additions.

=== Possible solutions ===

1. Just accept it as a known shortcoming.

2. Revert directory rename detection from master...again.

   (Please, please, let's not pick this one.)

3. Claim it's not a bug at all.  In particular, the git-rebase(1) manpage
   states for the `--merge` option:

       Use merging strategies to rebase.  When the recursive (default)
       merge strategy is used, this allows rebase to be aware of
       renames on the upstream side.

   While "renames" is precisely why rebase --merge was added in commit
   58634dbff822 back on 2006-06-21, am-based rebases gained the
   ability to handle normal (non-directory) renames back in commit
   18cdf802ca6e from 2008-11-10.  So folks might be used to and expect
   their simple rebases to handle renames the same as --merge or
   --interactive rebases do.

4. Edit the 2.18 release notes.  Point out that directory renames can
   be detected by cherry-pick, merge, and rebases with either the -i
   or -m options (or any rebase option that implies either -i or -m);
   but not always by other rebases.  Folks may want to set pull.rebase
   to 'interactive' and just get used to immediately saving the
   popped-up list of commits and exiting to avoid this problem.

   Also, there's no good way to avoid this problem when using git-am
   directly.  At best, we can tell users to carefully select which
   older commit base to apply the patches on top of and then
   immediately run rebase --merge afterward.

5. Add an --am option to git-rebase, and make it not be the default
   (though make it be triggered by am-specific options like
   --whitespace or -C).  For the default, use either:

     5a. do_merge=t

     5b. interactive_rebase=implied

   As a slight aside, since there are only a few small changes
   necessary to make git-rebase--interactive handle ALL functionality
   from git-rebase--merge, we could take 5b one step further and
   delete git-rebase--merge and have one less rebase type.

   Either 5a or 5b may have a minor performance hit here, though
   personally I don't see this as a major factor.  But I'm happy to
   discuss at a high level if folks are curious (exact figures would
   depend heavily on the repo, size of commits, etc.)

6. Since there's not much time left before 2.18, do (1) and (4) for
   now, and (5) for 2.19.  Continue doing (1) or (3) for git-am itself.

7. Other solutions I haven't thought of?

=== Final Notes ===

I have patches implementing 5b, including the bonus removal of
git-rebase--merge.  However:

  * while my patches apply cleanly on master or next, there are lots
    of minor conflicts for the final two patches with ag/rebase-p in
    pu (in fact, even master conflicts with ag/rebase-p).  So I'm
    curious if I should wait off on sending these patches until that
    topic advances.

  * My first 9 patches are 7 independent fixes and cleanups, so I'll
    send those out shortly.


Thanks for reading this far,
Elijah

[PATCH] t3401: add directory rename testcases for rebase and am

[Elijah Newren]

Add a simple directory rename testcase, in conjunction with each of the
types of rebases:
  git-rebase--interactive
  git-rebase--am
  git-rebase--merge
and also use the same testcase for
  git am --3way

This demonstrates an inconsistency between the different rebase backends
and which can detect the directory rename.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3401-rebase-and-am-rename.sh | 105 ++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100755 t/t3401-rebase-and-am-rename.sh

diff --git a/t/t3401-rebase-and-am-rename.sh b/t/t3401-rebase-and-am-rename.sh
new file mode 100755
index 0000000000..8f832957fc
--- /dev/null
+++ b/t/t3401-rebase-and-am-rename.sh
@@ -0,0 +1,105 @@
+#!/bin/sh
+
+test_description='git rebase + directory rename tests'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup testcase' '
+	test_create_repo dir-rename &&
+	(
+		cd dir-rename &&
+
+		mkdir x &&
+		test_seq  1 10 >x/a &&
+		test_seq 11 20 >x/b &&
+		test_seq 21 30 >x/c &&
+		test_write_lines a b c d e f g h i >l &&
+		git add x l &&
+		git commit -m "Initial" &&
+
+		git branch O &&
+		git branch A &&
+		git branch B &&
+
+		git checkout A &&
+		git mv x y &&
+		git mv l letters &&
+		git commit -m "Rename x to y, l to letters" &&
+
+		git checkout B &&
+		echo j >>l &&
+		test_seq 31 40 >x/d &&
+		git add l x/d &&
+		git commit -m "Modify l, add x/d"
+	)
+'
+
+test_expect_success 'rebase --interactive: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		set_fake_editor &&
+		FAKE_LINES="1" git rebase --interactive A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_failure 'rebase (am): directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		git rebase A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_success 'rebase --merge: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		git rebase --merge A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_failure 'am: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout A^0 &&
+
+		git format-patch -1 B &&
+
+		git am --3way 0001*.patch &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_done
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH] apply: fix grammar error in comment

[Elijah Newren]

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 apply.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/apply.c b/apply.c
index d79e61591b..85f2c92740 100644
--- a/apply.c
+++ b/apply.c
@@ -4053,7 +4053,7 @@ static int preimage_oid_in_gitlink_patch(struct patch *p, struct object_id *oid)
 	return get_oid_hex(p->old_sha1_prefix, oid);
 }
 
-/* Build an index that contains the just the files needed for a 3way merge */
+/* Build an index that contains just the files needed for a 3way merge */
 static int build_fake_ancestor(struct apply_state *state, struct patch *list)
 {
 	struct patch *patch;
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH] t5407: fix test to cover intended arguments

[Elijah Newren]

Test 8 in t5407 appears to be an accidental exact duplicate of of test 5;
the testcode is identical and has identical repo state, but the test
description is different and suggests that rebase -m followed by rebase
--skip was what was actually supposed to be tested.  Modify the test to
include the -m option.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t5407-post-rewrite-hook.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t5407-post-rewrite-hook.sh b/t/t5407-post-rewrite-hook.sh
index 7a48236e87..9b2a274c71 100755
--- a/t/t5407-post-rewrite-hook.sh
+++ b/t/t5407-post-rewrite-hook.sh
@@ -113,7 +113,7 @@ test_expect_success 'git rebase -m' '
 test_expect_success 'git rebase -m --skip' '
 	git reset --hard D &&
 	clear_hook_input &&
-	test_must_fail git rebase --onto A B &&
+	test_must_fail git rebase -m --onto A B &&
 	test_must_fail git rebase --skip &&
 	echo D > foo &&
 	git add foo &&
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH] git-rebase--merge: modernize "git-$cmd" to "git $cmd"

[Elijah Newren]

<Comments for after diffstat:>
I tend to think git-rebase--merge is less popular than the other rebase
types, leading to it being more overlooked and less well tested than the
other ones.  This git-$cmd usage seems to support that argument.

Anyway, this patch may be irrelevant if others agree with my goal to
delete git-rebase--merge and implement --merge on top of the --interactive
machinery, but sending it along in case others don't agree with that goal.
</Comments>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase--merge.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-rebase--merge.sh b/git-rebase--merge.sh
index cf4c042214..aa2f2f0872 100644
--- a/git-rebase--merge.sh
+++ b/git-rebase--merge.sh
@@ -71,7 +71,7 @@ call_merge () {
 	test -z "$strategy" && strategy=recursive
 	# If cmt doesn't have a parent, don't include it as a base
 	base=$(git rev-parse --verify --quiet $cmt^)
-	eval 'git-merge-$strategy' $strategy_opts $base ' -- "$hd" "$cmt"'
+	eval 'git merge-$strategy' $strategy_opts $base ' -- "$hd" "$cmt"'
 	rv=$?
 	case "$rv" in
 	0)
@@ -88,7 +88,7 @@ call_merge () {
 		;;
 	*)
 		die "Unknown exit code ($rv) from command:" \
-			"git-merge-$strategy $cmt^ -- HEAD $cmt"
+			"git merge-$strategy $cmt^ -- HEAD $cmt"
 		;;
 	esac
 }
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH 1/2] t3422: new testcases for checking when incompatible options passed

[Elijah Newren]

git rebase is split into three types: am, merge, and interactive.  Various
options imply different types, and which mode we are using determine which
sub-script (git-rebase--$type) is executed to finish the work.  Not all
options work with all types, so add tests for combinations where we expect
to receive an error rather than having options be silently ignored.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3422-rebase-incompatible-options.sh | 69 ++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100755 t/t3422-rebase-incompatible-options.sh

diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
new file mode 100755
index 0000000000..04cdae921b
--- /dev/null
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -0,0 +1,69 @@
+#!/bin/sh
+
+test_description='test if rebase detects and aborts on incompatible options'
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	test_seq 2 9 >foo &&
+	git add foo &&
+	git commit -m orig &&
+
+	git branch A &&
+	git branch B &&
+
+	git checkout A &&
+	test_seq 1 9 >foo &&
+	git add foo &&
+	git commit -m A &&
+
+	git checkout B &&
+	# This is indented with HT SP HT.
+	echo "	 	foo();" >>foo &&
+	git add foo &&
+	git commit -m B
+'
+
+#
+# Rebase has lots of useful options like --whitepsace=fix, which are
+# actually all built in terms of flags to git-am.  Since neither
+# --merge nor --interactive (nor any options that imply those two) use
+# git-am, using them together will result in flags like --whitespace=fix
+# being ignored.  Make sure rebase warns the user and aborts instead.
+#
+
+test_run_rebase () {
+	opt=$1
+	shift
+	test_expect_failure "$opt incompatible with --merge" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --merge A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --interactive" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --interactive A
+	"
+
+	test_expect_failure "$opt incompatible with --exec" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --exec 'true' A
+	"
+
+}
+
+test_run_rebase --whitespace=fix
+test_run_rebase --ignore-whitespace
+test_run_rebase --committer-date-is-author-date
+test_run_rebase -C4
+
+test_done
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH 2/2] git-rebase: error out when incompatible options passed

[Elijah Newren]

git rebase has three different types: am, merge, and interactive, all of
which are implemented in terms of separate scripts.  am builds on git-am,
merge builds on git-merge-recursive, and interactive builds on
git-cherry-pick.  We make use of features in those lower-level commands in
the different rebase types, but those features don't exist in all of the
lower level commands so we have a range of incompatibilities.  Previously,
we just accepted nearly any argument and silently ignored whichever ones
weren't implemented for the type of rebase specified.  Change this so the
incompatibilities are documented, included in the testsuite, and tested
for at runtime with an appropriate error message shown.

Some exceptions I left out:

  * --merge and --interactive are technically incompatible since they are
    supposed to run different underlying scripts, but with a few small
    changes, --interactive can do everything that --merge can.  In fact,
    I'll shortly be sending another patch to remove git-rebase--merge and
    reimplement it on top of git-rebase--interactive.

  * One could argue that --interactive and --quiet are incompatible since
    --interactive doesn't implement a --quiet mode (perhaps since
    cherry-pick itself does not implement one).  However, the interactive
    mode is more quiet than the other modes in general with progress
    messages, so one could argue that it's already quiet.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt           | 15 +++++++++++++--
 git-rebase.sh                          | 17 +++++++++++++++++
 t/t3422-rebase-incompatible-options.sh | 10 +++++-----
 3 files changed, 35 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 0e20a66e73..451252c173 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -243,6 +243,10 @@ leave out at most one of A and B, in which case it defaults to HEAD.
 --keep-empty::
 	Keep the commits that do not change anything from its
 	parents in the result.
++
+This uses the `--interactive` machinery internally, and as such,
+anything that is incompatible with --interactive is incompatible
+with this option.
 
 --allow-empty-message::
 	By default, rebasing commits with an empty message will fail.
@@ -324,6 +328,8 @@ which makes little sense.
 	and after each change.  When fewer lines of surrounding
 	context exist they all must match.  By default no context is
 	ever ignored.
+	Incompatible with the --merge and --interactive options, or
+	anything that implies those options or their machinery.
 
 -f::
 --force-rebase::
@@ -355,13 +361,15 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 --whitespace=<option>::
 	These flag are passed to the 'git apply' program
 	(see linkgit:git-apply[1]) that applies the patch.
-	Incompatible with the --interactive option.
+	Incompatible with the --merge and --interactive options, or
+	anything that implies those options or their machinery.
 
 --committer-date-is-author-date::
 --ignore-date::
 	These flags are passed to 'git am' to easily change the dates
 	of the rebased commits (see linkgit:git-am[1]).
-	Incompatible with the --interactive option.
+	Incompatible with the --merge and --interactive options, or
+	anything that implies those options or their machinery.
 
 --signoff::
 	Add a Signed-off-by: trailer to all the rebased commits. Note
@@ -400,6 +408,9 @@ The `--rebase-merges` mode is similar in spirit to `--preserve-merges`, but
 in contrast to that option works well in interactive rebases: commits can be
 reordered, inserted and dropped at will.
 +
+This uses the `--interactive` machinery internally, but it can be run
+without an explicit `--interactive`.
++
 It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
 explicit `exec git merge -s <strategy> [...]` commands.
diff --git a/git-rebase.sh b/git-rebase.sh
index 40be59ecc4..f1dbecba18 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -503,6 +503,23 @@ then
 	git_format_patch_opt="$git_format_patch_opt --progress"
 fi
 
+if test -n "$git_am_opt"; then
+	incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
+	if test -n "$interactive_rebase"
+	then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
+		fi
+	fi
+	if test -n "$do_merge"; then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
+		fi
+	fi
+fi
+
 if test -n "$signoff"
 then
 	test -n "$preserve_merges" &&
diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
index 04cdae921b..66a83363bf 100755
--- a/t/t3422-rebase-incompatible-options.sh
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -34,27 +34,27 @@ test_expect_success 'setup' '
 test_run_rebase () {
 	opt=$1
 	shift
-	test_expect_failure "$opt incompatible with --merge" "
+	test_expect_success "$opt incompatible with --merge" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --merge A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy=ours" "
+	test_expect_success "$opt incompatible with --strategy=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+	test_expect_success "$opt incompatible with --strategy-option=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --interactive" "
+	test_expect_success "$opt incompatible with --interactive" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --interactive A
 	"
 
-	test_expect_failure "$opt incompatible with --exec" "
+	test_expect_success "$opt incompatible with --exec" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --exec 'true' A
 	"
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH] git-rebase.sh: handle keep-empty like all other options

[Elijah Newren]

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 40be59ecc4..a56b286372 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -276,6 +276,7 @@ do
 		;;
 	--keep-empty)
 		keep_empty=yes
+		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
 	--allow-empty-message)
 		allow_empty_message=--allow-empty-message
@@ -480,11 +481,6 @@ then
 	test -z "$interactive_rebase" && interactive_rebase=implied
 fi
 
-if test -n "$keep_empty"
-then
-	test -z "$interactive_rebase" && interactive_rebase=implied
-fi
-
 if test -n "$interactive_rebase"
 then
 	type=interactive
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Elijah Newren]

We are not passing the same args to merge strategies when we are doing an
--interactive rebase as we do with a --merge rebase.  The merge strategy
should not need to be aware of which type of rebase is in effect.  Add a
testcase which checks for the appropriate args.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3418-rebase-continue.sh | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 03bf1b8a3b..872022106f 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
 	test -f funny.was.run
 '
 
+test_expect_failure 'rebase -i --continue handles merge strategy and options' '
+	rm -fr .git/rebase-* &&
+	git reset --hard commit-new-file-F2-on-topic-branch &&
+	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
+	test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
+	mkdir test-bin &&
+	cat >test-bin/git-merge-funny <<-EOF &&
+	#!$SHELL_PATH
+	echo "\$@" >>funny.args
+	case "\$1" in --opt) ;; *) exit 2 ;; esac
+	case "\$2" in --foo) ;; *) exit 2 ;; esac
+	case "\$4" in --) ;; *) exit 2 ;; esac
+	shift 2 &&
+	>funny.was.run &&
+	exec git merge-recursive "\$@"
+	EOF
+	chmod +x test-bin/git-merge-funny &&
+	(
+		PATH=./test-bin:$PATH
+		test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
+	) &&
+	test -f funny.was.run &&
+	rm funny.was.run &&
+	echo "Resolved" >F2 &&
+	git add F2 &&
+	(
+		PATH=./test-bin:$PATH
+		git rebase --continue
+	) &&
+	test -f funny.was.run
+'
+
 test_expect_success 'rebase passes merge strategy options correctly' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F3-on-topic-branch &&
-- 
2.18.0.rc0.46.g9cee8fce43

[PATCH 2/2] Fix use of strategy options with interactive rebases

[Elijah Newren]

git-rebase.sh wrote stuff like
  '--ours'  '--renormalize'
to .git/rebase-merge/strategy_opts.  Note the double spaces.

git-rebase--interactive uses sequencer.c to parse that file, and
sequencer.c used split_cmdline() to get the individual strategy options.
After splitting, sequencer.c prefixed each "option" with a double dash,
so, concatenating all its options would result in:
  -- --ours -- --renormalize

So, when it ended up calling try_merge_strategy(), that in turn would run
  git merge-$strategy -- --ours -- --renormalize $merge_base -- $head $remote

instead of the expected
  git merge-$strategy --ours --renormalize $merge_base -- $head $remote

Remove the extra spaces so that split_cmdline() will work as expected.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh              | 2 +-
 sequencer.c                | 7 ++++++-
 t/t3418-rebase-continue.sh | 2 +-
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 40be59ecc4..224d5ebc29 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -316,7 +316,7 @@ do
 		do_merge=t
 		;;
 	--strategy-option=*)
-		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}")"
+		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}" | sed -e s/^.//)"
 		do_merge=t
 		test -z "$strategy" && strategy=recursive
 		;;
diff --git a/sequencer.c b/sequencer.c
index cca968043e..a2843e3906 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2203,6 +2203,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 {
 	int i;
+	char *strategy_opts_string;
 
 	strbuf_reset(buf);
 	if (!read_oneliner(buf, rebase_path_strategy(), 0))
@@ -2211,7 +2212,11 @@ static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
 		return;
 
-	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
+	strategy_opts_string = buf->buf;
+	if (*strategy_opts_string == ' ')
+		strategy_opts_string++;
+	opts->xopts_nr = split_cmdline(strategy_opts_string,
+				       (const char ***)&opts->xopts);
 	for (i = 0; i < opts->xopts_nr; i++) {
 		const char *arg = opts->xopts[i];
 
diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 872022106f..7ca6cbc415 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -74,7 +74,7 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
 	test -f funny.was.run
 '
 
-test_expect_failure 'rebase -i --continue handles merge strategy and options' '
+test_expect_success 'rebase -i --continue handles merge strategy and options' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F2-on-topic-branch &&
 	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
-- 
2.18.0.rc0.46.g9cee8fce43

Re: [PATCH] git-rebase--merge: modernize "git-$cmd" to "git $cmd"

[Elijah Newren]

On Wed, Jun 6, 2018 at 10:06 PM, Elijah Newren <newren@gmail.com> wrote:
> <Comments for after diffstat:>
> I tend to think git-rebase--merge is less popular than the other rebase
> types, leading to it being more overlooked and less well tested than the
> other ones.  This git-$cmd usage seems to support that argument.
>
> Anyway, this patch may be irrelevant if others agree with my goal to
> delete git-rebase--merge and implement --merge on top of the --interactive
> machinery, but sending it along in case others don't agree with that goal.
> </Comments>

I would forget to pull that out of the commit message area and put
that next to the diffstat in the email.  Whoops.

Oh, well.

[RFC PATCH 0/3] Send more rebases through interactive machinery

[Elijah Newren]

This series:

  * deletes git-rebase--merge, making git-rebase--interactive handle
    those cases instead
  * adds an --am option to git rebase, and changes the rebase default
    from using git-rebase--am to git-rebase--interactive, fixing
    directory rename detection for default rebases.

However:

  * this series has several minor conflicts with ag/rebase-p in pu.
    But it's just an RFC for now; I can re-roll on top of that after
    getting some feedback.

  * this series depends on the other preparatory fixups under
      https://public-inbox.org/git/CABPp-BGxaroePB6aKWAkZeADLB7VE3y1CPy2RyNwpn=+C01g3A@mail.gmail.com/
    To get this series with the preparatory fixups:

Fetch-It-Via: git fetch https://github.com/newren/git rebase-new-default

Elijah Newren (3):
  git-rebase, sequencer: add a --quiet option for the interactive
    machinery
  rebase: Implement --merge via git-rebase--interactive
  git-rebase.sh: make git-rebase--interactive the default

 .gitignore                        |   1 -
 Documentation/git-rebase.txt      |  24 +++--
 Makefile                          |   1 -
 git-rebase--interactive.sh        |  14 ++-
 git-rebase--merge.sh              | 164 ------------------------------
 git-rebase.sh                     |  75 ++++++++------
 sequencer.c                       |  19 ++--
 sequencer.h                       |   1 +

Yes, there are a significant number of testcase changes below.  See
the relevant commit messages where I explain why each was changed.

 t/t3400-rebase.sh                 |   7 +-
 t/t3401-rebase-and-am-rename.sh   |  18 +++-
 t/t3404-rebase-interactive.sh     |   2 +-
 t/t3406-rebase-message.sh         |   7 +-
 t/t3407-rebase-abort.sh           |  28 ++++-
 t/t3420-rebase-autostash.sh       |  89 +++-------------
 t/t3421-rebase-topology-linear.sh |  10 +-
 t/t3425-rebase-topology-merges.sh |  15 +--
 t/t5407-post-rewrite-hook.sh      |   5 +-
 t/t5520-pull.sh                   |   9 +-
 t/t9903-bash-prompt.sh            |  12 ++-
 19 files changed, 180 insertions(+), 321 deletions(-)
 delete mode 100644 git-rebase--merge.sh

-- 
2.18.0.rc1.12.g2996b9442d

[RFC PATCH 1/3] git-rebase, sequencer: add a --quiet option for the interactive machinery

[Elijah Newren]

While 'quiet' and 'interactive' may sound like antonyms, the interactive
machinery actually has logic that implements several
interactive_rebase=implied cases (--exec, --keep-empty, --rebase-merges)
which won't pop up an editor.  Further, we want to make the interactive
machinery also take over for git-rebase--merge and become the default
merge strategy, so it makes sense for these other cases to have a quiet
option.

git-rebase--interactive was already somewhat quieter than
git-rebase--merge and git-rebase--am, possibly because cherry-pick has
just traditionally been quieter.  As such, we only drop a few
informational messages -- "Rebasing (n/m)" and "Succesfully rebased..."

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase--interactive.sh |  9 +++++++--
 git-rebase.sh              |  2 +-
 sequencer.c                | 19 +++++++++++++------
 sequencer.h                |  1 +
 4 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 06a7b79307..1f2401f702 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -139,8 +139,12 @@ mark_action_done () {
 	if test "$last_count" != "$new_count"
 	then
 		last_count=$new_count
-		eval_gettext "Rebasing (\$new_count/\$total)"; printf "\r"
-		test -z "$verbose" || echo
+		if test -z "$GIT_QUIET"
+		then
+			eval_gettext "Rebasing (\$new_count/\$total)";
+			printf "\r"
+			test -z "$verbose" || echo
+		fi
 	fi
 }
 
@@ -713,6 +717,7 @@ Commit or stash your changes, and then run
 		"$hook" rebase < "$rewritten_list"
 		true # we don't care if this hook failed
 	fi &&
+		test -z "$GIT_QUIET" &&
 		warn "$(eval_gettext "Successfully rebased and updated \$head_name.")"
 
 	return 1 # not failure; just to break the do_rest loop
diff --git a/git-rebase.sh b/git-rebase.sh
index 7d1612b31b..b639c0d4fe 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -136,7 +136,7 @@ write_basic_state () {
 	echo "$head_name" > "$state_dir"/head-name &&
 	echo "$onto" > "$state_dir"/onto &&
 	echo "$orig_head" > "$state_dir"/orig-head &&
-	echo "$GIT_QUIET" > "$state_dir"/quiet &&
+	test t = "$GIT_QUIET" && : > "$state_dir"/quiet
 	test t = "$verbose" && : > "$state_dir"/verbose
 	test -n "$strategy" && echo "$strategy" > "$state_dir"/strategy
 	test -n "$strategy_opts" && echo "$strategy_opts" > \
diff --git a/sequencer.c b/sequencer.c
index a2843e3906..d418d40bee 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -143,6 +143,7 @@ static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
+static GIT_PATH_FUNC(rebase_path_quiet, "rebase-merge/quiet")
 static GIT_PATH_FUNC(rebase_path_signoff, "rebase-merge/signoff")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
@@ -2251,6 +2252,9 @@ static int read_populate_opts(struct replay_opts *opts)
 		if (file_exists(rebase_path_verbose()))
 			opts->verbose = 1;
 
+		if (file_exists(rebase_path_quiet()))
+			opts->quiet = 1;
+
 		if (file_exists(rebase_path_signoff())) {
 			opts->allow_ff = 0;
 			opts->signoff = 1;
@@ -3171,10 +3175,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					fprintf(f, "%d\n", todo_list->done_nr);
 					fclose(f);
 				}
-				fprintf(stderr, "Rebasing (%d/%d)%s",
-					todo_list->done_nr,
-					todo_list->total_nr,
-					opts->verbose ? "\n" : "\r");
+				if (!opts->quiet)
+					fprintf(stderr, "Rebasing (%d/%d)%s",
+						todo_list->done_nr,
+						todo_list->total_nr,
+						opts->verbose ? "\n" : "\r");
 			}
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
@@ -3385,8 +3390,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
-		fprintf(stderr, "Successfully rebased and updated %s.\n",
-			head_ref.buf);
+		if (!opts->quiet)
+			fprintf(stderr,
+				"Successfully rebased and updated %s.\n",
+				head_ref.buf);
 
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
diff --git a/sequencer.h b/sequencer.h
index c5787c6b56..da652a421f 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -33,6 +33,7 @@ struct replay_opts {
 	int allow_empty_message;
 	int keep_redundant_commits;
 	int verbose;
+	int quiet;
 
 	int mainline;
 
-- 
2.18.0.rc1.12.g2996b9442d

[RFC PATCH 2/3] rebase: Implement --merge via git-rebase--interactive

[Elijah Newren]

Interactive rebases are implemented in terms of cherry-pick rather than
the merge-recursive builtin, but cherry-pick also calls into the recursive
merge machinery by default and can accept special merge strategies and/or
special strategy options.  As such, there really is not any need for
having both git-rebase--merge and git-rebase--interactive anymore.

Delete git-rebase--merge.sh and have the --merge option call in to
git-rebase--interactive.  Add one new variable ($actually_interactive) to
help keep cases separate, and adjust the detection of the special
circumstances under which a rebase boils down to a simple fast forward so
that it keeps working with the new structure.

Note that this change fixes a few known test failures (see t3421).

testcase modification notes:
  t3406: --interactive and --merge had slightly different progress output
         while running; adjust a test to match
  t3420: tests of precise output while running, but rebase--am,
         rebase--merge, and rebase--interactive all were built on very
         different commands (am, merge-recursive, cherry-pick), so the
         tests expected different output for each type.  Now we expect
         --merge and --interactive to have the same output.
  t3421: --interactive fixes some bugs in --merge!  Wahoo!
  t3425: topology linearization was inconsistent across flavors of rebase,
         as already noted in a TODO comment in the testcase.  This was not
         considered a bug before, so getting a different linearization due
         to switching out backends should not be considered a bug now.
  t5407: different rebase types varied slightly in how many times checkout
         or commit or equivalents were called based on a quick comparison
         of this tests and previous ones which covered different rebase
         flavors.  I think this is just attributable to this difference.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 .gitignore                        |   1 -
 Documentation/git-rebase.txt      |  16 +--
 Makefile                          |   1 -
 git-rebase--interactive.sh        |   5 +-
 git-rebase--merge.sh              | 164 ------------------------------
 git-rebase.sh                     |  49 ++++-----
 t/t3406-rebase-message.sh         |   7 +-
 t/t3420-rebase-autostash.sh       |  78 ++------------
 t/t3421-rebase-topology-linear.sh |  10 +-
 t/t3425-rebase-topology-merges.sh |   6 +-
 t/t5407-post-rewrite-hook.sh      |   1 +
 11 files changed, 55 insertions(+), 283 deletions(-)
 delete mode 100644 git-rebase--merge.sh

diff --git a/.gitignore b/.gitignore
index 388cc4beee..747be69d10 100644
--- a/.gitignore
+++ b/.gitignore
@@ -118,7 +118,6 @@
 /git-rebase--am
 /git-rebase--helper
 /git-rebase--interactive
-/git-rebase--merge
 /git-receive-pack
 /git-reflog
 /git-remote
diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 451252c173..28d1658d7a 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -275,6 +275,10 @@ branch on top of the <upstream> branch.  Because of this, when a merge
 conflict happens, the side reported as 'ours' is the so-far rebased
 series, starting with <upstream>, and 'theirs' is the working branch.  In
 other words, the sides are swapped.
++
+This uses the `--interactive` machinery internally, and as such,
+anything that is incompatible with --interactive is incompatible
+with this option.
 
 -s <strategy>::
 --strategy=<strategy>::
@@ -328,8 +332,8 @@ which makes little sense.
 	and after each change.  When fewer lines of surrounding
 	context exist they all must match.  By default no context is
 	ever ignored.
-	Incompatible with the --merge and --interactive options, or
-	anything that implies those options or their machinery.
+	Incompatible with the --interactive option, or anything that
+	uses the `--interactive` machinery.
 
 -f::
 --force-rebase::
@@ -361,15 +365,15 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 --whitespace=<option>::
 	These flag are passed to the 'git apply' program
 	(see linkgit:git-apply[1]) that applies the patch.
-	Incompatible with the --merge and --interactive options, or
-	anything that implies those options or their machinery.
+	Incompatible with the --interactive option, or anything that
+	uses the `--interactive` machinery.
 
 --committer-date-is-author-date::
 --ignore-date::
 	These flags are passed to 'git am' to easily change the dates
 	of the rebased commits (see linkgit:git-am[1]).
-	Incompatible with the --merge and --interactive options, or
-	anything that implies those options or their machinery.
+	Incompatible with the --interactive option, or anything that
+	uses the `--interactive` machinery.
 
 --signoff::
 	Add a Signed-off-by: trailer to all the rebased commits. Note
diff --git a/Makefile b/Makefile
index 1d27f36365..8d24aaf638 100644
--- a/Makefile
+++ b/Makefile
@@ -620,7 +620,6 @@ SCRIPT_LIB += git-mergetool--lib
 SCRIPT_LIB += git-parse-remote
 SCRIPT_LIB += git-rebase--am
 SCRIPT_LIB += git-rebase--interactive
-SCRIPT_LIB += git-rebase--merge
 SCRIPT_LIB += git-sh-setup
 SCRIPT_LIB += git-sh-i18n
 
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 1f2401f702..dcc4a26a78 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -885,7 +885,10 @@ init_basic_state () {
 	mkdir -p "$state_dir" || die "$(eval_gettext "Could not create temporary \$state_dir")"
 	rm -f "$(git rev-parse --git-path REBASE_HEAD)"
 
-	: > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
+	if test -n "$actually_interactive"
+	then
+		: > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
+	fi
 	write_basic_state
 }
 
diff --git a/git-rebase--merge.sh b/git-rebase--merge.sh
deleted file mode 100644
index aa2f2f0872..0000000000
--- a/git-rebase--merge.sh
+++ /dev/null
@@ -1,164 +0,0 @@
-# This shell script fragment is sourced by git-rebase to implement
-# its merge-based non-interactive mode that copes well with renamed
-# files.
-#
-# Copyright (c) 2010 Junio C Hamano.
-#
-
-prec=4
-
-read_state () {
-	onto_name=$(cat "$state_dir"/onto_name) &&
-	end=$(cat "$state_dir"/end) &&
-	msgnum=$(cat "$state_dir"/msgnum)
-}
-
-continue_merge () {
-	test -d "$state_dir" || die "$state_dir directory does not exist"
-
-	unmerged=$(git ls-files -u)
-	if test -n "$unmerged"
-	then
-		echo "You still have unmerged paths in your index"
-		echo "did you forget to use git add?"
-		die "$resolvemsg"
-	fi
-
-	cmt=$(cat "$state_dir/current")
-	if ! git diff-index --quiet --ignore-submodules HEAD --
-	then
-		if ! git commit ${gpg_sign_opt:+"$gpg_sign_opt"} $signoff $allow_empty_message \
-			--no-verify -C "$cmt"
-		then
-			echo "Commit failed, please do not call \"git commit\""
-			echo "directly, but instead do one of the following: "
-			die "$resolvemsg"
-		fi
-		if test -z "$GIT_QUIET"
-		then
-			printf "Committed: %0${prec}d " $msgnum
-		fi
-		echo "$cmt $(git rev-parse HEAD^0)" >> "$state_dir/rewritten"
-	else
-		if test -z "$GIT_QUIET"
-		then
-			printf "Already applied: %0${prec}d " $msgnum
-		fi
-	fi
-	test -z "$GIT_QUIET" &&
-	GIT_PAGER='' git log --format=%s -1 "$cmt"
-
-	# onto the next patch:
-	msgnum=$(($msgnum + 1))
-	echo "$msgnum" >"$state_dir/msgnum"
-}
-
-call_merge () {
-	msgnum="$1"
-	echo "$msgnum" >"$state_dir/msgnum"
-	cmt="$(cat "$state_dir/cmt.$msgnum")"
-	echo "$cmt" > "$state_dir/current"
-	git update-ref REBASE_HEAD "$cmt"
-	hd=$(git rev-parse --verify HEAD)
-	cmt_name=$(git symbolic-ref HEAD 2> /dev/null || echo HEAD)
-	eval GITHEAD_$cmt='"${cmt_name##refs/heads/}~$(($end - $msgnum))"'
-	eval GITHEAD_$hd='$onto_name'
-	export GITHEAD_$cmt GITHEAD_$hd
-	if test -n "$GIT_QUIET"
-	then
-		GIT_MERGE_VERBOSITY=1 && export GIT_MERGE_VERBOSITY
-	fi
-	test -z "$strategy" && strategy=recursive
-	# If cmt doesn't have a parent, don't include it as a base
-	base=$(git rev-parse --verify --quiet $cmt^)
-	eval 'git merge-$strategy' $strategy_opts $base ' -- "$hd" "$cmt"'
-	rv=$?
-	case "$rv" in
-	0)
-		unset GITHEAD_$cmt GITHEAD_$hd
-		return
-		;;
-	1)
-		git rerere $allow_rerere_autoupdate
-		die "$resolvemsg"
-		;;
-	2)
-		echo "Strategy: $strategy failed, try another" 1>&2
-		die "$resolvemsg"
-		;;
-	*)
-		die "Unknown exit code ($rv) from command:" \
-			"git merge-$strategy $cmt^ -- HEAD $cmt"
-		;;
-	esac
-}
-
-finish_rb_merge () {
-	move_to_original_branch
-	if test -s "$state_dir"/rewritten
-	then
-		git notes copy --for-rewrite=rebase <"$state_dir"/rewritten
-		hook="$(git rev-parse --git-path hooks/post-rewrite)"
-		test -x "$hook" && "$hook" rebase <"$state_dir"/rewritten
-	fi
-	say All done.
-}
-
-git_rebase__merge () {
-
-case "$action" in
-continue)
-	read_state
-	continue_merge
-	while test "$msgnum" -le "$end"
-	do
-		call_merge "$msgnum"
-		continue_merge
-	done
-	finish_rb_merge
-	return
-	;;
-skip)
-	read_state
-	git rerere clear
-	msgnum=$(($msgnum + 1))
-	while test "$msgnum" -le "$end"
-	do
-		call_merge "$msgnum"
-		continue_merge
-	done
-	finish_rb_merge
-	return
-	;;
-show-current-patch)
-	exec git show REBASE_HEAD --
-	;;
-esac
-
-mkdir -p "$state_dir"
-echo "$onto_name" > "$state_dir/onto_name"
-write_basic_state
-rm -f "$(git rev-parse --git-path REBASE_HEAD)"
-
-msgnum=0
-for cmt in $(git rev-list --reverse --no-merges "$revisions")
-do
-	msgnum=$(($msgnum + 1))
-	echo "$cmt" > "$state_dir/cmt.$msgnum"
-done
-
-echo 1 >"$state_dir/msgnum"
-echo $msgnum >"$state_dir/end"
-
-end=$msgnum
-msgnum=1
-
-while test "$msgnum" -le "$end"
-do
-	call_merge "$msgnum"
-	continue_merge
-done
-
-finish_rb_merge
-
-}
diff --git a/git-rebase.sh b/git-rebase.sh
index b639c0d4fe..5ac1dee30b 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -239,12 +239,10 @@ then
 	state_dir="$apply_dir"
 elif test -d "$merge_dir"
 then
+	type=interactive
 	if test -f "$merge_dir"/interactive
 	then
-		type=interactive
 		interactive_rebase=explicit
-	else
-		type=merge
 	fi
 	state_dir="$merge_dir"
 fi
@@ -481,13 +479,16 @@ then
 	test -z "$interactive_rebase" && interactive_rebase=implied
 fi
 
+actually_interactive=
 if test -n "$interactive_rebase"
 then
 	type=interactive
+	actually_interactive=t
 	state_dir="$merge_dir"
 elif test -n "$do_merge"
 then
-	type=merge
+	interactive_rebase=implied
+	type=interactive
 	state_dir="$merge_dir"
 else
 	type=am
@@ -501,17 +502,11 @@ fi
 
 if test -n "$git_am_opt"; then
 	incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
-	if test -n "$interactive_rebase"
+	if test -n "$incompatible_opts"
 	then
-		if test -n "$incompatible_opts"
+		if test -n "$actually_interactive" || test "$do_merge"
 		then
-			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
-		fi
-	fi
-	if test -n "$do_merge"; then
-		if test -n "$incompatible_opts"
-		then
-			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
+			die "$(gettext "error: cannot combine am options ($incompatible_opts) with either interactive or merge options")"
 		fi
 	fi
 fi
@@ -660,7 +655,7 @@ require_clean_work_tree "rebase" "$(gettext "Please commit or stash them.")"
 # but this should be done only when upstream and onto are the same
 # and if this is not an interactive rebase.
 mb=$(git merge-base "$onto" "$orig_head")
-if test "$type" != interactive && test "$upstream" = "$onto" &&
+if test -z "$actually_interactive" && test "$upstream" = "$onto" &&
 	test "$mb" = "$onto" && test -z "$restrict_revision" &&
 	# linear history?
 	! (git rev-list --parents "$onto".."$orig_head" | sane_grep " .* ") > /dev/null
@@ -704,6 +699,22 @@ then
 	GIT_PAGER='' git diff --stat --summary "$mb" "$onto"
 fi
 
+if test -z "$actually_interactive"
+then
+	# If the $onto is a proper descendant of the tip of the branch, then
+	# we just fast-forwarded.
+	if test "$mb" = "$orig_head"
+	then
+		say "$(eval_gettext "Fast-forwarded \$branch_name to \$onto_name.")"
+		GIT_REFLOG_ACTION="$GIT_REFLOG_ACTION: checkout $onto_name" \
+			git checkout -q "$onto^0" || die "could not detach HEAD"
+		git update-ref ORIG_HEAD $orig_head
+		move_to_original_branch
+		finish_rebase
+		exit 0
+	fi
+fi
+
 test "$type" = interactive && run_specific_rebase
 
 # Detach HEAD and reset the tree
@@ -713,16 +724,6 @@ GIT_REFLOG_ACTION="$GIT_REFLOG_ACTION: checkout $onto_name" \
 	git checkout -q "$onto^0" || die "could not detach HEAD"
 git update-ref ORIG_HEAD $orig_head
 
-# If the $onto is a proper descendant of the tip of the branch, then
-# we just fast-forwarded.
-if test "$mb" = "$orig_head"
-then
-	say "$(eval_gettext "Fast-forwarded \$branch_name to \$onto_name.")"
-	move_to_original_branch
-	finish_rebase
-	exit 0
-fi
-
 if test -n "$rebase_root"
 then
 	revisions="$onto..$orig_head"
diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
index 0392e36d23..04d6c71899 100755
--- a/t/t3406-rebase-message.sh
+++ b/t/t3406-rebase-message.sh
@@ -17,14 +17,9 @@ test_expect_success 'setup' '
 	git tag start
 '
 
-cat >expect <<\EOF
-Already applied: 0001 A
-Already applied: 0002 B
-Committed: 0003 Z
-EOF
-
 test_expect_success 'rebase -m' '
 	git rebase -m master >report &&
+	>expect &&
 	sed -n -e "/^Already applied: /p" \
 		-e "/^Committed: /p" report >actual &&
 	test_cmp expect actual
diff --git a/t/t3420-rebase-autostash.sh b/t/t3420-rebase-autostash.sh
index e243700660..c465713672 100755
--- a/t/t3420-rebase-autostash.sh
+++ b/t/t3420-rebase-autostash.sh
@@ -53,41 +53,6 @@ create_expected_success_interactive () {
 	EOF
 }
 
-create_expected_success_merge () {
-	cat >expected <<-EOF
-	$(grep "^Created autostash: [0-9a-f][0-9a-f]*\$" actual)
-	HEAD is now at $(git rev-parse --short feature-branch) third commit
-	First, rewinding head to replay your work on top of it...
-	Merging unrelated-onto-branch with HEAD~1
-	Merging:
-	$(git rev-parse --short unrelated-onto-branch) unrelated commit
-	$(git rev-parse --short feature-branch^) second commit
-	found 1 common ancestor:
-	$(git rev-parse --short feature-branch~2) initial commit
-	[detached HEAD $(git rev-parse --short rebased-feature-branch~1)] second commit
-	 Author: A U Thor <author@example.com>
-	 Date: Thu Apr 7 15:14:13 2005 -0700
-	 2 files changed, 2 insertions(+)
-	 create mode 100644 file1
-	 create mode 100644 file2
-	Committed: 0001 second commit
-	Merging unrelated-onto-branch with HEAD~0
-	Merging:
-	$(git rev-parse --short rebased-feature-branch~1) second commit
-	$(git rev-parse --short feature-branch) third commit
-	found 1 common ancestor:
-	$(git rev-parse --short feature-branch~1) second commit
-	[detached HEAD $(git rev-parse --short rebased-feature-branch)] third commit
-	 Author: A U Thor <author@example.com>
-	 Date: Thu Apr 7 15:15:13 2005 -0700
-	 1 file changed, 1 insertion(+)
-	 create mode 100644 file3
-	Committed: 0002 third commit
-	All done.
-	Applied autostash.
-	EOF
-}
-
 create_expected_failure_am () {
 	cat >expected <<-EOF
 	$(grep "^Created autostash: [0-9a-f][0-9a-f]*\$" actual)
@@ -112,43 +77,6 @@ create_expected_failure_interactive () {
 	EOF
 }
 
-create_expected_failure_merge () {
-	cat >expected <<-EOF
-	$(grep "^Created autostash: [0-9a-f][0-9a-f]*\$" actual)
-	HEAD is now at $(git rev-parse --short feature-branch) third commit
-	First, rewinding head to replay your work on top of it...
-	Merging unrelated-onto-branch with HEAD~1
-	Merging:
-	$(git rev-parse --short unrelated-onto-branch) unrelated commit
-	$(git rev-parse --short feature-branch^) second commit
-	found 1 common ancestor:
-	$(git rev-parse --short feature-branch~2) initial commit
-	[detached HEAD $(git rev-parse --short rebased-feature-branch~1)] second commit
-	 Author: A U Thor <author@example.com>
-	 Date: Thu Apr 7 15:14:13 2005 -0700
-	 2 files changed, 2 insertions(+)
-	 create mode 100644 file1
-	 create mode 100644 file2
-	Committed: 0001 second commit
-	Merging unrelated-onto-branch with HEAD~0
-	Merging:
-	$(git rev-parse --short rebased-feature-branch~1) second commit
-	$(git rev-parse --short feature-branch) third commit
-	found 1 common ancestor:
-	$(git rev-parse --short feature-branch~1) second commit
-	[detached HEAD $(git rev-parse --short rebased-feature-branch)] third commit
-	 Author: A U Thor <author@example.com>
-	 Date: Thu Apr 7 15:15:13 2005 -0700
-	 1 file changed, 1 insertion(+)
-	 create mode 100644 file3
-	Committed: 0002 third commit
-	All done.
-	Applying autostash resulted in conflicts.
-	Your changes are safe in the stash.
-	You can run "git stash pop" or "git stash drop" at any time.
-	EOF
-}
-
 testrebase () {
 	type=$1
 	dotest=$2
@@ -177,6 +105,9 @@ testrebase () {
 	test_expect_success "rebase$type --autostash: check output" '
 		test_when_finished git branch -D rebased-feature-branch &&
 		suffix=${type#\ --} && suffix=${suffix:-am} &&
+		if test ${suffix} = "merge"; then
+			suffix=interactive
+		fi &&
 		create_expected_success_$suffix &&
 		test_i18ncmp expected actual
 	'
@@ -274,6 +205,9 @@ testrebase () {
 	test_expect_success "rebase$type: check output with conflicting stash" '
 		test_when_finished git branch -D rebased-feature-branch &&
 		suffix=${type#\ --} && suffix=${suffix:-am} &&
+		if test ${suffix} = "merge"; then
+			suffix=interactive
+		fi &&
 		create_expected_failure_$suffix &&
 		test_i18ncmp expected actual
 	'
diff --git a/t/t3421-rebase-topology-linear.sh b/t/t3421-rebase-topology-linear.sh
index 99b2aac921..911ef49f70 100755
--- a/t/t3421-rebase-topology-linear.sh
+++ b/t/t3421-rebase-topology-linear.sh
@@ -111,7 +111,7 @@ test_run_rebase () {
 	"
 }
 test_run_rebase success ''
-test_run_rebase failure -m
+test_run_rebase success -m
 test_run_rebase success -i
 test_run_rebase success -p
 
@@ -126,7 +126,7 @@ test_run_rebase () {
 	"
 }
 test_run_rebase success ''
-test_run_rebase failure -m
+test_run_rebase success -m
 test_run_rebase success -i
 test_run_rebase success -p
 
@@ -141,7 +141,7 @@ test_run_rebase () {
 	"
 }
 test_run_rebase success ''
-test_run_rebase failure -m
+test_run_rebase success -m
 test_run_rebase success -i
 test_run_rebase success -p
 
@@ -284,7 +284,7 @@ test_run_rebase () {
 	"
 }
 test_run_rebase success ''
-test_run_rebase failure -m
+test_run_rebase success -m
 test_run_rebase success -i
 test_run_rebase success -p
 
@@ -315,7 +315,7 @@ test_run_rebase () {
 	"
 }
 test_run_rebase success ''
-test_run_rebase failure -m
+test_run_rebase success -m
 test_run_rebase success -i
 test_run_rebase failure -p
 
diff --git a/t/t3425-rebase-topology-merges.sh b/t/t3425-rebase-topology-merges.sh
index 846f85c27e..cd505c0711 100755
--- a/t/t3425-rebase-topology-merges.sh
+++ b/t/t3425-rebase-topology-merges.sh
@@ -72,7 +72,7 @@ test_run_rebase () {
 }
 #TODO: make order consistent across all flavors of rebase
 test_run_rebase success 'e n o' ''
-test_run_rebase success 'e n o' -m
+test_run_rebase success 'n o e' -m
 test_run_rebase success 'n o e' -i
 
 test_run_rebase () {
@@ -89,7 +89,7 @@ test_run_rebase () {
 }
 #TODO: make order consistent across all flavors of rebase
 test_run_rebase success 'd e n o' ''
-test_run_rebase success 'd e n o' -m
+test_run_rebase success 'd n o e' -m
 test_run_rebase success 'd n o e' -i
 
 test_run_rebase () {
@@ -106,7 +106,7 @@ test_run_rebase () {
 }
 #TODO: make order consistent across all flavors of rebase
 test_run_rebase success 'd e n o' ''
-test_run_rebase success 'd e n o' -m
+test_run_rebase success 'd n o e' -m
 test_run_rebase success 'd n o e' -i
 
 test_expect_success "rebase -p is no-op in non-linear history" "
diff --git a/t/t5407-post-rewrite-hook.sh b/t/t5407-post-rewrite-hook.sh
index 9b2a274c71..145c61251d 100755
--- a/t/t5407-post-rewrite-hook.sh
+++ b/t/t5407-post-rewrite-hook.sh
@@ -120,6 +120,7 @@ test_expect_success 'git rebase -m --skip' '
 	git rebase --continue &&
 	echo rebase >expected.args &&
 	cat >expected.data <<-EOF &&
+	$(git rev-parse C) $(git rev-parse HEAD^)
 	$(git rev-parse D) $(git rev-parse HEAD)
 	EOF
 	verify_hook_input
-- 
2.18.0.rc1.12.g2996b9442d

[RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Elijah Newren]

am-based rebases suffer from an reduced ability to detect directory
renames upstream, which is fundamental to the fact that it throws away
information about the history: in particular, it dispenses with the
original commits involved by turning them into patches, and without the
knowledge of the original commits we cannot determine a proper merge base.

The am-based rebase will proceed by first trying git-apply, and, only if
it fails, will try to reconstruct a provisional merge base using
build_fake_ancestor().  This fake ancestor will ONLY contain the files
modified in the patch; without the full list of files in the real merge
base, renames of any missing files cannot be detected.  Directory rename
detection works by looking at individual file renames and deducing when a
full directory has been renamed.

Trying to modify build_fake_ancestor() to instead create a merge base that
includes common file information by looking for a commit that contained
all the same blobs as the pre-image of the patch would require a very
expensive search through history, may find the wrong commit, and outside
of rebase may not find any commit that matches.

But we had all the relevant information to start.  So, instead of
attempting to fix am which just doesn't have the relevant information,
instead note its strength and weaknesses, and change the default rebase
machinery to interactive since it does not suffer from the same problems.

Documentation updates:

  Several documentation updates were made as part of previous patches to
  note which sets of options were incompatible.  However, adding a new
  --am option allows us to make it clear which options imply this
  machinery and simplify the discussion of incompatible sets of options
  significantly.

testcase modification nodes:
  t3400: git-rebase--interactive.sh explicitly captures output and
         adds die "could not detach HEAD"; adjust to match
  t3401: fixes directory rename detection problems for rebase by default,
         though --am still fails on it
  t3404: testcase said it was for a non-interactive rebase, so add --am
  t3407: instead of having one test per rebase type, also add an extra one
         for the default rebase type.  That'll duplicate --merge for now,
         but will make switching default machinery again in the future
         clearer, if we ever choose to do so.  Also, previously there was
         a conspicuously missing test for all --quit combinations.
  t3420: command output varies by type of rebase and this test script
         has lots of helper functions providing the appropriate
         expectation.  Make sure we call the relevant one.
  t3425: Similar to t3407, add default rebase type to list of types to
         test.  See also comments about t3425 in the switchover of
         --merge to the interactive machinery.
  t5407: This suite has tests for different rebase types, so specify some
         are --am ones.
  t5520: interactive rebases capture and reprint informational message
         about conflicts to stdout rather than stderr.  Also, patches for
         interactive rebases are stored differently under .git than for
         am rebases.
  t9903: Add another testcase for am rebases.  Prompt for default rebases
         is now REBASE-m.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt      | 26 ++++++++++++--------------
 git-rebase.sh                     | 30 +++++++++++++++++++++---------
 t/t3400-rebase.sh                 |  7 ++++---
 t/t3401-rebase-and-am-rename.sh   | 18 +++++++++++++++++-
 t/t3404-rebase-interactive.sh     |  2 +-
 t/t3407-rebase-abort.sh           | 28 +++++++++++++++++++++++++++-
 t/t3420-rebase-autostash.sh       | 15 ++++++++++-----
 t/t3425-rebase-topology-merges.sh |  9 ++++++---
 t/t5407-post-rewrite-hook.sh      |  4 ++--
 t/t5520-pull.sh                   |  9 ++++++---
 t/t9903-bash-prompt.sh            | 12 +++++++++++-
 11 files changed, 117 insertions(+), 43 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 28d1658d7a..6cfb7479fd 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -240,13 +240,16 @@ leave out at most one of A and B, in which case it defaults to HEAD.
 	original branch. The index and working tree are also left
 	unchanged as a result.
 
+--am:
+	Use git-am internally to rebase.  This may run faster, but have
+	problems with rename detection on the upstream side.
+	Incompatible with the --interactive option, or anything that
+	uses the `--interactive` machinery.
+
 --keep-empty::
 	Keep the commits that do not change anything from its
 	parents in the result.
-+
-This uses the `--interactive` machinery internally, and as such,
-anything that is incompatible with --interactive is incompatible
-with this option.
+	This uses the `--interactive` machinery internally.
 
 --allow-empty-message::
 	By default, rebasing commits with an empty message will fail.
@@ -268,7 +271,7 @@ with this option.
 --merge::
 	Use merging strategies to rebase.  When the recursive (default) merge
 	strategy is used, this allows rebase to be aware of renames on the
-	upstream side.
+	upstream side.  This is the default.
 +
 Note that a rebase merge works by replaying each commit from the working
 branch on top of the <upstream> branch.  Because of this, when a merge
@@ -276,9 +279,7 @@ conflict happens, the side reported as 'ours' is the so-far rebased
 series, starting with <upstream>, and 'theirs' is the working branch.  In
 other words, the sides are swapped.
 +
-This uses the `--interactive` machinery internally, and as such,
-anything that is incompatible with --interactive is incompatible
-with this option.
+This uses the `--interactive` machinery internally.
 
 -s <strategy>::
 --strategy=<strategy>::
@@ -332,8 +333,7 @@ which makes little sense.
 	and after each change.  When fewer lines of surrounding
 	context exist they all must match.  By default no context is
 	ever ignored.
-	Incompatible with the --interactive option, or anything that
-	uses the `--interactive` machinery.
+	Implies --am.
 
 -f::
 --force-rebase::
@@ -365,15 +365,13 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 --whitespace=<option>::
 	These flag are passed to the 'git apply' program
 	(see linkgit:git-apply[1]) that applies the patch.
-	Incompatible with the --interactive option, or anything that
-	uses the `--interactive` machinery.
+	Implies --am.
 
 --committer-date-is-author-date::
 --ignore-date::
 	These flags are passed to 'git am' to easily change the dates
 	of the rebased commits (see linkgit:git-am[1]).
-	Incompatible with the --interactive option, or anything that
-	uses the `--interactive` machinery.
+	Implies --am.
 
 --signoff::
 	Add a Signed-off-by: trailer to all the rebased commits. Note
diff --git a/git-rebase.sh b/git-rebase.sh
index 5ac1dee30b..c8c3d0d05a 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -21,7 +21,8 @@ r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
-m,merge!           use merging strategies to rebase
+am                 use git-am to rebase
+m,merge!           use merging strategies to rebase; this is the default
 i,interactive!     let the user edit the list of commits to rebase
 x,exec=!           add exec lines after each commit of the editable list
 k,keep-empty	   preserve empty commits during rebase
@@ -69,6 +70,7 @@ unset restrict_revision
 cmd=
 strategy=
 strategy_opts=
+use_am=
 do_merge=
 merge_dir="$GIT_DIR"/rebase-merge
 apply_dir="$GIT_DIR"/rebase-apply
@@ -311,6 +313,9 @@ do
 	--no-fork-point)
 		fork_point=
 		;;
+	--am)
+		use_am=t
+		;;
 	--merge)
 		do_merge=t
 		;;
@@ -479,6 +484,7 @@ then
 	test -z "$interactive_rebase" && interactive_rebase=implied
 fi
 
+interactive_only_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
 actually_interactive=
 if test -n "$interactive_rebase"
 then
@@ -490,9 +496,18 @@ then
 	interactive_rebase=implied
 	type=interactive
 	state_dir="$merge_dir"
-else
+elif test -n "$use_am"
+then
 	type=am
 	state_dir="$apply_dir"
+elif test -n "$interactive_only_opts"
+then
+	type=am
+	state_dir="$apply_dir"
+else
+	interactive_rebase=implied
+	type=interactive
+	state_dir="$merge_dir"
 fi
 
 if test -t 2 && test -z "$GIT_QUIET"
@@ -500,14 +515,11 @@ then
 	git_format_patch_opt="$git_format_patch_opt --progress"
 fi
 
-if test -n "$git_am_opt"; then
-	incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
-	if test -n "$incompatible_opts"
+if test -n "$interactive_only_opts"
+then
+	if test -n "$actually_interactive" || test "$do_merge"
 	then
-		if test -n "$actually_interactive" || test "$do_merge"
-		then
-			die "$(gettext "error: cannot combine am options ($incompatible_opts) with either interactive or merge options")"
-		fi
+		die "$(gettext "error: cannot combine am options ($interactive_only_opts) with either interactive or merge options")"
 	fi
 fi
 
diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
index 72d9564747..03713572d6 100755
--- a/t/t3400-rebase.sh
+++ b/t/t3400-rebase.sh
@@ -136,8 +136,9 @@ test_expect_success 'setup: recover' '
 test_expect_success 'Show verbose error when HEAD could not be detached' '
 	>B &&
 	test_must_fail git rebase topic 2>output.err >output.out &&
-	test_i18ngrep "The following untracked working tree files would be overwritten by checkout:" output.err &&
-	test_i18ngrep B output.err
+	test_i18ngrep "The following untracked working tree files would be overwritten by checkout:" output.out &&
+	test_i18ngrep B output.out &&
+	test_i18ngrep "could not detach HEAD" output.err
 '
 rm -f B
 
@@ -289,7 +290,7 @@ test_expect_success 'rebase--am.sh and --show-current-patch' '
 		git tag two &&
 		test_must_fail git rebase --onto init HEAD^ &&
 		GIT_TRACE=1 git rebase --show-current-patch >/dev/null 2>stderr &&
-		grep "show.*$(git rev-parse two)" stderr
+		grep "show REBASE_HEAD" stderr
 	)
 '
 
diff --git a/t/t3401-rebase-and-am-rename.sh b/t/t3401-rebase-and-am-rename.sh
index ec87439bc6..6b54031201 100755
--- a/t/t3401-rebase-and-am-rename.sh
+++ b/t/t3401-rebase-and-am-rename.sh
@@ -54,7 +54,7 @@ test_expect_success 'rebase --interactive: directory rename detected' '
 	)
 '
 
-test_expect_failure 'rebase (am): directory rename detected' '
+test_expect_success 'rebase (default): directory rename detected' '
 	(
 		cd dir-rename &&
 
@@ -70,6 +70,22 @@ test_expect_failure 'rebase (am): directory rename detected' '
 	)
 '
 
+test_expect_failure 'rebase --am: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		git rebase --am A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
 test_expect_success 'rebase --merge: directory rename detected' '
 	(
 		cd dir-rename &&
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index c65826ddac..116060204e 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -975,7 +975,7 @@ test_expect_success C_LOCALE_OUTPUT 'rebase --edit-todo does not work on non-int
 	git reset --hard &&
 	git checkout conflict-branch &&
 	set_fake_editor &&
-	test_must_fail git rebase --onto HEAD~2 HEAD~ &&
+	test_must_fail git rebase --am --onto HEAD~2 HEAD~ &&
 	test_must_fail git rebase --edit-todo &&
 	git rebase --abort
 '
diff --git a/t/t3407-rebase-abort.sh b/t/t3407-rebase-abort.sh
index 910f218284..33084b17f5 100755
--- a/t/t3407-rebase-abort.sh
+++ b/t/t3407-rebase-abort.sh
@@ -96,14 +96,28 @@ testrebase() {
 	'
 }
 
-testrebase "" .git/rebase-apply
+testrebase " --am" .git/rebase-apply
+testrebase "" .git/rebase-merge
 testrebase " --merge" .git/rebase-merge
+testrebase " --interactive" .git/rebase-merge
 
 test_expect_success 'rebase --quit' '
 	cd "$work_dir" &&
 	# Clean up the state from the previous one
 	git reset --hard pre-rebase &&
 	test_must_fail git rebase master &&
+	test_path_is_dir .git/rebase-merge &&
+	head_before=$(git rev-parse HEAD) &&
+	git rebase --quit &&
+	test $(git rev-parse HEAD) = $head_before &&
+	test ! -d .git/rebase-merge
+'
+
+test_expect_success 'rebase --am --quit' '
+	cd "$work_dir" &&
+	# Clean up the state from the previous one
+	git reset --hard pre-rebase &&
+	test_must_fail git rebase --am master &&
 	test_path_is_dir .git/rebase-apply &&
 	head_before=$(git rev-parse HEAD) &&
 	git rebase --quit &&
@@ -123,4 +137,16 @@ test_expect_success 'rebase --merge --quit' '
 	test ! -d .git/rebase-merge
 '
 
+test_expect_success 'rebase --interactive --quit' '
+	cd "$work_dir" &&
+	# Clean up the state from the previous one
+	git reset --hard pre-rebase &&
+	test_must_fail git rebase --interactive master &&
+	test_path_is_dir .git/rebase-merge &&
+	head_before=$(git rev-parse HEAD) &&
+	git rebase --quit &&
+	test $(git rev-parse HEAD) = $head_before &&
+	test ! -d .git/rebase-merge
+'
+
 test_done
diff --git a/t/t3420-rebase-autostash.sh b/t/t3420-rebase-autostash.sh
index c465713672..9af0d27435 100755
--- a/t/t3420-rebase-autostash.sh
+++ b/t/t3420-rebase-autostash.sh
@@ -104,8 +104,10 @@ testrebase () {
 
 	test_expect_success "rebase$type --autostash: check output" '
 		test_when_finished git branch -D rebased-feature-branch &&
-		suffix=${type#\ --} && suffix=${suffix:-am} &&
-		if test ${suffix} = "merge"; then
+		suffix=${type#\ --} &&
+		if test ${suffix} = "am"; then
+			suffix=am
+		else
 			suffix=interactive
 		fi &&
 		create_expected_success_$suffix &&
@@ -204,8 +206,10 @@ testrebase () {
 
 	test_expect_success "rebase$type: check output with conflicting stash" '
 		test_when_finished git branch -D rebased-feature-branch &&
-		suffix=${type#\ --} && suffix=${suffix:-am} &&
-		if test ${suffix} = "merge"; then
+		suffix=${type#\ --} &&
+		if test ${suffix} = "am"; then
+			suffix=am
+		else
 			suffix=interactive
 		fi &&
 		create_expected_failure_$suffix &&
@@ -235,7 +239,8 @@ test_expect_success "rebase: noop rebase" '
 	git checkout feature-branch
 '
 
-testrebase "" .git/rebase-apply
+testrebase " --am" .git/rebase-apply
+testrebase "" .git/rebase-merge
 testrebase " --merge" .git/rebase-merge
 testrebase " --interactive" .git/rebase-merge
 
diff --git a/t/t3425-rebase-topology-merges.sh b/t/t3425-rebase-topology-merges.sh
index cd505c0711..8fd14c86d3 100755
--- a/t/t3425-rebase-topology-merges.sh
+++ b/t/t3425-rebase-topology-merges.sh
@@ -71,7 +71,8 @@ test_run_rebase () {
 	"
 }
 #TODO: make order consistent across all flavors of rebase
-test_run_rebase success 'e n o' ''
+test_run_rebase success 'e n o' --am
+test_run_rebase success 'n o e' ''
 test_run_rebase success 'n o e' -m
 test_run_rebase success 'n o e' -i
 
@@ -88,7 +89,8 @@ test_run_rebase () {
 	"
 }
 #TODO: make order consistent across all flavors of rebase
-test_run_rebase success 'd e n o' ''
+test_run_rebase success 'd e n o' --am
+test_run_rebase success 'd n o e' ''
 test_run_rebase success 'd n o e' -m
 test_run_rebase success 'd n o e' -i
 
@@ -105,7 +107,8 @@ test_run_rebase () {
 	"
 }
 #TODO: make order consistent across all flavors of rebase
-test_run_rebase success 'd e n o' ''
+test_run_rebase success 'd e n o' --am
+test_run_rebase success 'd n o e' ''
 test_run_rebase success 'd n o e' -m
 test_run_rebase success 'd n o e' -i
 
diff --git a/t/t5407-post-rewrite-hook.sh b/t/t5407-post-rewrite-hook.sh
index 145c61251d..f5b5f4f04c 100755
--- a/t/t5407-post-rewrite-hook.sh
+++ b/t/t5407-post-rewrite-hook.sh
@@ -71,7 +71,7 @@ test_expect_success 'git rebase' '
 test_expect_success 'git rebase --skip' '
 	git reset --hard D &&
 	clear_hook_input &&
-	test_must_fail git rebase --onto A B &&
+	test_must_fail git rebase --am --onto A B &&
 	test_must_fail git rebase --skip &&
 	echo D > foo &&
 	git add foo &&
@@ -86,7 +86,7 @@ test_expect_success 'git rebase --skip' '
 test_expect_success 'git rebase --skip the last one' '
 	git reset --hard F &&
 	clear_hook_input &&
-	test_must_fail git rebase --onto D A &&
+	test_must_fail git rebase --am --onto D A &&
 	git rebase --skip &&
 	echo rebase >expected.args &&
 	cat >expected.data <<-EOF &&
diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
index 59c4b778d3..d7c915a209 100755
--- a/t/t5520-pull.sh
+++ b/t/t5520-pull.sh
@@ -305,7 +305,7 @@ test_expect_success '--rebase with conflicts shows advice' '
 	test_tick &&
 	git commit -m "Create conflict" seq.txt &&
 	test_must_fail git pull --rebase . seq 2>err >out &&
-	test_i18ngrep "Resolve all conflicts manually" out
+	test_i18ngrep "Resolve all conflicts manually" err
 '
 
 test_expect_success 'failed --rebase shows advice' '
@@ -319,7 +319,7 @@ test_expect_success 'failed --rebase shows advice' '
 	git checkout -f -b fails-to-rebase HEAD^ &&
 	test_commit v2-without-cr file "2" file2-lf &&
 	test_must_fail git pull --rebase . diverging 2>err >out &&
-	test_i18ngrep "Resolve all conflicts manually" out
+	test_i18ngrep "Resolve all conflicts manually" err
 '
 
 test_expect_success '--rebase fails with multiple branches' '
@@ -671,7 +671,10 @@ test_expect_success 'setup for avoiding reapplying old patches' '
 test_expect_success 'git pull --rebase does not reapply old patches' '
 	(cd dst &&
 	 test_must_fail git pull --rebase &&
-	 test 1 = $(find .git/rebase-apply -name "000*" | wc -l)
+	 cat .git/rebase-merge/done .git/rebase-merge/git-rebase-todo >work &&
+	 grep -v -e \# -e ^$ work >patches &&
+	 test_line_count = 1 patches &&
+	 rm -f work patches
 	)
 '
 
diff --git a/t/t9903-bash-prompt.sh b/t/t9903-bash-prompt.sh
index c3b89ae783..29a83ae8cf 100755
--- a/t/t9903-bash-prompt.sh
+++ b/t/t9903-bash-prompt.sh
@@ -192,10 +192,20 @@ test_expect_success 'prompt - rebase merge' '
 	test_cmp expected "$actual"
 '
 
-test_expect_success 'prompt - rebase' '
+test_expect_success 'prompt - rebase am' '
 	printf " (b2|REBASE 1/3)" >expected &&
 	git checkout b2 &&
 	test_when_finished "git checkout master" &&
+	test_must_fail git rebase --am b1 b2 &&
+	test_when_finished "git rebase --abort" &&
+	__git_ps1 >"$actual" &&
+	test_cmp expected "$actual"
+'
+
+test_expect_success 'prompt - rebase' '
+	printf " (b2|REBASE-m 1/3)" >expected &&
+	git checkout b2 &&
+	test_when_finished "git checkout master" &&
 	test_must_fail git rebase b1 b2 &&
 	test_when_finished "git rebase --abort" &&
 	__git_ps1 >"$actual" &&
-- 
2.18.0.rc1.12.g2996b9442d

Re: [RFC PATCH 1/3] git-rebase, sequencer: add a --quiet option for the interactive machinery

[Johannes Schindelin]

Hi Elijah,

On Thu, 7 Jun 2018, Elijah Newren wrote:

> While 'quiet' and 'interactive' may sound like antonyms, the interactive
> machinery actually has logic that implements several
> interactive_rebase=implied cases (--exec, --keep-empty, --rebase-merges)
> which won't pop up an editor.  Further, we want to make the interactive
> machinery also take over for git-rebase--merge and become the default
> merge strategy, so it makes sense for these other cases to have a quiet
> option.
> 
> git-rebase--interactive was already somewhat quieter than
> git-rebase--merge and git-rebase--am, possibly because cherry-pick has
> just traditionally been quieter.  As such, we only drop a few
> informational messages -- "Rebasing (n/m)" and "Succesfully rebased..."

Makes sense. As long as it is coordinated with Alban and Pratik, as both
of their GSoC projects are affected by this.

In particular Pratik's project, I think, would actually *benefit* from
your work, as it might even make it possible to turn all modes but
--preserve-merges into pure builtin code, which would be awesome.

> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 06a7b79307..1f2401f702 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -139,8 +139,12 @@ mark_action_done () {
>  	if test "$last_count" != "$new_count"
>  	then
>  		last_count=$new_count
> -		eval_gettext "Rebasing (\$new_count/\$total)"; printf "\r"
> -		test -z "$verbose" || echo
> +		if test -z "$GIT_QUIET"
> +		then
> +			eval_gettext "Rebasing (\$new_count/\$total)";
> +			printf "\r"
> +			test -z "$verbose" || echo
> +		fi
>  	fi
>  }
>  
> @@ -713,6 +717,7 @@ Commit or stash your changes, and then run
>  		"$hook" rebase < "$rewritten_list"
>  		true # we don't care if this hook failed
>  	fi &&
> +		test -z "$GIT_QUIET" &&
>  		warn "$(eval_gettext "Successfully rebased and updated \$head_name.")"

In general, I tried the statements to return success at all times. That
means that

		test -n "$GIT_QUIET" ||

would be better in this case.

> diff --git a/git-rebase.sh b/git-rebase.sh
> index 7d1612b31b..b639c0d4fe 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -136,7 +136,7 @@ write_basic_state () {
>  	echo "$head_name" > "$state_dir"/head-name &&
>  	echo "$onto" > "$state_dir"/onto &&
>  	echo "$orig_head" > "$state_dir"/orig-head &&
> -	echo "$GIT_QUIET" > "$state_dir"/quiet &&
> +	test t = "$GIT_QUIET" && : > "$state_dir"/quiet

Maybe it would be better to `echo t` into that file? That way, scripts
that used the value in that file would continue to work. (But maybe there
were no scripts that could use it, as only the interactive rebase allows
scripting, and it did not handle that flag before?)

The rest looks obviously good.

Thank you!
Dscho

Re: [RFC PATCH 2/3] rebase: Implement --merge via git-rebase--interactive

[Johannes Schindelin]

Hi Elijah,

On Thu, 7 Jun 2018, Elijah Newren wrote:

> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 451252c173..28d1658d7a 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -275,6 +275,10 @@ branch on top of the <upstream> branch.  Because of this, when a merge
>  conflict happens, the side reported as 'ours' is the so-far rebased
>  series, starting with <upstream>, and 'theirs' is the working branch.  In
>  other words, the sides are swapped.
> ++
> +This uses the `--interactive` machinery internally, and as such,
> +anything that is incompatible with --interactive is incompatible
> +with this option.

I am not sure I like this change, as it describes an implementation detail
that users should neither know, nor care about, nor rely on.

> @@ -328,8 +332,8 @@ which makes little sense.
>  	and after each change.  When fewer lines of surrounding
>  	context exist they all must match.  By default no context is
>  	ever ignored.
> -	Incompatible with the --merge and --interactive options, or
> -	anything that implies those options or their machinery.
> +	Incompatible with the --interactive option, or anything that
> +	uses the `--interactive` machinery.
>  
>  -f::
>  --force-rebase::
> @@ -361,15 +365,15 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
>  --whitespace=<option>::
>  	These flag are passed to the 'git apply' program
>  	(see linkgit:git-apply[1]) that applies the patch.
> -	Incompatible with the --merge and --interactive options, or
> -	anything that implies those options or their machinery.
> +	Incompatible with the --interactive option, or anything that
> +	uses the `--interactive` machinery.
>  
>  --committer-date-is-author-date::
>  --ignore-date::
>  	These flags are passed to 'git am' to easily change the dates
>  	of the rebased commits (see linkgit:git-am[1]).
> -	Incompatible with the --merge and --interactive options, or
> -	anything that implies those options or their machinery.
> +	Incompatible with the --interactive option, or anything that
> +	uses the `--interactive` machinery.
>  
>  --signoff::
>  	Add a Signed-off-by: trailer to all the rebased commits. Note

For above-mentioned reason, I'd suggest dropping these three hunks.

> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 1f2401f702..dcc4a26a78 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -885,7 +885,10 @@ init_basic_state () {
>  	mkdir -p "$state_dir" || die "$(eval_gettext "Could not create temporary \$state_dir")"
>  	rm -f "$(git rev-parse --git-path REBASE_HEAD)"
>  
> -	: > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
> +	if test -n "$actually_interactive"
> +	then
> +		: > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
> +	fi

Do we really care at this stage? IOW what breaks if we still write that
file, even in --merge mode?

> diff --git a/git-rebase.sh b/git-rebase.sh
> index b639c0d4fe..5ac1dee30b 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -239,12 +239,10 @@ then
>  	state_dir="$apply_dir"
>  elif test -d "$merge_dir"
>  then
> +	type=interactive
>  	if test -f "$merge_dir"/interactive
>  	then
> -		type=interactive
>  		interactive_rebase=explicit
> -	else
> -		type=merge
>  	fi
>  	state_dir="$merge_dir"
>  fi

This makes me think even more that we don't care...

> @@ -481,13 +479,16 @@ then
>  	test -z "$interactive_rebase" && interactive_rebase=implied
>  fi
>  
> +actually_interactive=
>  if test -n "$interactive_rebase"
>  then
>  	type=interactive
> +	actually_interactive=t
>  	state_dir="$merge_dir"
>  elif test -n "$do_merge"
>  then
> -	type=merge
> +	interactive_rebase=implied
> +	type=interactive
>  	state_dir="$merge_dir"
>  else
>  	type=am
> @@ -501,17 +502,11 @@ fi
>  
>  if test -n "$git_am_opt"; then
>  	incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
> -	if test -n "$interactive_rebase"
> +	if test -n "$incompatible_opts"

Did you not mean to turn this into a test for actually_interactve ||
do_merge?

>  	then
> -		if test -n "$incompatible_opts"
> +		if test -n "$actually_interactive" || test "$do_merge"

This could now be combined with the previous if (and the `-n` could be
added to the latter test):

	if test -n "$actually_interactive" -o -n "$do_merge" &&
		test -n "$incompatible_opts"

The indentation would change, but this hunk is already confusing to read,
anyway, so...

>  		then
> -			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
> -		fi
> -	fi
> -	if test -n "$do_merge"; then
> -		if test -n "$incompatible_opts"
> -		then
> -			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
> +			die "$(gettext "error: cannot combine am options ($incompatible_opts) with either interactive or merge options")"
>  		fi
>  	fi
>  fi
> @@ -660,7 +655,7 @@ require_clean_work_tree "rebase" "$(gettext "Please commit or stash them.")"
>  # but this should be done only when upstream and onto are the same
>  # and if this is not an interactive rebase.
>  mb=$(git merge-base "$onto" "$orig_head")
> -if test "$type" != interactive && test "$upstream" = "$onto" &&
> +if test -z "$actually_interactive" && test "$upstream" = "$onto" &&
>  	test "$mb" = "$onto" && test -z "$restrict_revision" &&
>  	# linear history?
>  	! (git rev-list --parents "$onto".."$orig_head" | sane_grep " .* ") > /dev/null
> @@ -704,6 +699,22 @@ then
>  	GIT_PAGER='' git diff --stat --summary "$mb" "$onto"
>  fi
>  
> +if test -z "$actually_interactive"
> +then
> +	# If the $onto is a proper descendant of the tip of the branch, then
> +	# we just fast-forwarded.
> +	if test "$mb" = "$orig_head"
> +	then

Likewise, this would be easier to read as

	if test -z "$actually_interactive" && test "$mb" = "$orig_head"

Also: how certain are we that "$mb" does not start with a dash? We may
have to use the `test a"$mb" = a"$orig_head"` trick here... But I guess
this is moved code, so if it is buggy, it was buggy before.

> +		say "$(eval_gettext "Fast-forwarded \$branch_name to \$onto_name.")"
> +		GIT_REFLOG_ACTION="$GIT_REFLOG_ACTION: checkout $onto_name" \
> +			git checkout -q "$onto^0" || die "could not detach HEAD"
> +		git update-ref ORIG_HEAD $orig_head
> +		move_to_original_branch
> +		finish_rebase
> +		exit 0
> +	fi
> +fi
> +
>  test "$type" = interactive && run_specific_rebase
>  
>  # Detach HEAD and reset the tree
> @@ -713,16 +724,6 @@ GIT_REFLOG_ACTION="$GIT_REFLOG_ACTION: checkout $onto_name" \
>  	git checkout -q "$onto^0" || die "could not detach HEAD"
>  git update-ref ORIG_HEAD $orig_head
>  
> -# If the $onto is a proper descendant of the tip of the branch, then
> -# we just fast-forwarded.
> -if test "$mb" = "$orig_head"
> -then
> -	say "$(eval_gettext "Fast-forwarded \$branch_name to \$onto_name.")"
> -	move_to_original_branch
> -	finish_rebase
> -	exit 0
> -fi
> -
>  if test -n "$rebase_root"
>  then
>  	revisions="$onto..$orig_head"
> diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
> index 0392e36d23..04d6c71899 100755
> --- a/t/t3406-rebase-message.sh
> +++ b/t/t3406-rebase-message.sh
> @@ -17,14 +17,9 @@ test_expect_success 'setup' '
>  	git tag start
>  '
>  
> -cat >expect <<\EOF
> -Already applied: 0001 A
> -Already applied: 0002 B
> -Committed: 0003 Z
> -EOF
> -
>  test_expect_success 'rebase -m' '
>  	git rebase -m master >report &&
> +	>expect &&
>  	sed -n -e "/^Already applied: /p" \
>  		-e "/^Committed: /p" report >actual &&
>  	test_cmp expect actual

This might be called a regression... I don't know any user of `git rebase
-m`, but I guess if I was one, I would like to keep seeing those messages.

It *should* be relatively easy to tell the sequencer.c to issue these
messages, I think. But it would be more work.

> diff --git a/t/t3421-rebase-topology-linear.sh b/t/t3421-rebase-topology-linear.sh
> index 99b2aac921..911ef49f70 100755
> --- a/t/t3421-rebase-topology-linear.sh
> +++ b/t/t3421-rebase-topology-linear.sh
> @@ -111,7 +111,7 @@ test_run_rebase () {
>  	"
>  }
>  test_run_rebase success ''
> -test_run_rebase failure -m
> +test_run_rebase success -m
>  test_run_rebase success -i
>  test_run_rebase success -p
>  
> @@ -126,7 +126,7 @@ test_run_rebase () {
>  	"
>  }
>  test_run_rebase success ''
> -test_run_rebase failure -m
> +test_run_rebase success -m
>  test_run_rebase success -i
>  test_run_rebase success -p
>  
> @@ -141,7 +141,7 @@ test_run_rebase () {
>  	"
>  }
>  test_run_rebase success ''
> -test_run_rebase failure -m
> +test_run_rebase success -m
>  test_run_rebase success -i
>  test_run_rebase success -p
>  
> @@ -284,7 +284,7 @@ test_run_rebase () {
>  	"
>  }
>  test_run_rebase success ''
> -test_run_rebase failure -m
> +test_run_rebase success -m
>  test_run_rebase success -i
>  test_run_rebase success -p
>  
> @@ -315,7 +315,7 @@ test_run_rebase () {
>  	"
>  }
>  test_run_rebase success ''
> -test_run_rebase failure -m
> +test_run_rebase success -m
>  test_run_rebase success -i
>  test_run_rebase failure -p

Neat!

> diff --git a/t/t3425-rebase-topology-merges.sh b/t/t3425-rebase-topology-merges.sh
> index 846f85c27e..cd505c0711 100755
> --- a/t/t3425-rebase-topology-merges.sh
> +++ b/t/t3425-rebase-topology-merges.sh
> @@ -72,7 +72,7 @@ test_run_rebase () {
>  }
>  #TODO: make order consistent across all flavors of rebase
>  test_run_rebase success 'e n o' ''
> -test_run_rebase success 'e n o' -m
> +test_run_rebase success 'n o e' -m
>  test_run_rebase success 'n o e' -i
>  
>  test_run_rebase () {
> @@ -89,7 +89,7 @@ test_run_rebase () {
>  }
>  #TODO: make order consistent across all flavors of rebase
>  test_run_rebase success 'd e n o' ''
> -test_run_rebase success 'd e n o' -m
> +test_run_rebase success 'd n o e' -m
>  test_run_rebase success 'd n o e' -i
>  
>  test_run_rebase () {
> @@ -106,7 +106,7 @@ test_run_rebase () {
>  }
>  #TODO: make order consistent across all flavors of rebase
>  test_run_rebase success 'd e n o' ''
> -test_run_rebase success 'd e n o' -m
> +test_run_rebase success 'd n o e' -m
>  test_run_rebase success 'd n o e' -i

Nice!

>  test_expect_success "rebase -p is no-op in non-linear history" "
> diff --git a/t/t5407-post-rewrite-hook.sh b/t/t5407-post-rewrite-hook.sh
> index 9b2a274c71..145c61251d 100755
> --- a/t/t5407-post-rewrite-hook.sh
> +++ b/t/t5407-post-rewrite-hook.sh
> @@ -120,6 +120,7 @@ test_expect_success 'git rebase -m --skip' '
>  	git rebase --continue &&
>  	echo rebase >expected.args &&
>  	cat >expected.data <<-EOF &&
> +	$(git rev-parse C) $(git rev-parse HEAD^)
>  	$(git rev-parse D) $(git rev-parse HEAD)
>  	EOF
>  	verify_hook_input

This is a bit sad, because it seems to suggest that we now do more
unnecessary work here, or is my reading incorrect?

Thanks,
Dscho

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Johannes Schindelin]

Hi Elijah,

On Thu, 7 Jun 2018, Elijah Newren wrote:

> am-based rebases suffer from an reduced ability to detect directory
> renames upstream, which is fundamental to the fact that it throws away
> information about the history: in particular, it dispenses with the
> original commits involved by turning them into patches, and without the
> knowledge of the original commits we cannot determine a proper merge base.
> 
> The am-based rebase will proceed by first trying git-apply, and, only if
> it fails, will try to reconstruct a provisional merge base using
> build_fake_ancestor().  This fake ancestor will ONLY contain the files
> modified in the patch; without the full list of files in the real merge
> base, renames of any missing files cannot be detected.  Directory rename
> detection works by looking at individual file renames and deducing when a
> full directory has been renamed.
> 
> Trying to modify build_fake_ancestor() to instead create a merge base that
> includes common file information by looking for a commit that contained
> all the same blobs as the pre-image of the patch would require a very
> expensive search through history, may find the wrong commit, and outside
> of rebase may not find any commit that matches.
> 
> But we had all the relevant information to start.  So, instead of
> attempting to fix am which just doesn't have the relevant information,
> instead note its strength and weaknesses, and change the default rebase
> machinery to interactive since it does not suffer from the same problems.

I'll let Eric comment on the grammar, and I'll comment on the idea behind
this commit instead.

IMHO `git rebase -i` is still too slow to be a true replacement for `git
rebase --am` for the cases where it serves the user well. Maybe we should
work on making `rebase -i` faster, first?

I imagine, for example, that it might make *tons* of sense to avoid
writing out the index and worktree files all the time. That was necessary
in the shell script version because if the ridiculous limitations we
subjected ourselves to, such as: no object-oriented state worth
mentioning, only string-based processing, etc. But we could now start to
do everything in memory (*maybe* write out the new blob/tree/commit
objects immediately, but maybe not) until the time when we either have
succeeded in the rebase, or when there was a problem and we have to exit
with an error. And only then write the files and the index.

In any case, I think that the rather noticeable change of the default
would probably necessitate a deprecation phase.

Ciao,
Dscho

Re: [RFC PATCH 1/3] git-rebase, sequencer: add a --quiet option for the interactive machinery

[Elijah Newren]

Hi Dscho,

On Sat, Jun 9, 2018 at 2:45 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> On Thu, 7 Jun 2018, Elijah Newren wrote:
>
>> While 'quiet' and 'interactive' may sound like antonyms, the interactive
>> machinery actually has logic that implements several
>> interactive_rebase=implied cases (--exec, --keep-empty, --rebase-merges)
>> which won't pop up an editor.  Further, we want to make the interactive
>> machinery also take over for git-rebase--merge and become the default
>> merge strategy, so it makes sense for these other cases to have a quiet
>> option.
>>
>> git-rebase--interactive was already somewhat quieter than
>> git-rebase--merge and git-rebase--am, possibly because cherry-pick has
>> just traditionally been quieter.  As such, we only drop a few
>> informational messages -- "Rebasing (n/m)" and "Succesfully rebased..."
>
> Makes sense. As long as it is coordinated with Alban and Pratik, as both
> of their GSoC projects are affected by this.

I had Alban cc'ed, and had looked briefly at the GSoC projects but
somehow missed that Pratik was also working in the area.  Adding him
to cc.

> In particular Pratik's project, I think, would actually *benefit* from
> your work, as it might even make it possible to turn all modes but
> --preserve-merges into pure builtin code, which would be awesome.

:-)

>> @@ -713,6 +717,7 @@ Commit or stash your changes, and then run
>>               "$hook" rebase < "$rewritten_list"
>>               true # we don't care if this hook failed
>>       fi &&
>> +             test -z "$GIT_QUIET" &&
>>               warn "$(eval_gettext "Successfully rebased and updated \$head_name.")"
>
> In general, I tried the statements to return success at all times. That
> means that
>
>                 test -n "$GIT_QUIET" ||
>
> would be better in this case.

Good point, I'll switch it over.

>> diff --git a/git-rebase.sh b/git-rebase.sh
>> index 7d1612b31b..b639c0d4fe 100755
>> --- a/git-rebase.sh
>> +++ b/git-rebase.sh
>> @@ -136,7 +136,7 @@ write_basic_state () {
>>       echo "$head_name" > "$state_dir"/head-name &&
>>       echo "$onto" > "$state_dir"/onto &&
>>       echo "$orig_head" > "$state_dir"/orig-head &&
>> -     echo "$GIT_QUIET" > "$state_dir"/quiet &&
>> +     test t = "$GIT_QUIET" && : > "$state_dir"/quiet
>
> Maybe it would be better to `echo t` into that file? That way, scripts
> that used the value in that file would continue to work. (But maybe there
> were no scripts that could use it, as only the interactive rebase allows
> scripting, and it did not handle that flag before?)

Right, I don't think we had users before, and I'd rather make the code
a little more self-consistent.  In particular, since
$state_dir/verbose work based off the presence of the file rather than
the contents, I'd rather we just did the same for $state_dir/quiet.

However, there is one bug here; in order to make it like verbose, I
need to also make the following change:

diff --git a/git-rebase.sh b/git-rebase.sh
index c8c3d0d05a..8f0c7a4738 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -119,7 +119,7 @@ read_basic_state () {
        else
                orig_head=$(cat "$state_dir"/head)
        fi &&
-       GIT_QUIET=$(cat "$state_dir"/quiet) &&
+       test -f "$state_dir"/quiet && GIT_QUIET=t
        test -f "$state_dir"/verbose && verbose=t
        test -f "$state_dir"/strategy && strategy="$(cat "$state_dir"/strategy)"
        test -f "$state_dir"/strategy_opts &&


> The rest looks obviously good.


Thanks!
Elijah

Re: [RFC PATCH 2/3] rebase: Implement --merge via git-rebase--interactive

[Elijah Newren]

Hi Dscho,

On Sat, Jun 9, 2018 at 3:04 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> On Thu, 7 Jun 2018, Elijah Newren wrote:
>
>> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
>> index 451252c173..28d1658d7a 100644
>> --- a/Documentation/git-rebase.txt
>> +++ b/Documentation/git-rebase.txt
>> @@ -275,6 +275,10 @@ branch on top of the <upstream> branch.  Because of this, when a merge
>>  conflict happens, the side reported as 'ours' is the so-far rebased
>>  series, starting with <upstream>, and 'theirs' is the working branch.  In
>>  other words, the sides are swapped.
>> ++
>> +This uses the `--interactive` machinery internally, and as such,
>> +anything that is incompatible with --interactive is incompatible
>> +with this option.
>
> I am not sure I like this change, as it describes an implementation detail
> that users should neither know, nor care about, nor rely on.

Let me back up for just a second to see if I can point out the real
problem I'm trying to address here, which you may have a better
solution for.  What should happen if a user runs
   git rebase --merge --whitespace=fix
?

git currently silently ignores the --whitepsace=fix argument, leaving
the whitespace damage present at the end of the rebase.  Same goes for
--interactive combined with any am-specific options  (Fix proposed at
https://public-inbox.org/git/20180607050654.19663-2-newren@gmail.com/).
This silent ignoring of some options depending on which other options
were specified has caused me problems in the past.

So, while I totally agree with you that users shouldn't need to know
implementation details, they do need to know how to use commands and
which options go well together and which are mutually incompatible.
Do you have any suggestions on alternate wording that would convey the
sets of mutually incompatible options without talking about
implementation details?  Would you rather only address that in the
code and not mention it in the documentation?

See also https://public-inbox.org/git/20180607050654.19663-1-newren@gmail.com/,
which proposes testcases for these incompatibility sets.


>> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
>> index 1f2401f702..dcc4a26a78 100644
>> --- a/git-rebase--interactive.sh
>> +++ b/git-rebase--interactive.sh
>> @@ -885,7 +885,10 @@ init_basic_state () {
>>       mkdir -p "$state_dir" || die "$(eval_gettext "Could not create temporary \$state_dir")"
>>       rm -f "$(git rev-parse --git-path REBASE_HEAD)"
>>
>> -     : > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
>> +     if test -n "$actually_interactive"
>> +     then
>> +             : > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
>> +     fi
>
> Do we really care at this stage? IOW what breaks if we still write that
> file, even in --merge mode?

Two things will change if we do that:
  * bash prompt will be different for those using git-prompt.sh
(REBASE-m vs. REBASE-i).
  * git-status output is no longer the same ('rebase in progress' vs.
'interactive rebase in progress...last command done:...pick 0dea123 my
wonderful commit')

I don't think the prompt is a big deal.  The status output might not
be either, but showing the "last command done" may be weird to someone
that never saw or edited a list of commands.  (Then again, that same
argument could be made for users of --exec, --rebase-merges, or
--keep-empty without an explicit --interactive)

Opinions on whether these two difference matter?  If others don't
think these differences are significant, I'm happy to update any
necessary testcases and just unconditionally create the
$state_dir/interactive file.


>> @@ -501,17 +502,11 @@ fi
>>
>>  if test -n "$git_am_opt"; then
>>       incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
>> -     if test -n "$interactive_rebase"
>> +     if test -n "$incompatible_opts"
>
> Did you not mean to turn this into a test for actually_interactve ||
> do_merge?
>
>>       then
>> -             if test -n "$incompatible_opts"
>> +             if test -n "$actually_interactive" || test "$do_merge"
>
> This could now be combined with the previous if (and the `-n` could be
> added to the latter test):
>
>         if test -n "$actually_interactive" -o -n "$do_merge" &&
>                 test -n "$incompatible_opts"
>
> The indentation would change, but this hunk is already confusing to read,
> anyway, so...

I'm happy to switch the order of the nesting as you suggest and agree
that it would make it easier to read.  However, I hesitate to combine
the two if-lines.  When I read your combined suggested line, I parsed
it as follows (using invalid pseduo-syntax just to convey grouping):

  ( -n "$actually_interactive ) || ( -n "$do_merge" && -n "$incompatible_opts" )

Granted, I parsed it wrong, putting the parentheses in the wrong
places, and bash parses it as you intended.  While you may have
precedence and left- vs. right- associativity rules down pat, I
clearly didn't.  If we combine the lines, I'll probably mis-read them
again when I see them in a year or more.

>> @@ -704,6 +699,22 @@ then
>>       GIT_PAGER='' git diff --stat --summary "$mb" "$onto"
>>  fi
>>
>> +if test -z "$actually_interactive"
>> +then
>> +     # If the $onto is a proper descendant of the tip of the branch, then
>> +     # we just fast-forwarded.
>> +     if test "$mb" = "$orig_head"
>> +     then
>
> Likewise, this would be easier to read as
>
>         if test -z "$actually_interactive" && test "$mb" = "$orig_head"

Good point, I'll fix that up.

> Also: how certain are we that "$mb" does not start with a dash? We may
> have to use the `test a"$mb" = a"$orig_head"` trick here... But I guess
> this is moved code, so if it is buggy, it was buggy before.

From earlier in the file,
mb=$(git merge-base ...)

So, unless we're expecting the output of git-merge-base to change in
the future to include leading dashes, we shouldn't hit any issues.  I
can make the change you suggest if you're worried, though.

>> diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
>> index 0392e36d23..04d6c71899 100755
>> --- a/t/t3406-rebase-message.sh
>> +++ b/t/t3406-rebase-message.sh
>> @@ -17,14 +17,9 @@ test_expect_success 'setup' '
>>       git tag start
>>  '
>>
>> -cat >expect <<\EOF
>> -Already applied: 0001 A
>> -Already applied: 0002 B
>> -Committed: 0003 Z
>> -EOF
>> -
>>  test_expect_success 'rebase -m' '
>>       git rebase -m master >report &&
>> +     >expect &&
>>       sed -n -e "/^Already applied: /p" \
>>               -e "/^Committed: /p" report >actual &&
>>       test_cmp expect actual
>
> This might be called a regression... I don't know any user of `git rebase
> -m`, but I guess if I was one, I would like to keep seeing those messages.
>
> It *should* be relatively easy to tell the sequencer.c to issue these
> messages, I think. But it would be more work.

You may well be right.  Here's where my thinking came from...

am-based, interactive-based, and merge-based rebases have lots of
little ways in which they have differed, this being just one of them.
It was sometimes hard making a judgement call when writing this series
about whether any given deviation was a difference I wanted to smooth
over or a difference I wanted to perpetuate between various flags.
Further, if it was a difference I wanted to smooth over, then I had to
decide which of the current behaviors was 'correct'.

In this particular case, I basically went off perceived usage.
am-based rebases have lots of special flags relevant to it only
(--whitespace, -C, etc.) and is the default, so it clearly has lots of
users.  interactive-based rebases are pretty prominent too, and have
very specific special capabilities the other modes don't.  In
contrast, merge-based rebases can't do a single thing that interactive
can't; and unless you're using a special merge strategy, for the last
10 years merge-based rebases haven't been able to do anything a normal
am-based rebase couldn't.  merge-based rebases were added in mid-2006
to handle renames, but am-based rebases gained that ability at the end
of 2008.  Basically, rebase -m was dormant and useless...until
directory rename detection became a thing this cycle.  (Also, in
config options and documentation merge tends to be overlooked; just a
single example is that pull.rebase can be set to interactive, but not
to merge.)

Anyway, with this in mind, I didn't think those extra messages were
all that important.  If others disagree I can look into adding them --
that'd also make the --quiet mode more useful for interactive, since
there'd be more messages for it to strip out.

>>  test_expect_success "rebase -p is no-op in non-linear history" "
>> diff --git a/t/t5407-post-rewrite-hook.sh b/t/t5407-post-rewrite-hook.sh
>> index 9b2a274c71..145c61251d 100755
>> --- a/t/t5407-post-rewrite-hook.sh
>> +++ b/t/t5407-post-rewrite-hook.sh
>> @@ -120,6 +120,7 @@ test_expect_success 'git rebase -m --skip' '
>>       git rebase --continue &&
>>       echo rebase >expected.args &&
>>       cat >expected.data <<-EOF &&
>> +     $(git rev-parse C) $(git rev-parse HEAD^)
>>       $(git rev-parse D) $(git rev-parse HEAD)
>>       EOF
>>       verify_hook_input
>
> This is a bit sad, because it seems to suggest that we now do more
> unnecessary work here, or is my reading incorrect?

I agree that it's a bit sad.  I spent a while looking at this testcase
and puzzling over what it meant, and my commit message pointed out
that I wasn't quite sure where it came from:

      t5407: different rebase types varied slightly in how many times checkout
             or commit or equivalents were called based on a quick comparison
             of this tests and previous ones which covered different rebase
             flavors.  I think this is just attributable to this difference.

It would be nice to avoid the extra work, but I'm worried tackling
that might cause me to step on toes of folks doing the rewrite of
interactive rebases from shell to C.  Maybe I should just add a TODO
note in the testcase, similar to the one in t3425 near a few lines I
touched in this patch?


Thanks for the detailed feedback and suggestions!

Elijah

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Elijah Newren]

Hi Dscho,

On Sat, Jun 9, 2018 at 3:11 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> On Thu, 7 Jun 2018, Elijah Newren wrote:
>
>> am-based rebases suffer from an reduced ability to detect directory
>> renames upstream, which is fundamental to the fact that it throws away
>> information about the history: in particular, it dispenses with the
>> original commits involved by turning them into patches, and without the
>> knowledge of the original commits we cannot determine a proper merge base.
>>
>> The am-based rebase will proceed by first trying git-apply, and, only if
>> it fails, will try to reconstruct a provisional merge base using
>> build_fake_ancestor().  This fake ancestor will ONLY contain the files
>> modified in the patch; without the full list of files in the real merge
>> base, renames of any missing files cannot be detected.  Directory rename
>> detection works by looking at individual file renames and deducing when a
>> full directory has been renamed.
>>
>> Trying to modify build_fake_ancestor() to instead create a merge base that
>> includes common file information by looking for a commit that contained
>> all the same blobs as the pre-image of the patch would require a very
>> expensive search through history, may find the wrong commit, and outside
>> of rebase may not find any commit that matches.
>>
>> But we had all the relevant information to start.  So, instead of
>> attempting to fix am which just doesn't have the relevant information,
>> instead note its strength and weaknesses, and change the default rebase
>> machinery to interactive since it does not suffer from the same problems.
>
> I'll let Eric comment on the grammar, and I'll comment on the idea behind
> this commit instead.

Going to dump the hard job on Eric, eh?  ;-)

> IMHO `git rebase -i` is still too slow to be a true replacement for `git
> rebase --am` for the cases where it serves the user well. Maybe we should
> work on making `rebase -i` faster, first?

That sounds fair.

> I imagine, for example, that it might make *tons* of sense to avoid
> writing out the index and worktree files all the time. That was necessary
> in the shell script version because if the ridiculous limitations we
> subjected ourselves to, such as: no object-oriented state worth
> mentioning, only string-based processing, etc. But we could now start to
> do everything in memory (*maybe* write out the new blob/tree/commit
> objects immediately, but maybe not) until the time when we either have
> succeeded in the rebase, or when there was a problem and we have to exit
> with an error. And only then write the files and the index.

Hmm...are you still planning on using cherry-pick (internally rather
than forking, of course)?  Because cherry-pick uses the
merge-recursive machinery, and the merge-recursive machinery doesn't
have a nice way of avoiding writing to the working tree or index.
Fixing that is on my radar; see the first block of
https://public-inbox.org/git/CABPp-BG2fZHm3s-yrzxyGj3Eh+O7_LHLz5pgstHhG2drigSyRA@mail.gmail.com/
(reading up until "At this point, I'd rather just fix the design flaw
rather than complicate the code further.")

However, also covered in my plans is a few things to speed up the
merge-recursive machinery, which should provide some other performance
benefits for interactive rebases.

> In any case, I think that the rather noticeable change of the default
> would probably necessitate a deprecation phase.

Why is it a "rather noticable change of the default"?  If we popped up
the editor and asked the user to edit the list of options, I'd agree,
or if folks thought that it was significantly slower by a big enough
margin (though you already suggested waiting and making sure we don't
do that).  What else remains that qualifies?

(Okay, the default behavior to just skip empty patches rather than
halt the rebase and ask the user to advise is different, but we could
fix that up too.  Is there anything else?)

Re: [PATCH] git-rebase.sh: handle keep-empty like all other options

[Phillip Wood]

Hi Elijah
On 07/06/18 06:07, Elijah Newren wrote:
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>   git-rebase.sh | 6 +-----
>   1 file changed, 1 insertion(+), 5 deletions(-)
> 
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 40be59ecc4..a56b286372 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -276,6 +276,7 @@ do
>   		;;
>   	--keep-empty)
>   		keep_empty=yes
> +		test -z "$interactive_rebase" && interactive_rebase=implied

I think you need to wait until all the options have been parsed before 
setting the implied interactive rebase in case the user specifies has 
'--keep-empty' in an alias and specifies '--no-keep-empty' with some am 
options on the command line.

Best Wishes

Phillip
>   		;;
>   	--allow-empty-message)
>   		allow_empty_message=--allow-empty-message
> @@ -480,11 +481,6 @@ then
>   	test -z "$interactive_rebase" && interactive_rebase=implied
>   fi
>   
> -if test -n "$keep_empty"
> -then
> -	test -z "$interactive_rebase" && interactive_rebase=implied
> -fi
> -
>   if test -n "$interactive_rebase"
>   then
>   	type=interactive
> 

Re: [PATCH 2/2] git-rebase: error out when incompatible options passed

[Phillip Wood]

On 07/06/18 06:06, Elijah Newren wrote:
> git rebase has three different types: am, merge, and interactive, all of
> which are implemented in terms of separate scripts.  am builds on git-am,
> merge builds on git-merge-recursive, and interactive builds on
> git-cherry-pick.  We make use of features in those lower-level commands in
> the different rebase types, but those features don't exist in all of the
> lower level commands so we have a range of incompatibilities.  Previously,
> we just accepted nearly any argument and silently ignored whichever ones
> weren't implemented for the type of rebase specified.  Change this so the
> incompatibilities are documented, included in the testsuite, and tested
> for at runtime with an appropriate error message shown.

I think this is a great improvement, it has always bothered me that 
rebase silently ignored the am options when they're given with 
interactive ones.
> 
> Some exceptions I left out:
> 
>    * --merge and --interactive are technically incompatible since they are
>      supposed to run different underlying scripts, but with a few small
>      changes, --interactive can do everything that --merge can.  In fact,
>      I'll shortly be sending another patch to remove git-rebase--merge and
>      reimplement it on top of git-rebase--interactive.

Excellent I've wondered about doing that but never got round to it. One 
thing I was slightly concerned about was that someone maybe parsing the 
output of git-rebase--merge and they'll get a nasty shock when that 
output changes as a result of using the sequencer.

> 
>    * One could argue that --interactive and --quiet are incompatible since
>      --interactive doesn't implement a --quiet mode (perhaps since
>      cherry-pick itself does not implement one).  However, the interactive
>      mode is more quiet than the other modes in general with progress
>      messages, so one could argue that it's already quiet.
> 
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>   Documentation/git-rebase.txt           | 15 +++++++++++++--
>   git-rebase.sh                          | 17 +++++++++++++++++
>   t/t3422-rebase-incompatible-options.sh | 10 +++++-----
>   3 files changed, 35 insertions(+), 7 deletions(-)
> 
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 0e20a66e73..451252c173 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -243,6 +243,10 @@ leave out at most one of A and B, in which case it defaults to HEAD.
>   --keep-empty::
>   	Keep the commits that do not change anything from its
>   	parents in the result.
> ++
> +This uses the `--interactive` machinery internally, and as such,
> +anything that is incompatible with --interactive is incompatible
> +with this option.
>   
>   --allow-empty-message::
>   	By default, rebasing commits with an empty message will fail.
> @@ -324,6 +328,8 @@ which makes little sense.
>   	and after each change.  When fewer lines of surrounding
>   	context exist they all must match.  By default no context is
>   	ever ignored.
> +	Incompatible with the --merge and --interactive options, or
> +	anything that implies those options or their machinery.

struct replay_opts has an allow_empty_message member so I'm not sure 
that's true.

>   -f::
>   --force-rebase::
> @@ -355,13 +361,15 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
>   --whitespace=<option>::
>   	These flag are passed to the 'git apply' program
>   	(see linkgit:git-apply[1]) that applies the patch.
> -	Incompatible with the --interactive option.
> +	Incompatible with the --merge and --interactive options, or
> +	anything that implies those options or their machinery.

I wonder if it is better just to list the incompatible options it might 
be a bit long but it would be nicer for the user than them having to 
work out which options imply --interactive.

>   --committer-date-is-author-date::
>   --ignore-date::
>   	These flags are passed to 'git am' to easily change the dates
>   	of the rebased commits (see linkgit:git-am[1]).
> -	Incompatible with the --interactive option.
> +	Incompatible with the --merge and --interactive options, or
> +	anything that implies those options or their machinery.
>   
>   --signoff::
>   	Add a Signed-off-by: trailer to all the rebased commits. Note
> @@ -400,6 +408,9 @@ The `--rebase-merges` mode is similar in spirit to `--preserve-merges`, but
>   in contrast to that option works well in interactive rebases: commits can be
>   reordered, inserted and dropped at will.
>   +
> +This uses the `--interactive` machinery internally, but it can be run
> +without an explicit `--interactive`.
> ++

Without more context it's hard to judge but I'm not sure this adds 
anything useful

>   It is currently only possible to recreate the merge commits using the
>   `recursive` merge strategy; Different merge strategies can be used only via
>   explicit `exec git merge -s <strategy> [...]` commands.
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 40be59ecc4..f1dbecba18 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -503,6 +503,23 @@ then
>   	git_format_patch_opt="$git_format_patch_opt --progress"
>   fi
>   
> +if test -n "$git_am_opt"; then
> +	incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`

I think the style guide recommends $() over ``

> +	if test -n "$interactive_rebase"
> +	then
> +		if test -n "$incompatible_opts"
> +		then
> +			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
> +		fi
> +	fi
> +	if test -n "$do_merge"; then
> +		if test -n "$incompatible_opts"
> +		then
> +			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
> +		fi
> +	fi
> +fi
> +
>   if test -n "$signoff"
>   then
>   	test -n "$preserve_merges" &&
> diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
> index 04cdae921b..66a83363bf 100755
> --- a/t/t3422-rebase-incompatible-options.sh
> +++ b/t/t3422-rebase-incompatible-options.sh
> @@ -34,27 +34,27 @@ test_expect_success 'setup' '
>   test_run_rebase () {
>   	opt=$1
>   	shift
> -	test_expect_failure "$opt incompatible with --merge" "
> +	test_expect_success "$opt incompatible with --merge" "
>   		git checkout B^0 &&
>   		test_must_fail git rebase $opt --merge A
>   	"
>   
> -	test_expect_failure "$opt incompatible with --strategy=ours" "
> +	test_expect_success "$opt incompatible with --strategy=ours" "
>   		git checkout B^0 &&
>   		test_must_fail git rebase $opt --strategy=ours A
>   	"
>   
> -	test_expect_failure "$opt incompatible with --strategy-option=ours" "
> +	test_expect_success "$opt incompatible with --strategy-option=ours" "
>   		git checkout B^0 &&
>   		test_must_fail git rebase $opt --strategy=ours A
>   	"
>   
> -	test_expect_failure "$opt incompatible with --interactive" "
> +	test_expect_success "$opt incompatible with --interactive" "
>   		git checkout B^0 &&
>   		test_must_fail git rebase $opt --interactive A
>   	"
>   
> -	test_expect_failure "$opt incompatible with --exec" "
> +	test_expect_success "$opt incompatible with --exec" "
>   		git checkout B^0 &&
>   		test_must_fail git rebase $opt --exec 'true' A
>   	"
> 

Re: [RFC PATCH 2/3] rebase: Implement --merge via git-rebase--interactive

[Phillip Wood]

Hi Elijah

On 10/06/18 02:14, Elijah Newren wrote:
> 
> Hi Dscho,
> 
> On Sat, Jun 9, 2018 at 3:04 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> On Thu, 7 Jun 2018, Elijah Newren wrote:
>>
>>> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
>>> index 451252c173..28d1658d7a 100644
>>> --- a/Documentation/git-rebase.txt
>>> +++ b/Documentation/git-rebase.txt
>>> @@ -275,6 +275,10 @@ branch on top of the <upstream> branch.  Because of this, when a merge
>>>  conflict happens, the side reported as 'ours' is the so-far rebased
>>>  series, starting with <upstream>, and 'theirs' is the working branch.  In
>>>  other words, the sides are swapped.
>>> ++
>>> +This uses the `--interactive` machinery internally, and as such,
>>> +anything that is incompatible with --interactive is incompatible
>>> +with this option.
>>
>> I am not sure I like this change, as it describes an implementation detail
>> that users should neither know, nor care about, nor rely on.
> 
> Let me back up for just a second to see if I can point out the real
> problem I'm trying to address here, which you may have a better
> solution for.  What should happen if a user runs
>    git rebase --merge --whitespace=fix
> ?
> 
> git currently silently ignores the --whitepsace=fix argument, leaving
> the whitespace damage present at the end of the rebase.  Same goes for
> --interactive combined with any am-specific options  (Fix proposed at
> https://public-inbox.org/git/20180607050654.19663-2-newren@gmail.com/).
> This silent ignoring of some options depending on which other options
> were specified has caused me problems in the past.
> 
> So, while I totally agree with you that users shouldn't need to know
> implementation details, they do need to know how to use commands and
> which options go well together and which are mutually incompatible.
> Do you have any suggestions on alternate wording that would convey the
> sets of mutually incompatible options without talking about
> implementation details?  Would you rather only address that in the
> code and not mention it in the documentation?
> 
> See also https://public-inbox.org/git/20180607050654.19663-1-newren@gmail.com/,
> which proposes testcases for these incompatibility sets.
> 
> 
>>> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
>>> index 1f2401f702..dcc4a26a78 100644
>>> --- a/git-rebase--interactive.sh
>>> +++ b/git-rebase--interactive.sh
>>> @@ -885,7 +885,10 @@ init_basic_state () {
>>>       mkdir -p "$state_dir" || die "$(eval_gettext "Could not create temporary \$state_dir")"
>>>       rm -f "$(git rev-parse --git-path REBASE_HEAD)"
>>>
>>> -     : > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
>>> +     if test -n "$actually_interactive"
>>> +     then
>>> +             : > "$state_dir"/interactive || die "$(gettext "Could not mark as interactive")"
>>> +     fi
>>
>> Do we really care at this stage? IOW what breaks if we still write that
>> file, even in --merge mode?
> 
> Two things will change if we do that:
>   * bash prompt will be different for those using git-prompt.sh
> (REBASE-m vs. REBASE-i).
>   * git-status output is no longer the same ('rebase in progress' vs.
> 'interactive rebase in progress...last command done:...pick 0dea123 my
> wonderful commit')
> 
> I don't think the prompt is a big deal.  The status output might not
> be either, but showing the "last command done" may be weird to someone
> that never saw or edited a list of commands.  (Then again, that same
> argument could be made for users of --exec, --rebase-merges, or
> --keep-empty without an explicit --interactive)
> 
> Opinions on whether these two difference matter?  If others don't
> think these differences are significant, I'm happy to update any
> necessary testcases and just unconditionally create the
> $state_dir/interactive file.

I'm inclined to agree that it should just write the file
unconditionally. As you point out it already does this for other things
that aren't "interactive" as far as the user is concerned. Someone could
always add an file to indicate the "real" mode later that would work for
all the implicit interactive rebase cases if it was important to them.

Best Wishes

Phillip

> 
>>> @@ -501,17 +502,11 @@ fi
>>>
>>>  if test -n "$git_am_opt"; then
>>>       incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
>>> -     if test -n "$interactive_rebase"
>>> +     if test -n "$incompatible_opts"
>>
>> Did you not mean to turn this into a test for actually_interactve ||
>> do_merge?
>>
>>>       then
>>> -             if test -n "$incompatible_opts"
>>> +             if test -n "$actually_interactive" || test "$do_merge"
>>
>> This could now be combined with the previous if (and the `-n` could be
>> added to the latter test):
>>
>>         if test -n "$actually_interactive" -o -n "$do_merge" &&
>>                 test -n "$incompatible_opts"
>>
>> The indentation would change, but this hunk is already confusing to read,
>> anyway, so...
> 
> I'm happy to switch the order of the nesting as you suggest and agree
> that it would make it easier to read.  However, I hesitate to combine
> the two if-lines.  When I read your combined suggested line, I parsed
> it as follows (using invalid pseduo-syntax just to convey grouping):
> 
>   ( -n "$actually_interactive ) || ( -n "$do_merge" && -n "$incompatible_opts" )
> 
> Granted, I parsed it wrong, putting the parentheses in the wrong
> places, and bash parses it as you intended.  While you may have
> precedence and left- vs. right- associativity rules down pat, I
> clearly didn't.  If we combine the lines, I'll probably mis-read them
> again when I see them in a year or more.
> 
>>> @@ -704,6 +699,22 @@ then
>>>       GIT_PAGER='' git diff --stat --summary "$mb" "$onto"
>>>  fi
>>>
>>> +if test -z "$actually_interactive"
>>> +then
>>> +     # If the $onto is a proper descendant of the tip of the branch, then
>>> +     # we just fast-forwarded.
>>> +     if test "$mb" = "$orig_head"
>>> +     then
>>
>> Likewise, this would be easier to read as
>>
>>         if test -z "$actually_interactive" && test "$mb" = "$orig_head"
> 
> Good point, I'll fix that up.
> 
>> Also: how certain are we that "$mb" does not start with a dash? We may
>> have to use the `test a"$mb" = a"$orig_head"` trick here... But I guess
>> this is moved code, so if it is buggy, it was buggy before.
> 
> From earlier in the file,
> mb=$(git merge-base ...)
> 
> So, unless we're expecting the output of git-merge-base to change in
> the future to include leading dashes, we shouldn't hit any issues.  I
> can make the change you suggest if you're worried, though.
> 
>>> diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
>>> index 0392e36d23..04d6c71899 100755
>>> --- a/t/t3406-rebase-message.sh
>>> +++ b/t/t3406-rebase-message.sh
>>> @@ -17,14 +17,9 @@ test_expect_success 'setup' '
>>>       git tag start
>>>  '
>>>
>>> -cat >expect <<\EOF
>>> -Already applied: 0001 A
>>> -Already applied: 0002 B
>>> -Committed: 0003 Z
>>> -EOF
>>> -
>>>  test_expect_success 'rebase -m' '
>>>       git rebase -m master >report &&
>>> +     >expect &&
>>>       sed -n -e "/^Already applied: /p" \
>>>               -e "/^Committed: /p" report >actual &&
>>>       test_cmp expect actual
>>
>> This might be called a regression... I don't know any user of `git rebase
>> -m`, but I guess if I was one, I would like to keep seeing those messages.
>>
>> It *should* be relatively easy to tell the sequencer.c to issue these
>> messages, I think. But it would be more work.
> 
> You may well be right.  Here's where my thinking came from...
> 
> am-based, interactive-based, and merge-based rebases have lots of
> little ways in which they have differed, this being just one of them.
> It was sometimes hard making a judgement call when writing this series
> about whether any given deviation was a difference I wanted to smooth
> over or a difference I wanted to perpetuate between various flags.
> Further, if it was a difference I wanted to smooth over, then I had to
> decide which of the current behaviors was 'correct'.
> 
> In this particular case, I basically went off perceived usage.
> am-based rebases have lots of special flags relevant to it only
> (--whitespace, -C, etc.) and is the default, so it clearly has lots of
> users.  interactive-based rebases are pretty prominent too, and have
> very specific special capabilities the other modes don't.  In
> contrast, merge-based rebases can't do a single thing that interactive
> can't; and unless you're using a special merge strategy, for the last
> 10 years merge-based rebases haven't been able to do anything a normal
> am-based rebase couldn't.  merge-based rebases were added in mid-2006
> to handle renames, but am-based rebases gained that ability at the end
> of 2008.  Basically, rebase -m was dormant and useless...until
> directory rename detection became a thing this cycle.  (Also, in
> config options and documentation merge tends to be overlooked; just a
> single example is that pull.rebase can be set to interactive, but not
> to merge.)
> 
> Anyway, with this in mind, I didn't think those extra messages were
> all that important.  If others disagree I can look into adding them --
> that'd also make the --quiet mode more useful for interactive, since
> there'd be more messages for it to strip out.
> 
>>>  test_expect_success "rebase -p is no-op in non-linear history" "
>>> diff --git a/t/t5407-post-rewrite-hook.sh b/t/t5407-post-rewrite-hook.sh
>>> index 9b2a274c71..145c61251d 100755
>>> --- a/t/t5407-post-rewrite-hook.sh
>>> +++ b/t/t5407-post-rewrite-hook.sh
>>> @@ -120,6 +120,7 @@ test_expect_success 'git rebase -m --skip' '
>>>       git rebase --continue &&
>>>       echo rebase >expected.args &&
>>>       cat >expected.data <<-EOF &&
>>> +     $(git rev-parse C) $(git rev-parse HEAD^)
>>>       $(git rev-parse D) $(git rev-parse HEAD)
>>>       EOF
>>>       verify_hook_input
>>
>> This is a bit sad, because it seems to suggest that we now do more
>> unnecessary work here, or is my reading incorrect?
> 
> I agree that it's a bit sad.  I spent a while looking at this testcase
> and puzzling over what it meant, and my commit message pointed out
> that I wasn't quite sure where it came from:
> 
>       t5407: different rebase types varied slightly in how many times checkout
>              or commit or equivalents were called based on a quick comparison
>              of this tests and previous ones which covered different rebase
>              flavors.  I think this is just attributable to this difference.
> 
> It would be nice to avoid the extra work, but I'm worried tackling
> that might cause me to step on toes of folks doing the rewrite of
> interactive rebases from shell to C.  Maybe I should just add a TODO
> note in the testcase, similar to the one in t3425 near a few lines I
> touched in this patch?
> 
> 
> Thanks for the detailed feedback and suggestions!
> 
> Elijah
> 

Re: [PATCH] git-rebase.sh: handle keep-empty like all other options

[Elijah Newren]

Hi Phillip,

On Sun, Jun 10, 2018 at 12:26 PM, Phillip Wood
<phillip.wood@talktalk.net> wrote:
> On 07/06/18 06:07, Elijah Newren wrote:
>>
>> Signed-off-by: Elijah Newren <newren@gmail.com>
>> ---
>>   git-rebase.sh | 6 +-----
>>   1 file changed, 1 insertion(+), 5 deletions(-)
>>
>> diff --git a/git-rebase.sh b/git-rebase.sh
>> index 40be59ecc4..a56b286372 100755
>> --- a/git-rebase.sh
>> +++ b/git-rebase.sh
>> @@ -276,6 +276,7 @@ do
>>                 ;;
>>         --keep-empty)
>>                 keep_empty=yes
>> +               test -z "$interactive_rebase" &&
>> interactive_rebase=implied
>
>
> I think you need to wait until all the options have been parsed before
> setting the implied interactive rebase in case the user specifies has
> '--keep-empty' in an alias and specifies '--no-keep-empty' with some am
> options on the command line.

Ah, indeed you are right.  Let's drop this patch then.

However, we have a bigger problem with empty commits, in that there
are really three modes rather than two:
  * Automatically drop empty commits (default for am-based rebase)
  * Automatically keep empty commits (as done with --keep-empty)
  * Halt the rebase and tell the user how to specify if they want to
keep it (default for interactive rebases)

Currently, only the first option is available to am-based rebases, and
only the second two options are available to interactive-based
rebases.  But if we want to make all three available to
interactive-based rebases, what should the command line option look
like?  --empty={drop,ask,keep}?

(And deprecate but continue to support --[no-]keep-empty?)

And should the two rebase modes really have a different default?  What
should the default be?

Re: [PATCH] git-rebase.sh: handle keep-empty like all other options

[Phillip Wood]

Hi Elijah

On 11/06/18 15:42, Elijah Newren wrote:
> Hi Phillip,
> 
> On Sun, Jun 10, 2018 at 12:26 PM, Phillip Wood
> <phillip.wood@talktalk.net> wrote:
>> On 07/06/18 06:07, Elijah Newren wrote:
>>>
>>> Signed-off-by: Elijah Newren <newren@gmail.com>
>>> ---
>>>    git-rebase.sh | 6 +-----
>>>    1 file changed, 1 insertion(+), 5 deletions(-)
>>>
>>> diff --git a/git-rebase.sh b/git-rebase.sh
>>> index 40be59ecc4..a56b286372 100755
>>> --- a/git-rebase.sh
>>> +++ b/git-rebase.sh
>>> @@ -276,6 +276,7 @@ do
>>>                  ;;
>>>          --keep-empty)
>>>                  keep_empty=yes
>>> +               test -z "$interactive_rebase" &&
>>> interactive_rebase=implied
>>
>>
>> I think you need to wait until all the options have been parsed before
>> setting the implied interactive rebase in case the user specifies has
>> '--keep-empty' in an alias and specifies '--no-keep-empty' with some am
>> options on the command line.
> 
> Ah, indeed you are right.  Let's drop this patch then.
> 
> However, we have a bigger problem with empty commits, in that there
> are really three modes rather than two:
>    * Automatically drop empty commits (default for am-based rebase)
>    * Automatically keep empty commits (as done with --keep-empty)
>    * Halt the rebase and tell the user how to specify if they want to
> keep it (default for interactive rebases)
> 
> Currently, only the first option is available to am-based rebases, and
> only the second two options are available to interactive-based
> rebases.  

I'm not sure that's the case, my understanding is that for an 
interactive rebase unless you give '--keep-empty' then any empty commits 
will be commented out in the todo list and therefore dropped unless 
they're uncommented when editing the list. The third case happens when a 
commit becomes empty when it's rebased (i.e. the original commit is not 
empty), I'm not sure what the am backend does for this. cherry-pick has 
'--keep-redundant-commits' for this case but that has never been added 
to rebase.

Best Wishes

Phillip

But if we want to make all three available to
> interactive-based rebases, what should the command line option look
> like?  --empty={drop,ask,keep}?
> 
> (And deprecate but continue to support --[no-]keep-empty?)
> 
> And should the two rebase modes really have a different default?  What
> should the default be?
> 

Re: [PATCH 2/2] git-rebase: error out when incompatible options passed

[Elijah Newren]

Hi Phillip

On Sun, Jun 10, 2018 at 12:40 PM, Phillip Wood
<phillip.wood@talktalk.net> wrote:

>>   Documentation/git-rebase.txt           | 15 +++++++++++++--
>>   git-rebase.sh                          | 17 +++++++++++++++++
>>   t/t3422-rebase-incompatible-options.sh | 10 +++++-----
>>   3 files changed, 35 insertions(+), 7 deletions(-)
>>
>> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
>> index 0e20a66e73..451252c173 100644
>> --- a/Documentation/git-rebase.txt
>> +++ b/Documentation/git-rebase.txt
>> @@ -243,6 +243,10 @@ leave out at most one of A and B, in which case it
>> defaults to HEAD.
>>   --keep-empty::
>>         Keep the commits that do not change anything from its
>>         parents in the result.
>> ++
>> +This uses the `--interactive` machinery internally, and as such,
>> +anything that is incompatible with --interactive is incompatible
>> +with this option.
>>     --allow-empty-message::
>>         By default, rebasing commits with an empty message will fail.
>> @@ -324,6 +328,8 @@ which makes little sense.
>>         and after each change.  When fewer lines of surrounding
>>         context exist they all must match.  By default no context is
>>         ever ignored.
>> +       Incompatible with the --merge and --interactive options, or
>> +       anything that implies those options or their machinery.
>
>
> struct replay_opts has an allow_empty_message member so I'm not sure that's
> true.

I think you were confused by the way the patch broke up.  The jump to
line 328 means that this comment is about the -C option, not the
--allow-empty-message option.

However, I probably should add a comment next to the
--allow-empty-message option, to not the reverse is true, i.e. that
it's incompatible with am-based rebases.  (git-rebase--am.sh ignores
the allow_empty_message variable set in git-rebase.sh, unlike
git-rebase--interactive.sh and git-rebase--merge.sh)

>>   -f::
>>   --force-rebase::
>> @@ -355,13 +361,15 @@ default is `--no-fork-point`, otherwise the default
>> is `--fork-point`.
>>   --whitespace=<option>::
>>         These flag are passed to the 'git apply' program
>>         (see linkgit:git-apply[1]) that applies the patch.
>> -       Incompatible with the --interactive option.
>> +       Incompatible with the --merge and --interactive options, or
>> +       anything that implies those options or their machinery.
>
>
> I wonder if it is better just to list the incompatible options it might be a
> bit long but it would be nicer for the user than them having to work out
> which options imply --interactive.

That could work.  Would this be done at the end of the 'OPTIONS'
section of the manpage?  Should I create an 'INCOMPATIBLE OPTIONS'
section that follows the 'OPTIONS' section?

>>   --committer-date-is-author-date::
>>   --ignore-date::
>>         These flags are passed to 'git am' to easily change the dates
>>         of the rebased commits (see linkgit:git-am[1]).
>> -       Incompatible with the --interactive option.
>> +       Incompatible with the --merge and --interactive options, or
>> +       anything that implies those options or their machinery.
>>     --signoff::
>>         Add a Signed-off-by: trailer to all the rebased commits. Note
>> @@ -400,6 +408,9 @@ The `--rebase-merges` mode is similar in spirit to
>> `--preserve-merges`, but
>>   in contrast to that option works well in interactive rebases: commits
>> can be
>>   reordered, inserted and dropped at will.
>>   +
>> +This uses the `--interactive` machinery internally, but it can be run
>> +without an explicit `--interactive`.
>> ++
>
> Without more context it's hard to judge but I'm not sure this adds anything
> useful

Hmm, yeah.  I noted that --exec had similar wording, noted that
--preserve-merges had something along the same lines but as a warning,
and didn't see the similar wording for --rebase-merges -- I somehow
missed the paragraph right above where I added these lines.  Oops.
Anyway, I'll pull it out.

>>   It is currently only possible to recreate the merge commits using the
>>   `recursive` merge strategy; Different merge strategies can be used only
>> via
>>   explicit `exec git merge -s <strategy> [...]` commands.
>> diff --git a/git-rebase.sh b/git-rebase.sh
>> index 40be59ecc4..f1dbecba18 100755
>> --- a/git-rebase.sh
>> +++ b/git-rebase.sh
>> @@ -503,6 +503,23 @@ then
>>         git_format_patch_opt="$git_format_patch_opt --progress"
>>   fi
>>   +if test -n "$git_am_opt"; then
>> +       incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
>
>
> I think the style guide recommends $() over ``

Will fix.


Thanks for taking a look!

Elijah

Re: [PATCH 2/2] git-rebase: error out when incompatible options passed

[Elijah Newren]

Another thing I missed...

On Sun, Jun 10, 2018 at 12:40 PM, Phillip Wood
<phillip.wood@talktalk.net> wrote:
> On 07/06/18 06:06, Elijah Newren wrote:

>> Some exceptions I left out:
>>
>>    * --merge and --interactive are technically incompatible since they are
>>      supposed to run different underlying scripts, but with a few small
>>      changes, --interactive can do everything that --merge can.  In fact,
>>      I'll shortly be sending another patch to remove git-rebase--merge and
>>      reimplement it on top of git-rebase--interactive.
>
> Excellent I've wondered about doing that but never got round to it. One
> thing I was slightly concerned about was that someone maybe parsing the
> output of git-rebase--merge and they'll get a nasty shock when that output
> changes as a result of using the sequencer.

I can see the minor worry, but I think upon inspection it's not
something that concerns me, for a few reasons:

In terms of use, given that rebase --merge was introduced to handle
renames in mid-2006, but plain rebase has been able to handle them
since late 2008 (though directory renames changes that again), the
utility of rebase --merge has been somewhat limited.  I think that
limits the exposure.  But to address the 'break' more directly...

If we were to agree that we needed to support folks parsing rebase
output, that would be a really strict requirement that I think would
prevent lots of fixes.  And if so, it's one we've violated a number of
times.  For example, I certainly wasn't thinking about rebase when I
modified messages in merge-recursive.c over the years, but they'd leak
through for rebase --merge.  (Those messages would not leak through to
rebase --interactive as much, since the sequencer sets o.buffer_output
and then only conditionally shows the output.)  Also, changes that
have occurred in the past, like adding 'git gc --auto' to rebase,
modifying error messages directly found in git-rebase--merge.sh would
have been considered breaks.

Finally, looking over all the differences in output while fixing up
testcases makes me think we've done much less around designing the
output based on what we want the user to see, and more around what
minimal fixups can we do to these lower level commands that provide
useful functionality to the user?  We linearize history differently
for different rebase modes, have different entries in the reflog
depending on which mode, and we often times implement features for
just one mode and then sometimes add it to others.  In fact, I think
the primary reason that am-based and merge-based rebases had a --quiet
option and the interactive rebases didn't, is mostly attributable to
the defaults of the lower level commands these three were built on top
of (git-am vs. git-merge-recursive vs. git-cherry-pick).  The noiser
two merited a quiet option, and the option was never added for the
last one.

Anyway, that's my rationale.  I'm curious to hear if folks disagree or
see things I'm overlooking or have reasons I might be weighting
tradeoffs less than optimally.

Re: [PATCH 2/2] git-rebase: error out when incompatible options passed

[Phillip Wood]

On 11/06/18 16:19, Elijah Newren wrote:
> Hi Phillip
> 
> On Sun, Jun 10, 2018 at 12:40 PM, Phillip Wood
> <phillip.wood@talktalk.net> wrote:
> 
>>>    Documentation/git-rebase.txt           | 15 +++++++++++++--
>>>    git-rebase.sh                          | 17 +++++++++++++++++
>>>    t/t3422-rebase-incompatible-options.sh | 10 +++++-----
>>>    3 files changed, 35 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
>>> index 0e20a66e73..451252c173 100644
>>> --- a/Documentation/git-rebase.txt
>>> +++ b/Documentation/git-rebase.txt
>>> @@ -243,6 +243,10 @@ leave out at most one of A and B, in which case it
>>> defaults to HEAD.
>>>    --keep-empty::
>>>          Keep the commits that do not change anything from its
>>>          parents in the result.
>>> ++
>>> +This uses the `--interactive` machinery internally, and as such,
>>> +anything that is incompatible with --interactive is incompatible
>>> +with this option.
>>>      --allow-empty-message::
>>>          By default, rebasing commits with an empty message will fail.
>>> @@ -324,6 +328,8 @@ which makes little sense.
>>>          and after each change.  When fewer lines of surrounding
>>>          context exist they all must match.  By default no context is
>>>          ever ignored.
>>> +       Incompatible with the --merge and --interactive options, or
>>> +       anything that implies those options or their machinery.
>>
>>
>> struct replay_opts has an allow_empty_message member so I'm not sure that's
>> true.
> 
> I think you were confused by the way the patch broke up.  The jump to
> line 328 means that this comment is about the -C option, not the
> --allow-empty-message option.

Ah you're right, I missed the hunk header

> However, I probably should add a comment next to the
> --allow-empty-message option, to not the reverse is true, i.e. that
> it's incompatible with am-based rebases.  (git-rebase--am.sh ignores
> the allow_empty_message variable set in git-rebase.sh, unlike
> git-rebase--interactive.sh and git-rebase--merge.sh)

That sounds like a good idea

>>>    -f::
>>>    --force-rebase::
>>> @@ -355,13 +361,15 @@ default is `--no-fork-point`, otherwise the default
>>> is `--fork-point`.
>>>    --whitespace=<option>::
>>>          These flag are passed to the 'git apply' program
>>>          (see linkgit:git-apply[1]) that applies the patch.
>>> -       Incompatible with the --interactive option.
>>> +       Incompatible with the --merge and --interactive options, or
>>> +       anything that implies those options or their machinery.
>>
>>
>> I wonder if it is better just to list the incompatible options it might be a
>> bit long but it would be nicer for the user than them having to work out
>> which options imply --interactive.
> 
> That could work.  Would this be done at the end of the 'OPTIONS'
> section of the manpage?  Should I create an 'INCOMPATIBLE OPTIONS'
> section that follows the 'OPTIONS' section?

I think that would be the best way of doing it, maybe with a note in the 
description of the am options to check in that section to see what they 
can be safely combined with.

>>>    --committer-date-is-author-date::
>>>    --ignore-date::
>>>          These flags are passed to 'git am' to easily change the dates
>>>          of the rebased commits (see linkgit:git-am[1]).
>>> -       Incompatible with the --interactive option.
>>> +       Incompatible with the --merge and --interactive options, or
>>> +       anything that implies those options or their machinery.
>>>      --signoff::
>>>          Add a Signed-off-by: trailer to all the rebased commits. Note
>>> @@ -400,6 +408,9 @@ The `--rebase-merges` mode is similar in spirit to
>>> `--preserve-merges`, but
>>>    in contrast to that option works well in interactive rebases: commits
>>> can be
>>>    reordered, inserted and dropped at will.
>>>    +
>>> +This uses the `--interactive` machinery internally, but it can be run
>>> +without an explicit `--interactive`.
>>> ++
>>
>> Without more context it's hard to judge but I'm not sure this adds anything
>> useful
> 
> Hmm, yeah.  I noted that --exec had similar wording, noted that
> --preserve-merges had something along the same lines but as a warning,
> and didn't see the similar wording for --rebase-merges -- I somehow
> missed the paragraph right above where I added these lines.  Oops.
> Anyway, I'll pull it out.

If we can get a good description of which options are compatible with 
what then hopefully we can remove the existing references to implicit 
interactive and am, the user should only have to worry about which 
options are compatible.

>>>    It is currently only possible to recreate the merge commits using the
>>>    `recursive` merge strategy; Different merge strategies can be used only
>>> via
>>>    explicit `exec git merge -s <strategy> [...]` commands.
>>> diff --git a/git-rebase.sh b/git-rebase.sh
>>> index 40be59ecc4..f1dbecba18 100755
>>> --- a/git-rebase.sh
>>> +++ b/git-rebase.sh
>>> @@ -503,6 +503,23 @@ then
>>>          git_format_patch_opt="$git_format_patch_opt --progress"
>>>    fi
>>>    +if test -n "$git_am_opt"; then
>>> +       incompatible_opts=`echo "$git_am_opt" | sed -e 's/ -q//'`
>>
>>
>> I think the style guide recommends $() over ``
> 
> Will fix.
> 
> 
> Thanks for taking a look!

Thanks for working on this, it should make things simpler for user's to 
understand

Best Wishes

Phillip
> Elijah
> 

Re: [PATCH] git-rebase.sh: handle keep-empty like all other options

[Elijah Newren]

Hi Phillip,

On Mon, Jun 11, 2018 at 8:16 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
> On 11/06/18 15:42, Elijah Newren wrote:

>> However, we have a bigger problem with empty commits, in that there
>> are really three modes rather than two:
>>    * Automatically drop empty commits (default for am-based rebase)
>>    * Automatically keep empty commits (as done with --keep-empty)
>>    * Halt the rebase and tell the user how to specify if they want to
>> keep it (default for interactive rebases)
>>
>> Currently, only the first option is available to am-based rebases, and
>> only the second two options are available to interactive-based
>> rebases.
>
>
> I'm not sure that's the case, my understanding is that for an interactive
> rebase unless you give '--keep-empty' then any empty commits will be
> commented out in the todo list and therefore dropped unless they're
> uncommented when editing the list.

Ah, yes, you are right; I was implicitly thinking about cases where
the commit became empty rather than when the commit started
empty...and mis-read --keep-empty (which as I learned now is only for
started-empty cases).

> The third case happens when a commit
> becomes empty when it's rebased (i.e. the original commit is not empty), I'm
> not sure what the am backend does for this.

The am backend does not halt the rebase; it simply drops the commit
and says "No changes -- Patch already applied."

It has always seemed jarring and confusing to me that one rebase mode
stops and asks the user what to do and the other assumes.  I think
both should have the same default (and have options to pick the
alternate behavior).

I'm less certain what the default should be, though.

> cherry-pick has
> '--keep-redundant-commits' for this case but that has never been added to
> rebase.

Thanks for the pointer.

Re: RFC: rebase inconsistency in 2.18 -- course of action?

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> In short, while interactive-based and merge-based rebases handle
> directory rename detection fine, am-based rebases do not.  This
> inconsistency may frustrate or surprise users.

FWIW, I actually do not quite think it is universally a good idea to
infer that I would have added the path X/F I added to my branch at
Y/F instead if I were on vacation while somebody else first moved
other paths X/A, X/B, etc. to Y/A, Y/B, etc., and I would even
imagine that I would be frustrated if my X/F, which the somebody
else wasn't even aware of, were moved to Y/F without even telling
me.  So in that sense, doing such extra and unasked-for "moves"
during a rebase may be a bug, not a feature.

In any case, I think I'll have to delay -rc2 by a few days to catch
up, so I won't be looking at any topic (including this one) that is
not about 2.18-rc regressions for a few days.  Please do not get
upset if people with RFC patches do not hear from me until the
final.

Thanks.

Re: RFC: rebase inconsistency in 2.18 -- course of action?

[Elijah Newren]

Hi Junio,

On Mon, Jun 11, 2018 at 10:44 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> In short, while interactive-based and merge-based rebases handle
>> directory rename detection fine, am-based rebases do not.  This
>> inconsistency may frustrate or surprise users.
>
> FWIW, I actually do not quite think it is universally a good idea to
> infer that I would have added the path X/F I added to my branch at
> Y/F instead if I were on vacation while somebody else first moved
> other paths X/A, X/B, etc. to Y/A, Y/B, etc., and I would even
> imagine that I would be frustrated if my X/F, which the somebody
> else wasn't even aware of, were moved to Y/F without even telling
> me.  So in that sense, doing such extra and unasked-for "moves"
> during a rebase may be a bug, not a feature.

So...I'm a little confused.  Am I correct to understand that you're
arguing that directory rename detection is an anti-feature and it
shouldn't have been merged in the first place?  Or that it doesn't
give appropriate output/warning, and it should have configuration
flags?  Or are you trying to make a distinction between am-based
rebases vs.any one of rebase -i, rebase -m, cherry-pick, and merge [-s
recursive]?

I was leaning towards "this inconsistency is no big deal; we can
address it later", but now you have me wondering about a different
angle.

> In any case, I think I'll have to delay -rc2 by a few days to catch
> up, so I won't be looking at any topic (including this one) that is
> not about 2.18-rc regressions for a few days.  Please do not get
> upset if people with RFC patches do not hear from me until the
> final.

I emailed about this rebase inconsistency because I was worried that
*an* inconsistency between the various rebase types was bad.  But the
more I've dug, the more I've found they are inconsistent in lots of
ways anyway (levels and types of output, reflog entries, automatically
dropping became-empty patches vs. halting and asking the user,
accepting commits with empty commit messages vs. requiring an
--allow-empty-message flag to do so, large sets of mutually
incompatible options, etc.)  Adding one more inconsistency is feeling
less worrisome to me.  I'd still like to lessen all these
inconsistencies between rebase types, but my current opinion is that
it's not that big a deal and my patches can all wait for the 2.19
cycle to start.

Re: [PATCH 2/2] git-rebase: error out when incompatible options passed

[Phillip Wood]

On 11/06/18 16:49, Elijah Newren wrote:
> Another thing I missed...
> 
> On Sun, Jun 10, 2018 at 12:40 PM, Phillip Wood
> <phillip.wood@talktalk.net> wrote:
>> On 07/06/18 06:06, Elijah Newren wrote:
> 
>>> Some exceptions I left out:
>>>
>>>    * --merge and --interactive are technically incompatible since they are
>>>      supposed to run different underlying scripts, but with a few small
>>>      changes, --interactive can do everything that --merge can.  In fact,
>>>      I'll shortly be sending another patch to remove git-rebase--merge and
>>>      reimplement it on top of git-rebase--interactive.
>>
>> Excellent I've wondered about doing that but never got round to it. One
>> thing I was slightly concerned about was that someone maybe parsing the
>> output of git-rebase--merge and they'll get a nasty shock when that output
>> changes as a result of using the sequencer.
> 
> I can see the minor worry, but I think upon inspection it's not
> something that concerns me, for a few reasons:
> 
> In terms of use, given that rebase --merge was introduced to handle
> renames in mid-2006, but plain rebase has been able to handle them
> since late 2008 (though directory renames changes that again), the
> utility of rebase --merge has been somewhat limited.  I think that
> limits the exposure.  But to address the 'break' more directly...
> 
> If we were to agree that we needed to support folks parsing rebase
> output, that would be a really strict requirement that I think would
> prevent lots of fixes.  And if so, it's one we've violated a number of
> times.  For example, I certainly wasn't thinking about rebase when I
> modified messages in merge-recursive.c over the years, but they'd leak
> through for rebase --merge.  (Those messages would not leak through to
> rebase --interactive as much, since the sequencer sets o.buffer_output
> and then only conditionally shows the output.)  Also, changes that
> have occurred in the past, like adding 'git gc --auto' to rebase,
> modifying error messages directly found in git-rebase--merge.sh would
> have been considered breaks.
> 
> Finally, looking over all the differences in output while fixing up
> testcases makes me think we've done much less around designing the
> output based on what we want the user to see, and more around what
> minimal fixups can we do to these lower level commands that provide
> useful functionality to the user?  We linearize history differently
> for different rebase modes, have different entries in the reflog
> depending on which mode, and we often times implement features for
> just one mode and then sometimes add it to others.  In fact, I think
> the primary reason that am-based and merge-based rebases had a --quiet
> option and the interactive rebases didn't, is mostly attributable to
> the defaults of the lower level commands these three were built on top
> of (git-am vs. git-merge-recursive vs. git-cherry-pick).  The noiser
> two merited a quiet option, and the option was never added for the
> last one.
> 
> Anyway, that's my rationale.  I'm curious to hear if folks disagree or
> see things I'm overlooking or have reasons I might be weighting
> tradeoffs less than optimally.
> 

I agree that there are already plenty of inconsistencies, (it's great to
see you reducing them). If we can avoid emulating the ouput of
git-rebase--merge.sh in sequencer.c that would definitely be my
preferred option (the code is already a bit hard to follow in places
where there it's doing slightly different things for cherry-pick and
rebase -i). Hopefully no one is relying on the output, as you say it's
just whatever the underlying plumbing prints rather than designed for a
specific purpose.

Best Wishes

Phillip

Re: [PATCH] git-rebase.sh: handle keep-empty like all other options

[Phillip Wood]

On 11/06/18 17:08, Elijah Newren wrote:
> Hi Phillip,
> 
> On Mon, Jun 11, 2018 at 8:16 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
>> On 11/06/18 15:42, Elijah Newren wrote:
> 
>>> However, we have a bigger problem with empty commits, in that there
>>> are really three modes rather than two:
>>>    * Automatically drop empty commits (default for am-based rebase)
>>>    * Automatically keep empty commits (as done with --keep-empty)
>>>    * Halt the rebase and tell the user how to specify if they want to
>>> keep it (default for interactive rebases)
>>>
>>> Currently, only the first option is available to am-based rebases, and
>>> only the second two options are available to interactive-based
>>> rebases.
>>
>>
>> I'm not sure that's the case, my understanding is that for an interactive
>> rebase unless you give '--keep-empty' then any empty commits will be
>> commented out in the todo list and therefore dropped unless they're
>> uncommented when editing the list.
> 
> Ah, yes, you are right; I was implicitly thinking about cases where
> the commit became empty rather than when the commit started
> empty...and mis-read --keep-empty (which as I learned now is only for
> started-empty cases).
> 
>> The third case happens when a commit
>> becomes empty when it's rebased (i.e. the original commit is not empty), I'm
>> not sure what the am backend does for this.
> 
> The am backend does not halt the rebase; it simply drops the commit
> and says "No changes -- Patch already applied."
> 
> It has always seemed jarring and confusing to me that one rebase mode
> stops and asks the user what to do and the other assumes.  I think
> both should have the same default (and have options to pick the
> alternate behavior).
> 
> I'm less certain what the default should be, though.

I'm not really sure either, on the one hand most of the time it is
convenient for rebase just to skip over it, on the other if you have
some important information in the commit message or a note then maybe
you want rebase to stop so you can selvage that information.

> 
>> cherry-pick has
>> '--keep-redundant-commits' for this case but that has never been added to
>> rebase.

On path forwards is to always stop, and implement
--keep-redundant-commits for rebase. That would be very easy for
interactive rebases as it shares the same code as cherry-pick. I've just
had a quick look at builtin/am.c and I think it would be fairly
straightforward to add some code to check if it's rebasing and stop if
the patch is already applied.

Best Wishes

Phillip

> Thanks for the pointer.
> 

[RFC PATCH v2 0/7] Document/fix/warn about rebase incompatibilities and inconsistences

[Elijah Newren]

git-rebase has lots of options that are mutually incompatible.  Even among
aspects of its behavior that is common to all rebase types, it has a number
of inconsistencies.  This series tries to document, fix, and/or warn users
about many of these.

I have a much higher than average expectation that folks will object
to some of these patches.  I've tried to divide them up so that any parts
we decide to drop or redo can be more easily excised.

No branch-diff; because it's a significant re-work; instead I'll comment
briefly on the individual patches...


Elijah Newren (7):
  git-rebase.txt: document incompatible options

Both Dscho (on a related patch series) and Phillip suggested changing the
documentation to avoid implementational details.  I instead made a separate
section with sets of incompatible options...but it still mentions the
different backends while doing so.  Does that seem alright?

  git-rebase.sh: update help messages a bit

Minor tweaks to `git rebase -h` output.

  t3422: new testcases for checking when incompatible options passed

The one unmodified patch from the first round.

  git-rebase: error out when incompatible options passed

Almost the same as the first round, except:
  * Documentation pulled into a separate patch (patch 1)
  * $() instead of ``

  git-rebase.txt: document behavioral inconsistencies between modes

Add another section to the documentation for aspects that ideally
should be common between all modes but are handled differently.

  git-rebase.txt: address confusion between --no-ff vs --force-rebase

This came up on the list not that long ago; fix the documentation.

  git-rebase: make --allow-empty-message the default

Address the easiest of the inconsistencies, assuming the am-based backend
has the correct default and the merge-based and interactive-based backends
are the ones that need to change.

 Documentation/git-rebase.txt           | 154 ++++++++++++++++++++-----
 git-rebase.sh                          |  25 +++-
 t/t3404-rebase-interactive.sh          |   7 +-
 t/t3405-rebase-malformed.sh            |  11 +-
 t/t3422-rebase-incompatible-options.sh |  69 +++++++++++
 5 files changed, 224 insertions(+), 42 deletions(-)
 create mode 100755 t/t3422-rebase-incompatible-options.sh

-- 
2.18.0.rc2.1.g5453d3f70b.dirty

[RFC PATCH v2 1/7] git-rebase.txt: document incompatible options

[Elijah Newren]

git rebase has many options that only work with one of its three backends.
It also has a few other pairs of incompatible options.  Document these.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 84 ++++++++++++++++++++++++++++++++----
 1 file changed, 76 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 0e20a66e73..adccc15284 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -243,11 +243,15 @@ leave out at most one of A and B, in which case it defaults to HEAD.
 --keep-empty::
 	Keep the commits that do not change anything from its
 	parents in the result.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --allow-empty-message::
 	By default, rebasing commits with an empty message will fail.
 	This option overrides that behavior, allowing commits with empty
 	messages to be rebased.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --skip::
 	Restart the rebasing process by skipping the current patch.
@@ -271,6 +275,8 @@ branch on top of the <upstream> branch.  Because of this, when a merge
 conflict happens, the side reported as 'ours' is the so-far rebased
 series, starting with <upstream>, and 'theirs' is the working branch.  In
 other words, the sides are swapped.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -s <strategy>::
 --strategy=<strategy>::
@@ -280,8 +286,10 @@ other words, the sides are swapped.
 +
 Because 'git rebase' replays each commit from the working branch
 on top of the <upstream> branch using the given strategy, using
-the 'ours' strategy simply discards all patches from the <branch>,
+the 'ours' strategy simply empties all patches from the <branch>,
 which makes little sense.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -X <strategy-option>::
 --strategy-option=<strategy-option>::
@@ -289,6 +297,8 @@ which makes little sense.
 	This implies `--merge` and, if no strategy has been
 	specified, `-s recursive`.  Note the reversal of 'ours' and
 	'theirs' as noted above for the `-m` option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -S[<keyid>]::
 --gpg-sign[=<keyid>]::
@@ -324,6 +334,8 @@ which makes little sense.
 	and after each change.  When fewer lines of surrounding
 	context exist they all must match.  By default no context is
 	ever ignored.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -f::
 --force-rebase::
@@ -355,19 +367,22 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 --whitespace=<option>::
 	These flag are passed to the 'git apply' program
 	(see linkgit:git-apply[1]) that applies the patch.
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --committer-date-is-author-date::
 --ignore-date::
 	These flags are passed to 'git am' to easily change the dates
 	of the rebased commits (see linkgit:git-am[1]).
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --signoff::
 	Add a Signed-off-by: trailer to all the rebased commits. Note
 	that if `--interactive` is given then only commits marked to be
-	picked, edited or reworded will have the trailer added. Incompatible
-	with the `--preserve-merges` option.
+	picked, edited or reworded will have the trailer added.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -i::
 --interactive::
@@ -378,6 +393,8 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -r::
 --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
@@ -404,7 +421,7 @@ It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
 explicit `exec git merge -s <strategy> [...]` commands.
 +
-See also REBASING MERGES below.
+See also REBASING MERGES and INCOMPATIBLE OPTIONS below.
 
 -p::
 --preserve-merges::
@@ -415,6 +432,8 @@ See also REBASING MERGES below.
 This uses the `--interactive` machinery internally, but combining it
 with the `--interactive` option explicitly is generally not a good
 idea unless you know what you are doing (see BUGS below).
++
+See also INCOMPATIBLE OPTIONS below.
 
 -x <cmd>::
 --exec <cmd>::
@@ -437,6 +456,8 @@ squash/fixup series.
 +
 This uses the `--interactive` machinery internally, but it can be run
 without an explicit `--interactive`.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --root::
 	Rebase all commits reachable from <branch>, instead of
@@ -447,6 +468,8 @@ without an explicit `--interactive`.
 	When used together with both --onto and --preserve-merges,
 	'all' root commits will be rewritten to have <newbase> as parent
 	instead.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autosquash::
 --no-autosquash::
@@ -461,11 +484,11 @@ without an explicit `--interactive`.
 	too.  The recommended way to create fixup/squash commits is by using
 	the `--fixup`/`--squash` options of linkgit:git-commit[1].
 +
-This option is only valid when the `--interactive` option is used.
-+
 If the `--autosquash` option is enabled by default using the
 configuration variable `rebase.autoSquash`, this option can be
 used to override and disable this setting.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autostash::
 --no-autostash::
@@ -487,6 +510,51 @@ recreates the topic branch with fresh commits so it can be remerged
 successfully without needing to "revert the reversion" (see the
 link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
 
+INCOMPATIBLE OPTIONS
+--------------------
+
+git-rebase has many flags that are incompatible with each other,
+predominantly due to the fact that it has three different underlying
+implementations:
+
+ * one based on linkgit:git-am[1] (the default)
+ * one based on linkgit:git-merge-recursive[1] (merge backend)
+ * one based on linkgit:git-cherry-pick[1] (interactive backend)
+
+Flags only understood by the am backend:
+
+ * --committer-date-is-author-date
+ * --ignore-date
+ * --whitespace
+ * --ignore-whitespace
+ * -C
+
+Flags understood by both merge and interactive backends:
+
+ * --merge
+ * --strategy
+ * --strategy-option
+ * --allow-empty-message
+
+Flags only understood by the interactive backend:
+
+ * --[no-]autosquash
+ * --rebase-merges
+ * --preserve-merges
+ * --interactive
+ * --exec
+ * --keep-empty
+ * --autosquash
+ * --edit-todo
+ * --root + --onto
+
+Other incompatible flag pairs:
+
+ * --preserve-merges && --interactive
+ * --preserve-merges && --signoff
+ * --preserve-merges && --rebase-merges
+ * --rebase-merges && --strategy
+
 include::merge-strategies.txt[]
 
 NOTES
-- 
2.18.0.rc2.1.g5453d3f70b.dirty

[RFC PATCH v2 2/7] git-rebase.sh: update help messages a bit

[Elijah Newren]

signoff is not specific to the am-backend.  Also, re-order a few options
to make like things (e.g. strategy and strategy-option) be near each
other.
---
 git-rebase.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 40be59ecc4..bf71b7fa20 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -20,23 +20,23 @@ onto=!             rebase onto given branch instead of upstream
 r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
+X,strategy-option=! pass the argument through to the merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
+f,force-rebase!    cherry-pick all commits, even if unchanged
 m,merge!           use merging strategies to rebase
 i,interactive!     let the user edit the list of commits to rebase
 x,exec=!           add exec lines after each commit of the editable list
 k,keep-empty	   preserve empty commits during rebase
 allow-empty-message allow rebasing commits with empty messages
-f,force-rebase!    force rebase even if branch is up to date
-X,strategy-option=! pass the argument through to the merge strategy
 stat!              display a diffstat of what changed upstream
 n,no-stat!         do not show diffstat of what changed upstream
 verify             allow pre-rebase hook to run
 rerere-autoupdate  allow rerere to update index with resolved conflicts
 root!              rebase all reachable commits up to the root(s)
 autosquash         move commits that begin with squash!/fixup! under -i
+signoff            add a Signed-off-by: line to each commit
 committer-date-is-author-date! passed to 'git am'
 ignore-date!       passed to 'git am'
-signoff            passed to 'git am'
 whitespace=!       passed to 'git apply'
 ignore-whitespace! passed to 'git apply'
 C=!                passed to 'git apply'
-- 
2.18.0.rc2.1.g5453d3f70b.dirty

[RFC PATCH v2 3/7] t3422: new testcases for checking when incompatible options passed

[Elijah Newren]

git rebase is split into three types: am, merge, and interactive.  Various
options imply different types, and which mode we are using determine which
sub-script (git-rebase--$type) is executed to finish the work.  Not all
options work with all types, so add tests for combinations where we expect
to receive an error rather than having options be silently ignored.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3422-rebase-incompatible-options.sh | 69 ++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100755 t/t3422-rebase-incompatible-options.sh

diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
new file mode 100755
index 0000000000..04cdae921b
--- /dev/null
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -0,0 +1,69 @@
+#!/bin/sh
+
+test_description='test if rebase detects and aborts on incompatible options'
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	test_seq 2 9 >foo &&
+	git add foo &&
+	git commit -m orig &&
+
+	git branch A &&
+	git branch B &&
+
+	git checkout A &&
+	test_seq 1 9 >foo &&
+	git add foo &&
+	git commit -m A &&
+
+	git checkout B &&
+	# This is indented with HT SP HT.
+	echo "	 	foo();" >>foo &&
+	git add foo &&
+	git commit -m B
+'
+
+#
+# Rebase has lots of useful options like --whitepsace=fix, which are
+# actually all built in terms of flags to git-am.  Since neither
+# --merge nor --interactive (nor any options that imply those two) use
+# git-am, using them together will result in flags like --whitespace=fix
+# being ignored.  Make sure rebase warns the user and aborts instead.
+#
+
+test_run_rebase () {
+	opt=$1
+	shift
+	test_expect_failure "$opt incompatible with --merge" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --merge A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --interactive" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --interactive A
+	"
+
+	test_expect_failure "$opt incompatible with --exec" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --exec 'true' A
+	"
+
+}
+
+test_run_rebase --whitespace=fix
+test_run_rebase --ignore-whitespace
+test_run_rebase --committer-date-is-author-date
+test_run_rebase -C4
+
+test_done
-- 
2.18.0.rc2.1.g5453d3f70b.dirty

[RFC PATCH v2 4/7] git-rebase: error out when incompatible options passed

[Elijah Newren]

git rebase has three different types: am, merge, and interactive, all of
which are implemented in terms of separate scripts.  am builds on git-am,
merge builds on git-merge-recursive, and interactive builds on
git-cherry-pick.  We make use of features in those lower-level commands in
the different rebase types, but those features don't exist in all of the
lower level commands so we have a range of incompatibilities.  Previously,
we just accepted nearly any argument and silently ignored whichever ones
weren't implemented for the type of rebase specified.  Change this so the
incompatibilities are documented, included in the testsuite, and tested
for at runtime with an appropriate error message shown.

Some exceptions I left out:

  * --merge and --interactive are technically incompatible since they are
    supposed to run different underlying scripts, but with a few small
    changes, --interactive can do everything that --merge can.  In fact,
    I'll shortly be sending another patch to remove git-rebase--merge and
    reimplement it on top of git-rebase--interactive.

  * One could argue that --interactive and --quiet are incompatible since
    --interactive doesn't implement a --quiet mode (perhaps since
    cherry-pick itself does not implement one).  However, the interactive
    mode is more quiet than the other modes in general with progress
    messages, so one could argue that it's already quiet.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh                          | 17 +++++++++++++++++
 t/t3422-rebase-incompatible-options.sh | 10 +++++-----
 2 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index bf71b7fa20..5f891214fa 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -503,6 +503,23 @@ then
 	git_format_patch_opt="$git_format_patch_opt --progress"
 fi
 
+if test -n "$git_am_opt"; then
+	incompatible_opts=$(echo "$git_am_opt" | sed -e 's/ -q//')
+	if test -n "$interactive_rebase"
+	then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
+		fi
+	fi
+	if test -n "$do_merge"; then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
+		fi
+	fi
+fi
+
 if test -n "$signoff"
 then
 	test -n "$preserve_merges" &&
diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
index 04cdae921b..66a83363bf 100755
--- a/t/t3422-rebase-incompatible-options.sh
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -34,27 +34,27 @@ test_expect_success 'setup' '
 test_run_rebase () {
 	opt=$1
 	shift
-	test_expect_failure "$opt incompatible with --merge" "
+	test_expect_success "$opt incompatible with --merge" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --merge A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy=ours" "
+	test_expect_success "$opt incompatible with --strategy=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+	test_expect_success "$opt incompatible with --strategy-option=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --interactive" "
+	test_expect_success "$opt incompatible with --interactive" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --interactive A
 	"
 
-	test_expect_failure "$opt incompatible with --exec" "
+	test_expect_success "$opt incompatible with --exec" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --exec 'true' A
 	"
-- 
2.18.0.rc2.1.g5453d3f70b.dirty

[RFC PATCH v2 5/7] git-rebase.txt: document behavioral inconsistencies between modes

[Elijah Newren]

There are a variety of aspects that are common to all rebases regardless
of which backend is in use; however, the behavior for these different
aspects varies in ways that could surprise users.  (In fact, it's not
clear -- to me at least -- that these differences were even desirable or
intentional.)  Document these differences.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 57 ++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index adccc15284..0dbfab06d0 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -555,6 +555,63 @@ Other incompatible flag pairs:
  * --preserve-merges && --rebase-merges
  * --rebase-merges && --strategy
 
+BEHAVIORAL INCONSISTENCIES
+--------------------------
+
+  * --no-ff vs. --force-rebase
+
+    These options are actually identical, though their description
+    leads people to believe they might not be.
+
+ * empty commits:
+
+    am-based rebase will drop any "empty" commits, whether the
+    commit started empty (had no changes relative to its parent to
+    start with) or ended empty (all changes were already applied
+    upstream in other commits).
+
+    merge-based rebase does the same.
+
+    interactive-based rebase will by default drop commits that
+    started empty and halt if it hits a commit that ended up empty.
+    The --keep-empty option exists for interactive rebases to allow
+    it to keep commits that started empty.
+
+  * empty commit messages:
+
+    am-based rebase will silently apply commits with empty commit
+    messages.
+
+    merge-based and interactive-based rebases will by default halt
+    on any such commits.  The --allow-empty-message option exists to
+    allow interactive-based rebases to apply such commits without
+    halting.
+
+  * directory rename detection:
+
+    merge-based and interactive-based rebases work fine with
+    directory rename detection.  am-based rebases sometimes do not.
+
+    git-am tries to avoid a full three way merge, instead calling
+    git-apply.  That prevents us from detecting renames at all,
+    which may defeat the directory rename detection.  There is a
+    fallback, though; if the initial git-apply fails and the user
+    has specified the -3 option, git-am will fall back to a three
+    way merge.  However, git-am lacks the necessary information to
+    do a "real" three way merge.  Instead, it has to use
+    build_fake_ancestor() to get a merge base that is missing files
+    whose rename may have been important to detect for directory
+    rename detection to function.
+
+    Since am-based rebases work by first generating a bunch of
+    patches (which no longer record what the original commits were
+    and thus don't have the necessary info from which we can find a
+    real merge-base), and then calling git-am, this implies that
+    am-based rebases will not always successfully detect directory
+    renames either.  merged-based rebases (rebase -m) and
+    cherry-pick-based rebases (rebase -i) are not affected by this
+    shortcoming.
+
 include::merge-strategies.txt[]
 
 NOTES
-- 
2.18.0.rc2.1.g5453d3f70b.dirty

[RFC PATCH v2 6/7] git-rebase.txt: address confusion between --no-ff vs --force-rebase

[Elijah Newren]

rebase was taught the --force-rebase option in commit b2f82e05de ("Teach
rebase to rebase even if upstream is up to date", 2009-02-13).  This flag
worked for the am and merge backends, but wasn't a valid option for the
interactive backend.

rebase was taught the --no-ff option for interactive rebases in commit
b499549401cb ("Teach rebase the --no-ff option.", 2010-03-24), to do the
exact same thing as --force-rebase does for non-interactive rebases.  This
commit explicitly documented the fact that --force-rebase was incompatible
with --interactive, though it made --no-ff a synonym for --force-rebase
for non-interactive rebases.  The choice of a new option was based on the
fact that "force rebase" didn't sound like an appropriate term for the
interactive machinery.

In commit 6bb4e485cff8 ("rebase: align variable names", 2011-02-06), the
separate parsing of command line options in the different rebase scripts
was removed, and whether on accident or because the author noticed that
these options did the same thing, the options became synonyms and both
were accepted by all three rebase types.

In commit 2d26d533a012 ("Documentation/git-rebase.txt: -f forces a rebase
that would otherwise be a no-op", 2014-08-12), which reworded the
description of the --force-rebase option, the (no-longer correct) sentence
stating that --force-rebase was incompatible with --interactive was
finally removed.

Finally, as explained at
https://public-inbox.org/git/98279912-0f52-969d-44a6-22242039387f@xiplink.com

    In the original discussion around this option [1], at one point I
    proposed teaching rebase--interactive to respect --force-rebase
    instead of adding a new option [2].  Ultimately --no-ff was chosen as
    the better user interface design [3], because an interactive rebase
    can't be "forced" to run.

We have accepted both --no-ff and --force-rebase as full synonyms for all
three rebase types for over seven years.  Documenting them differently
and in ways that suggest they might not be quite synonyms simply leads to
confusion.  Adjust the documentation to match reality.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 35 ++++++++++-------------------------
 1 file changed, 10 insertions(+), 25 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 0dbfab06d0..7a2ed9efdc 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -337,16 +337,18 @@ See also INCOMPATIBLE OPTIONS below.
 +
 See also INCOMPATIBLE OPTIONS below.
 
--f::
+--no-ff::
 --force-rebase::
-	Force a rebase even if the current branch is up to date and
-	the command without `--force` would return without doing anything.
+-f::
+	Individually replay all rebased commits instead of fast-forwarding
+	over the unchanged ones.  This ensures that the entire history of
+	the rebased branch is composed of new commits.
 +
-You may find this (or --no-ff with an interactive rebase) helpful after
-reverting a topic branch merge, as this option recreates the topic branch with
-fresh commits so it can be remerged successfully without needing to "revert
-the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
+You may find this helpful after reverting a topic branch merge, as this option
+recreates the topic branch with fresh commits so it can be remerged
+successfully without needing to "revert the reversion" (see the
+link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for
+details).
 
 --fork-point::
 --no-fork-point::
@@ -498,18 +500,6 @@ See also INCOMPATIBLE OPTIONS below.
 	with care: the final stash application after a successful
 	rebase might result in non-trivial conflicts.
 
---no-ff::
-	With --interactive, cherry-pick all rebased commits instead of
-	fast-forwarding over the unchanged ones.  This ensures that the
-	entire history of the rebased branch is composed of new commits.
-+
-Without --interactive, this is a synonym for --force-rebase.
-+
-You may find this helpful after reverting a topic branch merge, as this option
-recreates the topic branch with fresh commits so it can be remerged
-successfully without needing to "revert the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
-
 INCOMPATIBLE OPTIONS
 --------------------
 
@@ -558,11 +548,6 @@ Other incompatible flag pairs:
 BEHAVIORAL INCONSISTENCIES
 --------------------------
 
-  * --no-ff vs. --force-rebase
-
-    These options are actually identical, though their description
-    leads people to believe they might not be.
-
  * empty commits:
 
     am-based rebase will drop any "empty" commits, whether the
-- 
2.18.0.rc2.1.g5453d3f70b.dirty

[RFC PATCH v2 7/7] git-rebase: make --allow-empty-message the default

[Elijah Newren]

am-based rebases already apply commits with an empty commit message
without requiring the user to specify an extra flag.  Make merge-based and
interactive-based rebases behave the same.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt  | 10 ----------
 git-rebase.sh                 |  2 +-
 t/t3404-rebase-interactive.sh |  7 ++++---
 t/t3405-rebase-malformed.sh   | 11 +++--------
 4 files changed, 8 insertions(+), 22 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 7a2ed9efdc..a5608f481f 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -562,16 +562,6 @@ BEHAVIORAL INCONSISTENCIES
     The --keep-empty option exists for interactive rebases to allow
     it to keep commits that started empty.
 
-  * empty commit messages:
-
-    am-based rebase will silently apply commits with empty commit
-    messages.
-
-    merge-based and interactive-based rebases will by default halt
-    on any such commits.  The --allow-empty-message option exists to
-    allow interactive-based rebases to apply such commits without
-    halting.
-
   * directory rename detection:
 
     merge-based and interactive-based rebases work fine with
diff --git a/git-rebase.sh b/git-rebase.sh
index 5f891214fa..bf033da4e5 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -95,7 +95,7 @@ rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
-allow_empty_message=
+allow_empty_message=--allow-empty-message
 signoff=
 test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
 case "$(git config --bool commit.gpgsign)" in
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index c65826ddac..f84fa63b15 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -553,15 +553,16 @@ test_expect_success '--continue tries to commit, even for "edit"' '
 '
 
 test_expect_success 'aborted --continue does not squash commits after "edit"' '
+	test_when_finished "git rebase --abort" &&
 	old=$(git rev-parse HEAD) &&
 	test_tick &&
 	set_fake_editor &&
 	FAKE_LINES="edit 1" git rebase -i HEAD^ &&
 	echo "edited again" > file7 &&
 	git add file7 &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " git rebase --continue &&
-	test $old = $(git rev-parse HEAD) &&
-	git rebase --abort
+	echo all the things >>conflict &&
+	test_must_fail git rebase --continue &&
+	test $old = $(git rev-parse HEAD)
 '
 
 test_expect_success 'auto-amend only edited commits after "edit"' '
diff --git a/t/t3405-rebase-malformed.sh b/t/t3405-rebase-malformed.sh
index cb7c6de84a..da94dddc86 100755
--- a/t/t3405-rebase-malformed.sh
+++ b/t/t3405-rebase-malformed.sh
@@ -77,19 +77,14 @@ test_expect_success 'rebase commit with diff in message' '
 '
 
 test_expect_success 'rebase -m commit with empty message' '
-	test_must_fail git rebase -m master empty-message-merge &&
-	git rebase --abort &&
-	git rebase -m --allow-empty-message master empty-message-merge
+	git rebase -m master empty-message-merge
 '
 
 test_expect_success 'rebase -i commit with empty message' '
 	git checkout diff-in-message &&
 	set_fake_editor &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i HEAD^ &&
-	git rebase --abort &&
-	FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i --allow-empty-message HEAD^
+	env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
+		git rebase -i HEAD^
 '
 
 test_done
-- 
2.18.0.rc2.1.g5453d3f70b.dirty

Re: [RFC PATCH v2 7/7] git-rebase: make --allow-empty-message the default

[Phillip Wood]

Hi Elijah

On 17/06/18 06:58, Elijah Newren wrote:
> am-based rebases already apply commits with an empty commit message
> without requiring the user to specify an extra flag.  Make merge-based and
> interactive-based rebases behave the same.
> 
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>   Documentation/git-rebase.txt  | 10 ----------
>   git-rebase.sh                 |  2 +-
>   t/t3404-rebase-interactive.sh |  7 ++++---
>   t/t3405-rebase-malformed.sh   | 11 +++--------
>   4 files changed, 8 insertions(+), 22 deletions(-)
> 
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 7a2ed9efdc..a5608f481f 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -562,16 +562,6 @@ BEHAVIORAL INCONSISTENCIES
>       The --keep-empty option exists for interactive rebases to allow
>       it to keep commits that started empty.
>   
> -  * empty commit messages:
> -
> -    am-based rebase will silently apply commits with empty commit
> -    messages.
> -
> -    merge-based and interactive-based rebases will by default halt
> -    on any such commits.  The --allow-empty-message option exists to
> -    allow interactive-based rebases to apply such commits without
> -    halting.
> -
>     * directory rename detection:
>   
>       merge-based and interactive-based rebases work fine with
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 5f891214fa..bf033da4e5 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -95,7 +95,7 @@ rebase_cousins=
>   preserve_merges=
>   autosquash=
>   keep_empty=
> -allow_empty_message=
> +allow_empty_message=--allow-empty-message

Looking at the option parsing in git-rebase.sh it appears that it does 
not check for --no-allow-empty-message so there's no way to turn off the 
default for those modes that support it. I'm not sure what to think of 
this change, I'm slightly uneasy with changing to default to be 
different from cherry-pick, though one could argue rebasing is a 
different operation and just keeping the existing messages even if they 
are empty is more appropriate and having all the rebase modes do the 
default to the same behaviour is definitely an improvement.

Best Wishes

Phillip

>   signoff=
>   test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
>   case "$(git config --bool commit.gpgsign)" in
> diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
> index c65826ddac..f84fa63b15 100755
> --- a/t/t3404-rebase-interactive.sh
> +++ b/t/t3404-rebase-interactive.sh
> @@ -553,15 +553,16 @@ test_expect_success '--continue tries to commit, even for "edit"' '
>   '
>   
>   test_expect_success 'aborted --continue does not squash commits after "edit"' '
> +	test_when_finished "git rebase --abort" &&
>   	old=$(git rev-parse HEAD) &&
>   	test_tick &&
>   	set_fake_editor &&
>   	FAKE_LINES="edit 1" git rebase -i HEAD^ &&
>   	echo "edited again" > file7 &&
>   	git add file7 &&
> -	test_must_fail env FAKE_COMMIT_MESSAGE=" " git rebase --continue &&
> -	test $old = $(git rev-parse HEAD) &&
> -	git rebase --abort
> +	echo all the things >>conflict &&
> +	test_must_fail git rebase --continue &&
> +	test $old = $(git rev-parse HEAD)
>   '
>   
>   test_expect_success 'auto-amend only edited commits after "edit"' '
> diff --git a/t/t3405-rebase-malformed.sh b/t/t3405-rebase-malformed.sh
> index cb7c6de84a..da94dddc86 100755
> --- a/t/t3405-rebase-malformed.sh
> +++ b/t/t3405-rebase-malformed.sh
> @@ -77,19 +77,14 @@ test_expect_success 'rebase commit with diff in message' '
>   '
>   
>   test_expect_success 'rebase -m commit with empty message' '
> -	test_must_fail git rebase -m master empty-message-merge &&
> -	git rebase --abort &&
> -	git rebase -m --allow-empty-message master empty-message-merge
> +	git rebase -m master empty-message-merge
>   '
>   
>   test_expect_success 'rebase -i commit with empty message' '
>   	git checkout diff-in-message &&
>   	set_fake_editor &&
> -	test_must_fail env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
> -		git rebase -i HEAD^ &&
> -	git rebase --abort &&
> -	FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
> -		git rebase -i --allow-empty-message HEAD^
> +	env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
> +		git rebase -i HEAD^
>   '
>   
>   test_done
> 

Re: [RFC PATCH v2 1/7] git-rebase.txt: document incompatible options

[Phillip Wood]

Hi Elijah
On 17/06/18 06:58, Elijah Newren wrote:
> git rebase has many options that only work with one of its three backends.
> It also has a few other pairs of incompatible options.  Document these.
> 
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>   Documentation/git-rebase.txt | 84 ++++++++++++++++++++++++++++++++----
>   1 file changed, 76 insertions(+), 8 deletions(-)
> 
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 0e20a66e73..adccc15284 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -243,11 +243,15 @@ leave out at most one of A and B, in which case it defaults to HEAD.
>   --keep-empty::
>   	Keep the commits that do not change anything from its
>   	parents in the result.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   --allow-empty-message::
>   	By default, rebasing commits with an empty message will fail.
>   	This option overrides that behavior, allowing commits with empty
>   	messages to be rebased.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   --skip::
>   	Restart the rebasing process by skipping the current patch.
> @@ -271,6 +275,8 @@ branch on top of the <upstream> branch.  Because of this, when a merge
>   conflict happens, the side reported as 'ours' is the so-far rebased
>   series, starting with <upstream>, and 'theirs' is the working branch.  In
>   other words, the sides are swapped.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   -s <strategy>::
>   --strategy=<strategy>::
> @@ -280,8 +286,10 @@ other words, the sides are swapped.
>   +
>   Because 'git rebase' replays each commit from the working branch
>   on top of the <upstream> branch using the given strategy, using
> -the 'ours' strategy simply discards all patches from the <branch>,
> +the 'ours' strategy simply empties all patches from the <branch>,
>   which makes little sense.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   -X <strategy-option>::
>   --strategy-option=<strategy-option>::
> @@ -289,6 +297,8 @@ which makes little sense.
>   	This implies `--merge` and, if no strategy has been
>   	specified, `-s recursive`.  Note the reversal of 'ours' and
>   	'theirs' as noted above for the `-m` option.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   -S[<keyid>]::
>   --gpg-sign[=<keyid>]::
> @@ -324,6 +334,8 @@ which makes little sense.
>   	and after each change.  When fewer lines of surrounding
>   	context exist they all must match.  By default no context is
>   	ever ignored.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   -f::
>   --force-rebase::
> @@ -355,19 +367,22 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
>   --whitespace=<option>::
>   	These flag are passed to the 'git apply' program
>   	(see linkgit:git-apply[1]) that applies the patch.
> -	Incompatible with the --interactive option.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   --committer-date-is-author-date::
>   --ignore-date::
>   	These flags are passed to 'git am' to easily change the dates
>   	of the rebased commits (see linkgit:git-am[1]).
> -	Incompatible with the --interactive option.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   --signoff::
>   	Add a Signed-off-by: trailer to all the rebased commits. Note
>   	that if `--interactive` is given then only commits marked to be
> -	picked, edited or reworded will have the trailer added. Incompatible
> -	with the `--preserve-merges` option.
> +	picked, edited or reworded will have the trailer added.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   -i::
>   --interactive::
> @@ -378,6 +393,8 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
>   The commit list format can be changed by setting the configuration option
>   rebase.instructionFormat.  A customized instruction format will automatically
>   have the long commit hash prepended to the format.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   -r::
>   --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
> @@ -404,7 +421,7 @@ It is currently only possible to recreate the merge commits using the
>   `recursive` merge strategy; Different merge strategies can be used only via
>   explicit `exec git merge -s <strategy> [...]` commands.
>   +
> -See also REBASING MERGES below.
> +See also REBASING MERGES and INCOMPATIBLE OPTIONS below.
>   
>   -p::
>   --preserve-merges::
> @@ -415,6 +432,8 @@ See also REBASING MERGES below.
>   This uses the `--interactive` machinery internally, but combining it
>   with the `--interactive` option explicitly is generally not a good
>   idea unless you know what you are doing (see BUGS below).
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   -x <cmd>::
>   --exec <cmd>::
> @@ -437,6 +456,8 @@ squash/fixup series.
>   +
>   This uses the `--interactive` machinery internally, but it can be run
>   without an explicit `--interactive`.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   --root::
>   	Rebase all commits reachable from <branch>, instead of
> @@ -447,6 +468,8 @@ without an explicit `--interactive`.
>   	When used together with both --onto and --preserve-merges,
>   	'all' root commits will be rewritten to have <newbase> as parent
>   	instead.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   --autosquash::
>   --no-autosquash::
> @@ -461,11 +484,11 @@ without an explicit `--interactive`.
>   	too.  The recommended way to create fixup/squash commits is by using
>   	the `--fixup`/`--squash` options of linkgit:git-commit[1].
>   +
> -This option is only valid when the `--interactive` option is used.
> -+
>   If the `--autosquash` option is enabled by default using the
>   configuration variable `rebase.autoSquash`, this option can be
>   used to override and disable this setting.
> ++
> +See also INCOMPATIBLE OPTIONS below.
>   
>   --autostash::
>   --no-autostash::
> @@ -487,6 +510,51 @@ recreates the topic branch with fresh commits so it can be remerged
>   successfully without needing to "revert the reversion" (see the
>   link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
>   
> +INCOMPATIBLE OPTIONS
> +--------------------
> +
> +git-rebase has many flags that are incompatible with each other,
> +predominantly due to the fact that it has three different underlying
> +implementations:
> +
> + * one based on linkgit:git-am[1] (the default)
> + * one based on linkgit:git-merge-recursive[1] (merge backend)
> + * one based on linkgit:git-cherry-pick[1] (interactive backend)
> +
> +Flags only understood by the am backend:
> +
> + * --committer-date-is-author-date
> + * --ignore-date
> + * --whitespace
> + * --ignore-whitespace
> + * -C
> +
> +Flags understood by both merge and interactive backends:
> +
> + * --merge
> + * --strategy
> + * --strategy-option
> + * --allow-empty-message
> +
> +Flags only understood by the interactive backend:
> +
> + * --[no-]autosquash
> + * --rebase-merges
> + * --preserve-merges
> + * --interactive
> + * --exec
> + * --keep-empty
> + * --autosquash
> + * --edit-todo
> + * --root + --onto
> +
> +Other incompatible flag pairs:
> +
> + * --preserve-merges && --interactive
> + * --preserve-merges && --signoff
> + * --preserve-merges && --rebase-merges
> + * --rebase-merges && --strategy

Does --rebase-merges support --strategy-options?

> +
>   include::merge-strategies.txt[]
>   
>   NOTES
> 
This is a great step forward. In the long run it would be nice not to 
have to mention the backends. Once git-rebase--merge.sh is gone, the 
situation is simpler and it would be nice to reword this to say 
something like
   The --committer-date-is-author-date, --ignore-date, --whitespace,
   --ignore-whitespace and -C options are incompatible with the following
   --exec ... Also note that --rebase-merges is incompatible with
   --strategy; and --preserve-merges is incompatible with --interactive,
   --signoff, --rebase-merges.
rather than talking about am options etc.

Best Wishes

Phillip

Re: [RFC PATCH v2 0/7] Document/fix/warn about rebase incompatibilities and inconsistences

[Phillip Wood]

Hi Elijah

On 17/06/18 06:58, Elijah Newren wrote:
> git-rebase has lots of options that are mutually incompatible.  Even among
> aspects of its behavior that is common to all rebase types, it has a number
> of inconsistencies.  This series tries to document, fix, and/or warn users
> about many of these.
> 
> I have a much higher than average expectation that folks will object
> to some of these patches.  I've tried to divide them up so that any parts
> we decide to drop or redo can be more easily excised.
> 
> No branch-diff; because it's a significant re-work; instead I'll comment
> briefly on the individual patches...

I found this series well structured and easy to follow. I've commented 
on a couple of the patches, the others seemed fine to me. It's great to 
see these inconsistencies being documented and some being eliminated.

Best Wishes

Phillip

> 
> Elijah Newren (7):
>    git-rebase.txt: document incompatible options
> 
> Both Dscho (on a related patch series) and Phillip suggested changing the
> documentation to avoid implementational details.  I instead made a separate
> section with sets of incompatible options...but it still mentions the
> different backends while doing so.  Does that seem alright?
> 
>    git-rebase.sh: update help messages a bit
> 
> Minor tweaks to `git rebase -h` output.
> 
>    t3422: new testcases for checking when incompatible options passed
> 
> The one unmodified patch from the first round.
> 
>    git-rebase: error out when incompatible options passed
> 
> Almost the same as the first round, except:
>    * Documentation pulled into a separate patch (patch 1)
>    * $() instead of ``
> 
>    git-rebase.txt: document behavioral inconsistencies between modes
> 
> Add another section to the documentation for aspects that ideally
> should be common between all modes but are handled differently.
> 
>    git-rebase.txt: address confusion between --no-ff vs --force-rebase
> 
> This came up on the list not that long ago; fix the documentation.
> 
>    git-rebase: make --allow-empty-message the default
> 
> Address the easiest of the inconsistencies, assuming the am-based backend
> has the correct default and the merge-based and interactive-based backends
> are the ones that need to change.
> 
>   Documentation/git-rebase.txt           | 154 ++++++++++++++++++++-----
>   git-rebase.sh                          |  25 +++-
>   t/t3404-rebase-interactive.sh          |   7 +-
>   t/t3405-rebase-malformed.sh            |  11 +-
>   t/t3422-rebase-incompatible-options.sh |  69 +++++++++++
>   5 files changed, 224 insertions(+), 42 deletions(-)
>   create mode 100755 t/t3422-rebase-incompatible-options.sh
> 

Re: [RFC PATCH v2 1/7] git-rebase.txt: document incompatible options

[Eric Sunshine]

On Sun, Jun 17, 2018 at 1:59 AM Elijah Newren <newren@gmail.com> wrote:
> git rebase has many options that only work with one of its three backends.
> It also has a few other pairs of incompatible options.  Document these.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> @@ -487,6 +510,51 @@ recreates the topic branch with fresh commits so it can be remerged
> +Flags only understood by the interactive backend:
> + [...]
> + * --edit-todo
> + * --root + --onto

What does "+" mean? Is it "and"?

> +Other incompatible flag pairs:
> +
> + * --preserve-merges && --interactive
> + * --preserve-merges && --signoff
> + * --preserve-merges && --rebase-merges
> + * --rebase-merges && --strategy

It's somewhat odd seeing "programmer" notation in end-user
documentation. Perhaps replace "&&" with "and"?

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Johannes Schindelin]

Hi Elijah,

On Sat, 9 Jun 2018, Elijah Newren wrote:

> On Sat, Jun 9, 2018 at 3:11 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > On Thu, 7 Jun 2018, Elijah Newren wrote:
> >
> >> am-based rebases suffer from an reduced ability to detect directory
> >> renames upstream, which is fundamental to the fact that it throws
> >> away information about the history: in particular, it dispenses with
> >> the original commits involved by turning them into patches, and
> >> without the knowledge of the original commits we cannot determine a
> >> proper merge base.
> >>
> >> The am-based rebase will proceed by first trying git-apply, and, only
> >> if it fails, will try to reconstruct a provisional merge base using
> >> build_fake_ancestor().  This fake ancestor will ONLY contain the
> >> files modified in the patch; without the full list of files in the
> >> real merge base, renames of any missing files cannot be detected.
> >> Directory rename detection works by looking at individual file
> >> renames and deducing when a full directory has been renamed.
> >>
> >> Trying to modify build_fake_ancestor() to instead create a merge base
> >> that includes common file information by looking for a commit that
> >> contained all the same blobs as the pre-image of the patch would
> >> require a very expensive search through history, may find the wrong
> >> commit, and outside of rebase may not find any commit that matches.
> >>
> >> But we had all the relevant information to start.  So, instead of
> >> attempting to fix am which just doesn't have the relevant
> >> information, instead note its strength and weaknesses, and change the
> >> default rebase machinery to interactive since it does not suffer from
> >> the same problems.
> >
> > I'll let Eric comment on the grammar, and I'll comment on the idea
> > behind this commit instead.
> 
> Going to dump the hard job on Eric, eh?  ;-)

Just trying to learn how to delegate.

:0)

> > IMHO `git rebase -i` is still too slow to be a true replacement for
> > `git rebase --am` for the cases where it serves the user well. Maybe
> > we should work on making `rebase -i` faster, first?
> 
> That sounds fair.
> 
> > I imagine, for example, that it might make *tons* of sense to avoid
> > writing out the index and worktree files all the time. That was
> > necessary in the shell script version because if the ridiculous
> > limitations we subjected ourselves to, such as: no object-oriented
> > state worth mentioning, only string-based processing, etc. But we
> > could now start to do everything in memory (*maybe* write out the new
> > blob/tree/commit objects immediately, but maybe not) until the time
> > when we either have succeeded in the rebase, or when there was a
> > problem and we have to exit with an error. And only then write the
> > files and the index.
> 
> Hmm...are you still planning on using cherry-pick (internally rather
> than forking, of course)?

The sequencer side-steps the cherry-pick command by calling
merge_recursive() directly. But yes, this is what the interactive rebase
will always use.

>  Because cherry-pick uses the merge-recursive machinery, and the
>  merge-recursive machinery doesn't have a nice way of avoiding writing
>  to the working tree or index.

Exactly. I think it could be taught to perform its magic in memory. I am
not saying that it is necessarily easy to teach merge-recursive.c this
trick, but it should be possible.

> Fixing that is on my radar; see the first block of
> https://public-inbox.org/git/CABPp-BG2fZHm3s-yrzxyGj3Eh+O7_LHLz5pgstHhG2drigSyRA@mail.gmail.com/

Great!

> (reading up until "At this point, I'd rather just fix the design flaw
> rather than complicate the code further.")
> 
> However, also covered in my plans is a few things to speed up the
> merge-recursive machinery, which should provide some other performance
> benefits for interactive rebases.

I am looking forward to see this materialize.

> > In any case, I think that the rather noticeable change of the default
> > would probably necessitate a deprecation phase.
> 
> Why is it a "rather noticable change of the default"?

I was really referring to speed. But I have to admit that I do not have
any current numbers.

Another issue just hit me, though: rebase --am does not need to look at as
many Git objects as rebase --merge or rebase -i. Therefore, GVFS users
will still want to use --am wherever possible, to avoid "hydrating"
many objects during their rebase.

> If we popped up the editor and asked the user to edit the list of
> options, I'd agree, or if folks thought that it was significantly slower
> by a big enough margin (though you already suggested waiting and making
> sure we don't do that).  What else remains that qualifies?
> 
> (Okay, the default behavior to just skip empty patches rather than halt
> the rebase and ask the user to advise is different, but we could fix
> that up too.  Is there anything else?)

That is indeed a change in behavior that is rather easy to address.

As to speed: that might be harder. But then, the performance might already
be good enough. I do not have numbers (nor the time to generate them) to
back up my hunch that --am is substantially faster than --merge.

Ciao,
Dscho

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Elijah Newren]

Hi Dscho,

On Sun, Jun 17, 2018 at 2:44 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:

> I was really referring to speed. But I have to admit that I do not have
> any current numbers.
>
> Another issue just hit me, though: rebase --am does not need to look at as
> many Git objects as rebase --merge or rebase -i. Therefore, GVFS users
> will still want to use --am wherever possible, to avoid "hydrating"
> many objects during their rebase.

What is it that makes rebase --am need fewer Git objects than rebase
--merge or rebase -i?  I have one idea which isn't intrinsic to the
algorithm, so I'm curious if there's something else I'm unaware of.

My guess at what objects are needed by each type:

At a high level, rebase --am for each commit will need to compare the
commit to its parent to generate a diff (which thus involves walking
over the objects in both the commit and its parent, though it should
be able to skip over subtrees that are equal), and then will need to
look at all the objects in the target commit on which it needs to
apply the patch (in order to properly fill the index for a starting
point, and used later when creating a new commit).  If the application
of the diff fails, it falls back to a three-way merge, though the
three-way merge shouldn't need any additional objects.  So, to
summarize, rebase--am needs objects from the commit being rebased, its
parent, and the target commit onto which it is applying, though it can
short circuit some objects when the commit and its parent have
matching subtree(s).

rebase -i, if I understand correctly, does a three-way merge between
the commit, its parent, and the target commit.  Thus, we again walk
over objects in those three commits; I think unpack_trees() does not
take advantage of matching trees to avoid descending into subtrees,
but if so that's an optimization that we may be able to implement
(though it would require diving into unpack_trees() code, which is
never easy...).

(Side notes: (1) rebase --merge is basically the same as rebase -i
here; it's path to reaching the recursive merge machinery is a bit
different but the resulting arguments are the same; (2) a real merge
between branches would require more objects because it would have to
do some revision walking to find a merge base, and a real merge base
is likely to differ more than just the parent commit.  But finding
merge bases isn't relevant to rebase -m or rebase -i)

Is there something else I'm missing that fundamentally makes rebase -i
need more objects?

> As to speed: that might be harder. But then, the performance might already
> be good enough. I do not have numbers (nor the time to generate them) to
> back up my hunch that --am is substantially faster than --merge.

I too have a hunch that --am is faster than --merge, on big enough
repos or repos with enough renames.  I can partially back it up with
an indirect number: at [1], it was reported that cherry-picks could be
sped up by a factor of 20-30 on some repos with lots of renames.  I
believe there are other performance improvements possible too, for the
--merge or -i cases.

I'm also curious now whether your comment on hydrating objects might
uncover additional areas where performance improvements could be made
for non-am-based rebases of large-enough repos.

Elijah

[1] https://public-inbox.org/git/CABPp-BH4LLzeJjE5cvwWQJ8xTj3m9oC-41Tu8BM8c7R0gQTjWw@mail.gmail.com/
(see also Peter's last reply in that thread, and compare to his first
post)

Re: [RFC PATCH v2 7/7] git-rebase: make --allow-empty-message the default

[Elijah Newren]

Hi Phillip,

On Sun, Jun 17, 2018 at 8:30 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:

> Looking at the option parsing in git-rebase.sh it appears that it does not
> check for --no-allow-empty-message so there's no way to turn off the default
> for those modes that support it. I'm not sure what to think of this change,
> I'm slightly uneasy with changing to default to be different from
> cherry-pick, though one could argue rebasing is a different operation and
> just keeping the existing messages even if they are empty is more
> appropriate and having all the rebase modes do the default to the same
> behaviour is definitely an improvement.
>

Here's my extended rationale for why to change the default for rebase
-i/-m (and _maybe_ also for cherry-pick):

First, commits with an empty commit message are fairly rare, so it's
hard to gather that much data on; we're dealing with out-liers.  That
also means that folks who have touched the area in the past probably
weren't dealing with full information either; they just tweaked what
was already there.

It makes sense that git-commit would default to not allowing empty
commit messages without a flag.  We'd like to warn against that kind
of poor practice.  However, that implicitly means that commands which
build on top of git-commit might copy that default even when it
doesn't make sense.  So it's not at all clear to me that cherry-pick
or rebase -i/-m should have the same default.  They do, but that may
have been entirely accidental.  And when someone comes along and
notices the problematic default of stopping with empty commit
messages, they assume it was intended and add a flag to change it, as
happened for both commands.

On the other hand git-am was designed from the workflow of applying
many patches from the beginning, and has a different default.
am-based rebases copy that default of silently applying commits with
empty commit messages; it may not be clear that rebase should copy the
default from am, but it does.  Yet, despite the fact that ignoring
empty commit messages is the default backend for rebases, and thus is
likely used more than the other rebase backends, no one has ever
submitted a patch to add a flag to rebase to get the opposite
behavior.  To me, this argues pretty strongly that not only is
ignoring empty messages the right default, that it's a waste of time
to implement a flag (e.g. --no-allow-empty-message) providing the
opposite behavior for rebases.

It is obvious that one of the two rebase backends have the wrong
default, because they should have the same default as each other.  I
am inclined to believe, based on the above logic, that am-based rebase
has the correct default.  I also suspect that cherry-pick has the
wrong default, though it's not as cut-and-dry since it doesn't have
multiple backends with conflicting defaults.


Does that make sense?  Am I overlooking anything important?

Re: [RFC PATCH v2 1/7] git-rebase.txt: document incompatible options

[Elijah Newren]

On Sun, Jun 17, 2018 at 8:38 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
>> +Other incompatible flag pairs:
>> +
>> + * --preserve-merges && --interactive
>> + * --preserve-merges && --signoff
>> + * --preserve-merges && --rebase-merges
>> + * --rebase-merges && --strategy
>
> Does --rebase-merges support --strategy-options?

Good catch.  I hadn't yet tested --rebase-merges or --preserve-merges
myself, so I was just going off notes in the documentation or code for
those ones.  I created a simple case needing -Xignore-space-change to
merge cleanly, and found that any --strategy-option argument will be
silently ignored by --rebase-merges currently.  So, we should also
document and warn the user that those options are currently
incompatible.

>> +
>>   include::merge-strategies.txt[]
>>     NOTES
>>
> This is a great step forward. In the long run it would be nice not to have
> to mention the backends. Once git-rebase--merge.sh is gone, the situation is
> simpler and it would be nice to reword this to say something like
>   The --committer-date-is-author-date, --ignore-date, --whitespace,
>   --ignore-whitespace and -C options are incompatible with the following
>   --exec ... Also note that --rebase-merges is incompatible with
>   --strategy; and --preserve-merges is incompatible with --interactive,
>   --signoff, --rebase-merges.
> rather than talking about am options etc.

That sounds like it would be simpler for the incompatible options, but
what about discussing inconsistent behavior between different backends
(empty commit handling, empty commit message handling, and directory
rename detection)?  Do we first need to remove all those
inconsistencies before rewording the documentation here, or is there a
clever way to discuss those?

Re: [RFC PATCH v2 1/7] git-rebase.txt: document incompatible options

[Elijah Newren]

Hi Eric,

On Sun, Jun 17, 2018 at 10:15 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Sun, Jun 17, 2018 at 1:59 AM Elijah Newren <newren@gmail.com> wrote:
>> git rebase has many options that only work with one of its three backends.
>> It also has a few other pairs of incompatible options.  Document these.
>>
>> Signed-off-by: Elijah Newren <newren@gmail.com>
>> ---
>> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
>> @@ -487,6 +510,51 @@ recreates the topic branch with fresh commits so it can be remerged
>> +Flags only understood by the interactive backend:
>> + [...]
>> + * --edit-todo
>> + * --root + --onto
>
> What does "+" mean? Is it "and"?

Yes, it means "and", though perhaps it'd be even clearer to write "in
combination with" or "when used in combination with"

>> +Other incompatible flag pairs:
>> +
>> + * --preserve-merges && --interactive
>> + * --preserve-merges && --signoff
>> + * --preserve-merges && --rebase-merges
>> + * --rebase-merges && --strategy
>
> It's somewhat odd seeing "programmer" notation in end-user
> documentation. Perhaps replace "&&" with "and"?

Yes, good point.  I'll make that fix.

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Johannes Schindelin]

Hi Elijah,

On Wed, 20 Jun 2018, Elijah Newren wrote:

> On Sun, Jun 17, 2018 at 2:44 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> 
> > I was really referring to speed. But I have to admit that I do not have
> > any current numbers.
> >
> > Another issue just hit me, though: rebase --am does not need to look at as
> > many Git objects as rebase --merge or rebase -i. Therefore, GVFS users
> > will still want to use --am wherever possible, to avoid "hydrating"
> > many objects during their rebase.
> 
> What is it that makes rebase --am need fewer Git objects than rebase
> --merge or rebase -i?

Multiple things. The most obvious thing: --cherry-pick. When you call `git
rebase --am`, it does not try to exclude the patches by looking at
`rev-list --cherry-pick --right-only <upstream>..<head>`

This really bit us in GVFS Git because we have several thousand developers
working on the same code base, and you probably read the numbers elsewhere
how many commits are introduced while a developer goes on so much as a
week-long vacation.

Next, rename detection.

I think that bit us even harder because it tries to look at all the files
that have been changed in upstream in the meantime, which can be *a lot*.
And if you know that moving files outside of your cozy little sparse
checkout is unlikely (or moving from the outside into your checkout), then
all this hydration is pretty wasteful. That's why we had to switch off
rename detection in GVFS Git.

> My guess at what objects are needed by each type:
> 
> At a high level, rebase --am for each commit will need to compare the
> commit to its parent to generate a diff (which thus involves walking
> over the objects in both the commit and its parent, though it should
> be able to skip over subtrees that are equal), and then will need to
> look at all the objects in the target commit on which it needs to
> apply the patch (in order to properly fill the index for a starting
> point, and used later when creating a new commit).

No, in --am mode, it does not even need to look at all the objects in the
target commits. Only those objects that correspond to the files that are
touched by the diff.

In a massive code base, such as Windows', this makes a huge difference.
Even comparing the number of files touched by the patches that are to be
rebased to the number of files that were touched in upstream since you
rebased last is ridiculous. And a three-way merge needs to consider that
latter set of files.

> If the application of the diff fails, it falls back to a three-way
> merge, though the three-way merge shouldn't need any additional objects.

The three-way merge needs to reconcile the diff between branch point and
your changes to the diff between branch point and upstream's changes. The
latter can be a lot bigger than the former, in which case --am's
complexity (O(former)) is much nicer than --merge's (O(former u latter)).

> So, to summarize, rebase--am needs objects from the commit being
> rebased, its parent, and the target commit onto which it is applying,
> though it can short circuit some objects when the commit and its parent
> have matching subtree(s).
> 
> rebase -i, if I understand correctly, does a three-way merge between
> the commit, its parent, and the target commit.  Thus, we again walk
> over objects in those three commits; I think unpack_trees() does not
> take advantage of matching trees to avoid descending into subtrees,
> but if so that's an optimization that we may be able to implement
> (though it would require diving into unpack_trees() code, which is
> never easy...).
> 
> (Side notes: (1) rebase --merge is basically the same as rebase -i
> here; it's path to reaching the recursive merge machinery is a bit
> different but the resulting arguments are the same; (2) a real merge
> between branches would require more objects because it would have to
> do some revision walking to find a merge base, and a real merge base
> is likely to differ more than just the parent commit.  But finding
> merge bases isn't relevant to rebase -m or rebase -i)
> 
> Is there something else I'm missing that fundamentally makes rebase -i
> need more objects?
> 
> > As to speed: that might be harder. But then, the performance might already
> > be good enough. I do not have numbers (nor the time to generate them) to
> > back up my hunch that --am is substantially faster than --merge.
> 
> I too have a hunch that --am is faster than --merge, on big enough
> repos or repos with enough renames.  I can partially back it up with
> an indirect number: at [1], it was reported that cherry-picks could be
> sped up by a factor of 20-30 on some repos with lots of renames.  I
> believe there are other performance improvements possible too, for the
> --merge or -i cases.
> 
> I'm also curious now whether your comment on hydrating objects might
> uncover additional areas where performance improvements could be made
> for non-am-based rebases of large-enough repos.
> 
> Elijah
> 
> [1] https://public-inbox.org/git/CABPp-BH4LLzeJjE5cvwWQJ8xTj3m9oC-41Tu8BM8c7R0gQTjWw@mail.gmail.com/
> (see also Peter's last reply in that thread, and compare to his first
> post)

I am too unfamiliar with the rename detection to understand the details,
but you are familiar with it, so I trust your judgement!

Ciao,
Dscho

[PATCH v3 0/7] Document/fix/warn about rebase incompatibilities and inconsistencies

[Elijah Newren]

git-rebase has lots of options that are mutually incompatible.  Even among
aspects of its behavior that is common to all rebase types, it has a number
of inconsistencies.  This series tries to document, fix, and/or warn users
about many of these.

I have left patch 7 in RFC state.  We probably need more opinions
about it, and whether cherry-pick should also be changed to match.

Changes since v2 (full branch-diff below):
  - Slight changes to documentation wording
  - Add incompatibilitiy of --rebase-merges and --strategy-option
  - Add tests and code for incompatibilities involving either
    --preserve-merges or --rebase-merges
  - Slightly more detail in the commit message for making -m and -i behave
    the same with empty commit messages as am-based rebases.

Many thanks to Phillip for reviewing the last series, and Eric who pointed
out the wording improvements.

Elijah Newren (7):
  git-rebase.txt: document incompatible options
  git-rebase.sh: update help messages a bit
  t3422: new testcases for checking when incompatible options passed
  git-rebase: error out when incompatible options passed
  git-rebase.txt: document behavioral inconsistencies between modes
  git-rebase.txt: address confusion between --no-ff vs --force-rebase
  git-rebase: make --allow-empty-message the default

 Documentation/git-rebase.txt           | 155 ++++++++++++++++++++-----
 git-rebase.sh                          |  42 ++++++-
 t/t3404-rebase-interactive.sh          |   7 +-
 t/t3405-rebase-malformed.sh            |  11 +-
 t/t3422-rebase-incompatible-options.sh |  89 ++++++++++++++
 5 files changed, 262 insertions(+), 42 deletions(-)
 create mode 100755 t/t3422-rebase-incompatible-options.sh

 1: d184d5bd3f !  1: 4cdf9130cc git-rebase.txt: document incompatible options
    @@ -191,14 +191,15 @@
     + * --keep-empty
     + * --autosquash
     + * --edit-todo
    -+ * --root + --onto
    ++ * --root when used in combination with --onto
     +
     +Other incompatible flag pairs:
     +
    -+ * --preserve-merges && --interactive
    -+ * --preserve-merges && --signoff
    -+ * --preserve-merges && --rebase-merges
    -+ * --rebase-merges && --strategy
    ++ * --preserve-merges and --interactive
    ++ * --preserve-merges and --signoff
    ++ * --preserve-merges and --rebase-merges
    ++ * --rebase-merges and --strategy
    ++ * --rebase-merges and --strategy-option
     +
      include::merge-strategies.txt[]
      
 2: 788df9fa43 =  2: e336f76c5e git-rebase.sh: update help messages a bit
 3: d3d124795a !  3: 4ab38d8a5f t3422: new testcases for checking when incompatible options passed
    @@ -83,4 +83,24 @@
     +test_run_rebase --committer-date-is-author-date
     +test_run_rebase -C4
     +
    ++test_expect_success '--preserve-merges incompatible with --signoff' '
    ++	git checkout B^0 &&
    ++	test_must_fail git rebase --preserve-merges --signoff A
    ++'
    ++
    ++test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
    ++	git checkout B^0 &&
    ++	test_must_fail git rebase --preserve-merges --rebase-merges A
    ++'
    ++
    ++test_expect_failure '--rebase-merges incompatible with --strategy' '
    ++	git checkout B^0 &&
    ++	test_must_fail git rebase --rebase-merges -s resolve A
    ++'
    ++
    ++test_expect_failure '--rebase-merges incompatible with --strategy-option' '
    ++	git checkout B^0 &&
    ++	test_must_fail git rebase --rebase-merges -Xignore-space-change A
    ++'
    ++
     +test_done
 4: 28d2dfd49a !  4: 5223954caf git-rebase: error out when incompatible options passed
    @@ -56,6 +56,30 @@
      if test -n "$signoff"
      then
      	test -n "$preserve_merges" &&
    +@@
    + 	force_rebase=t
    + fi
    + 
    ++if test -n "$preserve_merges"
    ++then
    ++	# Note: incompatibility with --signoff handled in signoff block above
    ++	# Note: incompatibility with --interactive is just a strong warning;
    ++	#       git-rebase.txt caveats with "unless you know what you are doing"
    ++	test -n "$rebase_merges" &&
    ++		die "$(gettext "error: cannot combine '--preserve_merges' with '--rebase-merges'")"
    ++fi
    ++
    ++if test -n "$rebase_merges"
    ++then
    ++	test -n "$strategy_opts" &&
    ++		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy-option'")"
    ++	test -n "$strategy" &&
    ++		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy'")"
    ++fi
    ++
    + if test -z "$rebase_root"
    + then
    + 	case "$#" in
     
     diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
     --- a/t/t3422-rebase-incompatible-options.sh
    @@ -93,3 +117,24 @@
      		git checkout B^0 &&
      		test_must_fail git rebase $opt --exec 'true' A
      	"
    +@@
    + 	test_must_fail git rebase --preserve-merges --signoff A
    + '
    + 
    +-test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
    ++test_expect_success '--preserve-merges incompatible with --rebase-merges' '
    + 	git checkout B^0 &&
    + 	test_must_fail git rebase --preserve-merges --rebase-merges A
    + '
    + 
    +-test_expect_failure '--rebase-merges incompatible with --strategy' '
    ++test_expect_success '--rebase-merges incompatible with --strategy' '
    + 	git checkout B^0 &&
    + 	test_must_fail git rebase --rebase-merges -s resolve A
    + '
    + 
    +-test_expect_failure '--rebase-merges incompatible with --strategy-option' '
    ++test_expect_success '--rebase-merges incompatible with --strategy-option' '
    + 	git checkout B^0 &&
    + 	test_must_fail git rebase --rebase-merges -Xignore-space-change A
    + '
 5: b2eec2cc5a !  5: 96f7ba98bc git-rebase.txt: document behavioral inconsistencies between modes
    @@ -14,8 +14,8 @@
     --- a/Documentation/git-rebase.txt
     +++ b/Documentation/git-rebase.txt
     @@
    -  * --preserve-merges && --rebase-merges
    -  * --rebase-merges && --strategy
    +  * --rebase-merges and --strategy
    +  * --rebase-merges and --strategy-option
      
     +BEHAVIORAL INCONSISTENCIES
     +--------------------------
 6: cea18d7e60 =  6: 7bb7b380ac git-rebase.txt: address confusion between --no-ff vs --force-rebase
 7: ab8805c40a !  7: 3ed07548a6 git-rebase: make --allow-empty-message the default
    @@ -2,9 +2,13 @@
     
         git-rebase: make --allow-empty-message the default
     
    -    am-based rebases already apply commits with an empty commit message
    -    without requiring the user to specify an extra flag.  Make merge-based and
    -    interactive-based rebases behave the same.
    +    All rebase backends should behave the same with empty commit messages, but
    +    currently do not.  am-based rebases already apply commits with an empty
    +    commit message without stopping or requiring the user to specify an extra
    +    flag.  Since am-based rebases are the default rebase type, and since it
    +    appears no one has ever requested a --no-allow-empty-message flag to
    +    change this behavior, make --allow-empty-message the default so that
    +    merge-based and interactive-based rebases will behave the same.
     
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
-- 
2.18.0.rc2.92.g133ed01dde

[PATCH v3 1/7] git-rebase.txt: document incompatible options

[Elijah Newren]

git rebase has many options that only work with one of its three backends.
It also has a few other pairs of incompatible options.  Document these.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 85 ++++++++++++++++++++++++++++++++----
 1 file changed, 77 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 0e20a66e73..7de6523931 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -243,11 +243,15 @@ leave out at most one of A and B, in which case it defaults to HEAD.
 --keep-empty::
 	Keep the commits that do not change anything from its
 	parents in the result.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --allow-empty-message::
 	By default, rebasing commits with an empty message will fail.
 	This option overrides that behavior, allowing commits with empty
 	messages to be rebased.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --skip::
 	Restart the rebasing process by skipping the current patch.
@@ -271,6 +275,8 @@ branch on top of the <upstream> branch.  Because of this, when a merge
 conflict happens, the side reported as 'ours' is the so-far rebased
 series, starting with <upstream>, and 'theirs' is the working branch.  In
 other words, the sides are swapped.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -s <strategy>::
 --strategy=<strategy>::
@@ -280,8 +286,10 @@ other words, the sides are swapped.
 +
 Because 'git rebase' replays each commit from the working branch
 on top of the <upstream> branch using the given strategy, using
-the 'ours' strategy simply discards all patches from the <branch>,
+the 'ours' strategy simply empties all patches from the <branch>,
 which makes little sense.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -X <strategy-option>::
 --strategy-option=<strategy-option>::
@@ -289,6 +297,8 @@ which makes little sense.
 	This implies `--merge` and, if no strategy has been
 	specified, `-s recursive`.  Note the reversal of 'ours' and
 	'theirs' as noted above for the `-m` option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -S[<keyid>]::
 --gpg-sign[=<keyid>]::
@@ -324,6 +334,8 @@ which makes little sense.
 	and after each change.  When fewer lines of surrounding
 	context exist they all must match.  By default no context is
 	ever ignored.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -f::
 --force-rebase::
@@ -355,19 +367,22 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 --whitespace=<option>::
 	These flag are passed to the 'git apply' program
 	(see linkgit:git-apply[1]) that applies the patch.
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --committer-date-is-author-date::
 --ignore-date::
 	These flags are passed to 'git am' to easily change the dates
 	of the rebased commits (see linkgit:git-am[1]).
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --signoff::
 	Add a Signed-off-by: trailer to all the rebased commits. Note
 	that if `--interactive` is given then only commits marked to be
-	picked, edited or reworded will have the trailer added. Incompatible
-	with the `--preserve-merges` option.
+	picked, edited or reworded will have the trailer added.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -i::
 --interactive::
@@ -378,6 +393,8 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -r::
 --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
@@ -404,7 +421,7 @@ It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
 explicit `exec git merge -s <strategy> [...]` commands.
 +
-See also REBASING MERGES below.
+See also REBASING MERGES and INCOMPATIBLE OPTIONS below.
 
 -p::
 --preserve-merges::
@@ -415,6 +432,8 @@ See also REBASING MERGES below.
 This uses the `--interactive` machinery internally, but combining it
 with the `--interactive` option explicitly is generally not a good
 idea unless you know what you are doing (see BUGS below).
++
+See also INCOMPATIBLE OPTIONS below.
 
 -x <cmd>::
 --exec <cmd>::
@@ -437,6 +456,8 @@ squash/fixup series.
 +
 This uses the `--interactive` machinery internally, but it can be run
 without an explicit `--interactive`.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --root::
 	Rebase all commits reachable from <branch>, instead of
@@ -447,6 +468,8 @@ without an explicit `--interactive`.
 	When used together with both --onto and --preserve-merges,
 	'all' root commits will be rewritten to have <newbase> as parent
 	instead.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autosquash::
 --no-autosquash::
@@ -461,11 +484,11 @@ without an explicit `--interactive`.
 	too.  The recommended way to create fixup/squash commits is by using
 	the `--fixup`/`--squash` options of linkgit:git-commit[1].
 +
-This option is only valid when the `--interactive` option is used.
-+
 If the `--autosquash` option is enabled by default using the
 configuration variable `rebase.autoSquash`, this option can be
 used to override and disable this setting.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autostash::
 --no-autostash::
@@ -487,6 +510,52 @@ recreates the topic branch with fresh commits so it can be remerged
 successfully without needing to "revert the reversion" (see the
 link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
 
+INCOMPATIBLE OPTIONS
+--------------------
+
+git-rebase has many flags that are incompatible with each other,
+predominantly due to the fact that it has three different underlying
+implementations:
+
+ * one based on linkgit:git-am[1] (the default)
+ * one based on linkgit:git-merge-recursive[1] (merge backend)
+ * one based on linkgit:git-cherry-pick[1] (interactive backend)
+
+Flags only understood by the am backend:
+
+ * --committer-date-is-author-date
+ * --ignore-date
+ * --whitespace
+ * --ignore-whitespace
+ * -C
+
+Flags understood by both merge and interactive backends:
+
+ * --merge
+ * --strategy
+ * --strategy-option
+ * --allow-empty-message
+
+Flags only understood by the interactive backend:
+
+ * --[no-]autosquash
+ * --rebase-merges
+ * --preserve-merges
+ * --interactive
+ * --exec
+ * --keep-empty
+ * --autosquash
+ * --edit-todo
+ * --root when used in combination with --onto
+
+Other incompatible flag pairs:
+
+ * --preserve-merges and --interactive
+ * --preserve-merges and --signoff
+ * --preserve-merges and --rebase-merges
+ * --rebase-merges and --strategy
+ * --rebase-merges and --strategy-option
+
 include::merge-strategies.txt[]
 
 NOTES
-- 
2.18.0.rc2.92.g133ed01dde

[PATCH v3 2/7] git-rebase.sh: update help messages a bit

[Elijah Newren]

signoff is not specific to the am-backend.  Also, re-order a few options
to make like things (e.g. strategy and strategy-option) be near each
other.
---
 git-rebase.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 40be59ecc4..bf71b7fa20 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -20,23 +20,23 @@ onto=!             rebase onto given branch instead of upstream
 r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
+X,strategy-option=! pass the argument through to the merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
+f,force-rebase!    cherry-pick all commits, even if unchanged
 m,merge!           use merging strategies to rebase
 i,interactive!     let the user edit the list of commits to rebase
 x,exec=!           add exec lines after each commit of the editable list
 k,keep-empty	   preserve empty commits during rebase
 allow-empty-message allow rebasing commits with empty messages
-f,force-rebase!    force rebase even if branch is up to date
-X,strategy-option=! pass the argument through to the merge strategy
 stat!              display a diffstat of what changed upstream
 n,no-stat!         do not show diffstat of what changed upstream
 verify             allow pre-rebase hook to run
 rerere-autoupdate  allow rerere to update index with resolved conflicts
 root!              rebase all reachable commits up to the root(s)
 autosquash         move commits that begin with squash!/fixup! under -i
+signoff            add a Signed-off-by: line to each commit
 committer-date-is-author-date! passed to 'git am'
 ignore-date!       passed to 'git am'
-signoff            passed to 'git am'
 whitespace=!       passed to 'git apply'
 ignore-whitespace! passed to 'git apply'
 C=!                passed to 'git apply'
-- 
2.18.0.rc2.92.g133ed01dde

[PATCH v3 3/7] t3422: new testcases for checking when incompatible options passed

[Elijah Newren]

git rebase is split into three types: am, merge, and interactive.  Various
options imply different types, and which mode we are using determine which
sub-script (git-rebase--$type) is executed to finish the work.  Not all
options work with all types, so add tests for combinations where we expect
to receive an error rather than having options be silently ignored.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3422-rebase-incompatible-options.sh | 89 ++++++++++++++++++++++++++
 1 file changed, 89 insertions(+)
 create mode 100755 t/t3422-rebase-incompatible-options.sh

diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
new file mode 100755
index 0000000000..157374143f
--- /dev/null
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -0,0 +1,89 @@
+#!/bin/sh
+
+test_description='test if rebase detects and aborts on incompatible options'
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	test_seq 2 9 >foo &&
+	git add foo &&
+	git commit -m orig &&
+
+	git branch A &&
+	git branch B &&
+
+	git checkout A &&
+	test_seq 1 9 >foo &&
+	git add foo &&
+	git commit -m A &&
+
+	git checkout B &&
+	# This is indented with HT SP HT.
+	echo "	 	foo();" >>foo &&
+	git add foo &&
+	git commit -m B
+'
+
+#
+# Rebase has lots of useful options like --whitepsace=fix, which are
+# actually all built in terms of flags to git-am.  Since neither
+# --merge nor --interactive (nor any options that imply those two) use
+# git-am, using them together will result in flags like --whitespace=fix
+# being ignored.  Make sure rebase warns the user and aborts instead.
+#
+
+test_run_rebase () {
+	opt=$1
+	shift
+	test_expect_failure "$opt incompatible with --merge" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --merge A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --interactive" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --interactive A
+	"
+
+	test_expect_failure "$opt incompatible with --exec" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --exec 'true' A
+	"
+
+}
+
+test_run_rebase --whitespace=fix
+test_run_rebase --ignore-whitespace
+test_run_rebase --committer-date-is-author-date
+test_run_rebase -C4
+
+test_expect_success '--preserve-merges incompatible with --signoff' '
+	git checkout B^0 &&
+	test_must_fail git rebase --preserve-merges --signoff A
+'
+
+test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
+	git checkout B^0 &&
+	test_must_fail git rebase --preserve-merges --rebase-merges A
+'
+
+test_expect_failure '--rebase-merges incompatible with --strategy' '
+	git checkout B^0 &&
+	test_must_fail git rebase --rebase-merges -s resolve A
+'
+
+test_expect_failure '--rebase-merges incompatible with --strategy-option' '
+	git checkout B^0 &&
+	test_must_fail git rebase --rebase-merges -Xignore-space-change A
+'
+
+test_done
-- 
2.18.0.rc2.92.g133ed01dde

[PATCH v3 4/7] git-rebase: error out when incompatible options passed

[Elijah Newren]

git rebase has three different types: am, merge, and interactive, all of
which are implemented in terms of separate scripts.  am builds on git-am,
merge builds on git-merge-recursive, and interactive builds on
git-cherry-pick.  We make use of features in those lower-level commands in
the different rebase types, but those features don't exist in all of the
lower level commands so we have a range of incompatibilities.  Previously,
we just accepted nearly any argument and silently ignored whichever ones
weren't implemented for the type of rebase specified.  Change this so the
incompatibilities are documented, included in the testsuite, and tested
for at runtime with an appropriate error message shown.

Some exceptions I left out:

  * --merge and --interactive are technically incompatible since they are
    supposed to run different underlying scripts, but with a few small
    changes, --interactive can do everything that --merge can.  In fact,
    I'll shortly be sending another patch to remove git-rebase--merge and
    reimplement it on top of git-rebase--interactive.

  * One could argue that --interactive and --quiet are incompatible since
    --interactive doesn't implement a --quiet mode (perhaps since
    cherry-pick itself does not implement one).  However, the interactive
    mode is more quiet than the other modes in general with progress
    messages, so one could argue that it's already quiet.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh                          | 34 ++++++++++++++++++++++++++
 t/t3422-rebase-incompatible-options.sh | 16 ++++++------
 2 files changed, 42 insertions(+), 8 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index bf71b7fa20..590bdbb12a 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -503,6 +503,23 @@ then
 	git_format_patch_opt="$git_format_patch_opt --progress"
 fi
 
+if test -n "$git_am_opt"; then
+	incompatible_opts=$(echo "$git_am_opt" | sed -e 's/ -q//')
+	if test -n "$interactive_rebase"
+	then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
+		fi
+	fi
+	if test -n "$do_merge"; then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
+		fi
+	fi
+fi
+
 if test -n "$signoff"
 then
 	test -n "$preserve_merges" &&
@@ -511,6 +528,23 @@ then
 	force_rebase=t
 fi
 
+if test -n "$preserve_merges"
+then
+	# Note: incompatibility with --signoff handled in signoff block above
+	# Note: incompatibility with --interactive is just a strong warning;
+	#       git-rebase.txt caveats with "unless you know what you are doing"
+	test -n "$rebase_merges" &&
+		die "$(gettext "error: cannot combine '--preserve_merges' with '--rebase-merges'")"
+fi
+
+if test -n "$rebase_merges"
+then
+	test -n "$strategy_opts" &&
+		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy-option'")"
+	test -n "$strategy" &&
+		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy'")"
+fi
+
 if test -z "$rebase_root"
 then
 	case "$#" in
diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
index 157374143f..415d5226bf 100755
--- a/t/t3422-rebase-incompatible-options.sh
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -34,27 +34,27 @@ test_expect_success 'setup' '
 test_run_rebase () {
 	opt=$1
 	shift
-	test_expect_failure "$opt incompatible with --merge" "
+	test_expect_success "$opt incompatible with --merge" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --merge A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy=ours" "
+	test_expect_success "$opt incompatible with --strategy=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+	test_expect_success "$opt incompatible with --strategy-option=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --interactive" "
+	test_expect_success "$opt incompatible with --interactive" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --interactive A
 	"
 
-	test_expect_failure "$opt incompatible with --exec" "
+	test_expect_success "$opt incompatible with --exec" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --exec 'true' A
 	"
@@ -71,17 +71,17 @@ test_expect_success '--preserve-merges incompatible with --signoff' '
 	test_must_fail git rebase --preserve-merges --signoff A
 '
 
-test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
+test_expect_success '--preserve-merges incompatible with --rebase-merges' '
 	git checkout B^0 &&
 	test_must_fail git rebase --preserve-merges --rebase-merges A
 '
 
-test_expect_failure '--rebase-merges incompatible with --strategy' '
+test_expect_success '--rebase-merges incompatible with --strategy' '
 	git checkout B^0 &&
 	test_must_fail git rebase --rebase-merges -s resolve A
 '
 
-test_expect_failure '--rebase-merges incompatible with --strategy-option' '
+test_expect_success '--rebase-merges incompatible with --strategy-option' '
 	git checkout B^0 &&
 	test_must_fail git rebase --rebase-merges -Xignore-space-change A
 '
-- 
2.18.0.rc2.92.g133ed01dde

[PATCH v3 5/7] git-rebase.txt: document behavioral inconsistencies between modes

[Elijah Newren]

There are a variety of aspects that are common to all rebases regardless
of which backend is in use; however, the behavior for these different
aspects varies in ways that could surprise users.  (In fact, it's not
clear -- to me at least -- that these differences were even desirable or
intentional.)  Document these differences.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 57 ++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 7de6523931..340137e2cf 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -556,6 +556,63 @@ Other incompatible flag pairs:
  * --rebase-merges and --strategy
  * --rebase-merges and --strategy-option
 
+BEHAVIORAL INCONSISTENCIES
+--------------------------
+
+  * --no-ff vs. --force-rebase
+
+    These options are actually identical, though their description
+    leads people to believe they might not be.
+
+ * empty commits:
+
+    am-based rebase will drop any "empty" commits, whether the
+    commit started empty (had no changes relative to its parent to
+    start with) or ended empty (all changes were already applied
+    upstream in other commits).
+
+    merge-based rebase does the same.
+
+    interactive-based rebase will by default drop commits that
+    started empty and halt if it hits a commit that ended up empty.
+    The --keep-empty option exists for interactive rebases to allow
+    it to keep commits that started empty.
+
+  * empty commit messages:
+
+    am-based rebase will silently apply commits with empty commit
+    messages.
+
+    merge-based and interactive-based rebases will by default halt
+    on any such commits.  The --allow-empty-message option exists to
+    allow interactive-based rebases to apply such commits without
+    halting.
+
+  * directory rename detection:
+
+    merge-based and interactive-based rebases work fine with
+    directory rename detection.  am-based rebases sometimes do not.
+
+    git-am tries to avoid a full three way merge, instead calling
+    git-apply.  That prevents us from detecting renames at all,
+    which may defeat the directory rename detection.  There is a
+    fallback, though; if the initial git-apply fails and the user
+    has specified the -3 option, git-am will fall back to a three
+    way merge.  However, git-am lacks the necessary information to
+    do a "real" three way merge.  Instead, it has to use
+    build_fake_ancestor() to get a merge base that is missing files
+    whose rename may have been important to detect for directory
+    rename detection to function.
+
+    Since am-based rebases work by first generating a bunch of
+    patches (which no longer record what the original commits were
+    and thus don't have the necessary info from which we can find a
+    real merge-base), and then calling git-am, this implies that
+    am-based rebases will not always successfully detect directory
+    renames either.  merged-based rebases (rebase -m) and
+    cherry-pick-based rebases (rebase -i) are not affected by this
+    shortcoming.
+
 include::merge-strategies.txt[]
 
 NOTES
-- 
2.18.0.rc2.92.g133ed01dde

[PATCH v3 6/7] git-rebase.txt: address confusion between --no-ff vs --force-rebase

[Elijah Newren]

rebase was taught the --force-rebase option in commit b2f82e05de ("Teach
rebase to rebase even if upstream is up to date", 2009-02-13).  This flag
worked for the am and merge backends, but wasn't a valid option for the
interactive backend.

rebase was taught the --no-ff option for interactive rebases in commit
b499549401cb ("Teach rebase the --no-ff option.", 2010-03-24), to do the
exact same thing as --force-rebase does for non-interactive rebases.  This
commit explicitly documented the fact that --force-rebase was incompatible
with --interactive, though it made --no-ff a synonym for --force-rebase
for non-interactive rebases.  The choice of a new option was based on the
fact that "force rebase" didn't sound like an appropriate term for the
interactive machinery.

In commit 6bb4e485cff8 ("rebase: align variable names", 2011-02-06), the
separate parsing of command line options in the different rebase scripts
was removed, and whether on accident or because the author noticed that
these options did the same thing, the options became synonyms and both
were accepted by all three rebase types.

In commit 2d26d533a012 ("Documentation/git-rebase.txt: -f forces a rebase
that would otherwise be a no-op", 2014-08-12), which reworded the
description of the --force-rebase option, the (no-longer correct) sentence
stating that --force-rebase was incompatible with --interactive was
finally removed.

Finally, as explained at
https://public-inbox.org/git/98279912-0f52-969d-44a6-22242039387f@xiplink.com

    In the original discussion around this option [1], at one point I
    proposed teaching rebase--interactive to respect --force-rebase
    instead of adding a new option [2].  Ultimately --no-ff was chosen as
    the better user interface design [3], because an interactive rebase
    can't be "forced" to run.

We have accepted both --no-ff and --force-rebase as full synonyms for all
three rebase types for over seven years.  Documenting them differently
and in ways that suggest they might not be quite synonyms simply leads to
confusion.  Adjust the documentation to match reality.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 35 ++++++++++-------------------------
 1 file changed, 10 insertions(+), 25 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 340137e2cf..1f422cd0a7 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -337,16 +337,18 @@ See also INCOMPATIBLE OPTIONS below.
 +
 See also INCOMPATIBLE OPTIONS below.
 
--f::
+--no-ff::
 --force-rebase::
-	Force a rebase even if the current branch is up to date and
-	the command without `--force` would return without doing anything.
+-f::
+	Individually replay all rebased commits instead of fast-forwarding
+	over the unchanged ones.  This ensures that the entire history of
+	the rebased branch is composed of new commits.
 +
-You may find this (or --no-ff with an interactive rebase) helpful after
-reverting a topic branch merge, as this option recreates the topic branch with
-fresh commits so it can be remerged successfully without needing to "revert
-the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
+You may find this helpful after reverting a topic branch merge, as this option
+recreates the topic branch with fresh commits so it can be remerged
+successfully without needing to "revert the reversion" (see the
+link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for
+details).
 
 --fork-point::
 --no-fork-point::
@@ -498,18 +500,6 @@ See also INCOMPATIBLE OPTIONS below.
 	with care: the final stash application after a successful
 	rebase might result in non-trivial conflicts.
 
---no-ff::
-	With --interactive, cherry-pick all rebased commits instead of
-	fast-forwarding over the unchanged ones.  This ensures that the
-	entire history of the rebased branch is composed of new commits.
-+
-Without --interactive, this is a synonym for --force-rebase.
-+
-You may find this helpful after reverting a topic branch merge, as this option
-recreates the topic branch with fresh commits so it can be remerged
-successfully without needing to "revert the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
-
 INCOMPATIBLE OPTIONS
 --------------------
 
@@ -559,11 +549,6 @@ Other incompatible flag pairs:
 BEHAVIORAL INCONSISTENCIES
 --------------------------
 
-  * --no-ff vs. --force-rebase
-
-    These options are actually identical, though their description
-    leads people to believe they might not be.
-
  * empty commits:
 
     am-based rebase will drop any "empty" commits, whether the
-- 
2.18.0.rc2.92.g133ed01dde

[RFC PATCH v3 7/7] git-rebase: make --allow-empty-message the default

[Elijah Newren]

All rebase backends should behave the same with empty commit messages, but
currently do not.  am-based rebases already apply commits with an empty
commit message without stopping or requiring the user to specify an extra
flag.  Since am-based rebases are the default rebase type, and since it
appears no one has ever requested a --no-allow-empty-message flag to
change this behavior, make --allow-empty-message the default so that
merge-based and interactive-based rebases will behave the same.

Signed-off-by: Elijah Newren <newren@gmail.com>
---

See also
https://public-inbox.org/git/CABPp-BHrcUHX_zHxpojV5=sxJ1=NoDg9uhxv+NH5BsHsQYavPQ@mail.gmail.com/
for some discussion of this change.

 Documentation/git-rebase.txt  | 10 ----------
 git-rebase.sh                 |  2 +-
 t/t3404-rebase-interactive.sh |  7 ++++---
 t/t3405-rebase-malformed.sh   | 11 +++--------
 4 files changed, 8 insertions(+), 22 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 1f422cd0a7..3addcf8218 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -563,16 +563,6 @@ BEHAVIORAL INCONSISTENCIES
     The --keep-empty option exists for interactive rebases to allow
     it to keep commits that started empty.
 
-  * empty commit messages:
-
-    am-based rebase will silently apply commits with empty commit
-    messages.
-
-    merge-based and interactive-based rebases will by default halt
-    on any such commits.  The --allow-empty-message option exists to
-    allow interactive-based rebases to apply such commits without
-    halting.
-
   * directory rename detection:
 
     merge-based and interactive-based rebases work fine with
diff --git a/git-rebase.sh b/git-rebase.sh
index 590bdbb12a..a909a2cfb1 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -95,7 +95,7 @@ rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
-allow_empty_message=
+allow_empty_message=--allow-empty-message
 signoff=
 test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
 case "$(git config --bool commit.gpgsign)" in
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 352a52e59d..81ce9fe7f9 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -553,15 +553,16 @@ test_expect_success '--continue tries to commit, even for "edit"' '
 '
 
 test_expect_success 'aborted --continue does not squash commits after "edit"' '
+	test_when_finished "git rebase --abort" &&
 	old=$(git rev-parse HEAD) &&
 	test_tick &&
 	set_fake_editor &&
 	FAKE_LINES="edit 1" git rebase -i HEAD^ &&
 	echo "edited again" > file7 &&
 	git add file7 &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " git rebase --continue &&
-	test $old = $(git rev-parse HEAD) &&
-	git rebase --abort
+	echo all the things >>conflict &&
+	test_must_fail git rebase --continue &&
+	test $old = $(git rev-parse HEAD)
 '
 
 test_expect_success 'auto-amend only edited commits after "edit"' '
diff --git a/t/t3405-rebase-malformed.sh b/t/t3405-rebase-malformed.sh
index cb7c6de84a..da94dddc86 100755
--- a/t/t3405-rebase-malformed.sh
+++ b/t/t3405-rebase-malformed.sh
@@ -77,19 +77,14 @@ test_expect_success 'rebase commit with diff in message' '
 '
 
 test_expect_success 'rebase -m commit with empty message' '
-	test_must_fail git rebase -m master empty-message-merge &&
-	git rebase --abort &&
-	git rebase -m --allow-empty-message master empty-message-merge
+	git rebase -m master empty-message-merge
 '
 
 test_expect_success 'rebase -i commit with empty message' '
 	git checkout diff-in-message &&
 	set_fake_editor &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i HEAD^ &&
-	git rebase --abort &&
-	FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i --allow-empty-message HEAD^
+	env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
+		git rebase -i HEAD^
 '
 
 test_done
-- 
2.18.0.rc2.92.g133ed01dde

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Elijah Newren]

Hi Dscho,

Thanks for all the food for thought.  This is awesome.

On Thu, Jun 21, 2018 at 3:57 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> On Wed, 20 Jun 2018, Elijah Newren wrote:
>> On Sun, Jun 17, 2018 at 2:44 PM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>
>> > I was really referring to speed. But I have to admit that I do not have
>> > any current numbers.
>> >
>> > Another issue just hit me, though: rebase --am does not need to look at as
>> > many Git objects as rebase --merge or rebase -i. Therefore, GVFS users
>> > will still want to use --am wherever possible, to avoid "hydrating"
>> > many objects during their rebase.
>>
>> What is it that makes rebase --am need fewer Git objects than rebase
>> --merge or rebase -i?
>
> Multiple things. The most obvious thing: --cherry-pick. When you call `git
> rebase --am`, it does not try to exclude the patches by looking at
> `rev-list --cherry-pick --right-only <upstream>..<head>`
>
> This really bit us in GVFS Git because we have several thousand developers
> working on the same code base, and you probably read the numbers elsewhere
> how many commits are introduced while a developer goes on so much as a
> week-long vacation.

Ooh, that's interesting.  Here's a crazy idea: I'm curious if this is
essentially a workaround that we could expunge.  In particular,
am-based rebases will drop empty commits (that is, either commits that
are empty to start, or that after being applied are empty).  It never
grew a --no-allow-empty option either.  If rebase -i/-m were to behave
the same, for consistency sake, we could drop the `rev-list
--cherry-pick ...` call.  Granted, that would result in a slight
difference in behavior (it could result in conflicts in some cases
instead of an automatically empty commit), but may well be worth it
for consistencies' sake between rebase backends.  As a side effect of
making the backends consistent, it may also provide a nice performance
boost for the GVFS case by removing the need to do the --cherry-pick
thing.

> Next, rename detection.
>
> I think that bit us even harder because it tries to look at all the files
> that have been changed in upstream in the meantime, which can be *a lot*.
> And if you know that moving files outside of your cozy little sparse
> checkout is unlikely (or moving from the outside into your checkout), then
> all this hydration is pretty wasteful. That's why we had to switch off
> rename detection in GVFS Git.

Actually to be clear, for each side of the merge, rename detection
only needs to look at files that were not present in both the
merge-base and the branch.  Files which were modified but have the
same name are not involved (break detection is not active.).  But yes,
I understand how you can still get an awful lot of files, which makes
it worse when you then go and compare all of them in an O(N*M)
fashion.

>> My guess at what objects are needed by each type:
>>
>> At a high level, rebase --am for each commit will need to compare the
>> commit to its parent to generate a diff (which thus involves walking
>> over the objects in both the commit and its parent, though it should
>> be able to skip over subtrees that are equal), and then will need to
>> look at all the objects in the target commit on which it needs to
>> apply the patch (in order to properly fill the index for a starting
>> point, and used later when creating a new commit).
>
> No, in --am mode, it does not even need to look at all the objects in the
> target commits. Only those objects that correspond to the files that are
> touched by the diff.

Sorry, I mis-spoke.  Unless you've done something special with GVFS, I
believe the --am mode would indeed need to read all the _tree_ objects
in the target commits, but any _file_ object corresponding to a path
not modified by the other side need not be loaded since we can proceed
simply knowing its sha1sum.

> In a massive code base, such as Windows', this makes a huge difference.
> Even comparing the number of files touched by the patches that are to be
> rebased to the number of files that were touched in upstream since you
> rebased last is ridiculous. And a three-way merge needs to consider that
> latter set of files.

Actually, the three-way merge isn't all that different.  For a path
that exists in both branches and the merge base, it first compares
sha1sums (thus it does need to read all the tree objects), and then if
one side has not modified a certain path, it knows the merge result is
the sha1sum from the other side.  Thus, it won't need to read files
that were changed upstream so long as that path wasn't changed on your
side.

In fact, --merge and --interactive have a slight performance advantage
here: if upstream didn't modify the path, then it doesn't need to read
the file from your side to do any diff or comparison.  It can just use
whatever sha1sum you have.  In contrast --am mode will need to read
the relevant file more than once.  Since it first creates a series of
patches, it will have to read the file from your commit and its
parent, create a diff, and then later apply that diff to the target
version, and since the target version matches the merge base it will
end up just recovering the version of the file on your side that you
started with anyway.  (Since that file is already local, though, this
small performance advantage would currently be lost in the noise of
the other problems.)

>> If the application of the diff fails, it falls back to a three-way
>> merge, though the three-way merge shouldn't need any additional objects.
>
> The three-way merge needs to reconcile the diff between branch point and
> your changes to the diff between branch point and upstream's changes. The
> latter can be a lot bigger than the former, in which case --am's
> complexity (O(former)) is much nicer than --merge's (O(former u latter)).

Yeah, renames mess up that whole "a path that exists in both branches
and the merge base" requirement that I stipulated above, and widens
the scope considerably and affects performance rather markedly.  On
the positive side though, the more I read your description, the more I
think my rename performance work may give a speedup for the GVFS
usecase far in excess of the factor of 30 I saw for my usecase.  It
may still not quite match --am mode's outright avoidance of rename
detection, but I still think it should be pretty impressive.  Time
will tell...


Elijah

Re: [PATCH v3 1/7] git-rebase.txt: document incompatible options

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> @@ -280,8 +286,10 @@ other words, the sides are swapped.
>  +
>  Because 'git rebase' replays each commit from the working branch
>  on top of the <upstream> branch using the given strategy, using
> -the 'ours' strategy simply discards all patches from the <branch>,
> +the 'ours' strategy simply empties all patches from the <branch>,

I think overall the direction this series takes, and especially what
this step does---clarify what the current code & design does first
before attempting to improve the status quo, makes sense, but I had
trouble justifying this change.  Do you want to emphasize that the
operation discards the changes but still leaves empty commits, and
"discards all patches" imply there won't be any commits left on top
of the "onto" point?

>  which makes little sense.
> ++
> +See also INCOMPATIBLE OPTIONS below.

> @@ -487,6 +510,52 @@ recreates the topic branch with fresh commits so it can be remerged
>  successfully without needing to "revert the reversion" (see the
>  link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
>  
> +INCOMPATIBLE OPTIONS
> +--------------------
> +
> +git-rebase has many flags that are incompatible with each other,
> +predominantly due to the fact that it has three different underlying
> +implementations:
> +
> + * one based on linkgit:git-am[1] (the default)
> + * one based on linkgit:git-merge-recursive[1] (merge backend)
> + * one based on linkgit:git-cherry-pick[1] (interactive backend)
> +
> +Flags only understood by the am backend:
> +
> + * --committer-date-is-author-date
> + * --ignore-date
> + * --whitespace
> + * --ignore-whitespace
> + * -C
> +
> +Flags understood by both merge and interactive backends:
> +
> + * --merge
> + * --strategy
> + * --strategy-option
> + * --allow-empty-message
> +
> +Flags only understood by the interactive backend:
> +
> + * --[no-]autosquash
> + * --rebase-merges
> + * --preserve-merges
> + * --interactive
> + * --exec
> + * --keep-empty
> + * --autosquash
> + * --edit-todo
> + * --root when used in combination with --onto
> +
> +Other incompatible flag pairs:
> +
> + * --preserve-merges and --interactive
> + * --preserve-merges and --signoff
> + * --preserve-merges and --rebase-merges
> + * --rebase-merges and --strategy
> + * --rebase-merges and --strategy-option
> +
>  include::merge-strategies.txt[]

It is a bit unclear what these lists want to say.  If --ignore-date
is only understood by "rebase" and not "rebase -i", wouldn't that
make "--interactive and --ignore-date" an incompatible flag pair?
Or do you mean the "Other incompatible" list to enumerate the ones
that are explicitly rejected, as opposed to getting silently
ignored?

In any case, I'll need to read through the remainder of the series
to come to any conclusion, but given that -p is getting killed by
its primary author, with help from others to split out its
implementation from the --interactive codepath, it might be a bad
time to do this series.  Thanks for working on this topic.

Re: [PATCH v3 2/7] git-rebase.sh: update help messages a bit

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> signoff is not specific to the am-backend.  Also, re-order a few options
> to make like things (e.g. strategy and strategy-option) be near each
> other.
> ---
>  git-rebase.sh | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

Missing sign-off, but otherwise looks good.

>
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 40be59ecc4..bf71b7fa20 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -20,23 +20,23 @@ onto=!             rebase onto given branch instead of upstream
>  r,rebase-merges?   try to rebase merges instead of skipping them
>  p,preserve-merges! try to recreate merges instead of ignoring them
>  s,strategy=!       use the given merge strategy
> +X,strategy-option=! pass the argument through to the merge strategy
>  no-ff!             cherry-pick all commits, even if unchanged
> +f,force-rebase!    cherry-pick all commits, even if unchanged
>  m,merge!           use merging strategies to rebase
>  i,interactive!     let the user edit the list of commits to rebase
>  x,exec=!           add exec lines after each commit of the editable list
>  k,keep-empty	   preserve empty commits during rebase
>  allow-empty-message allow rebasing commits with empty messages
> -f,force-rebase!    force rebase even if branch is up to date
> -X,strategy-option=! pass the argument through to the merge strategy
>  stat!              display a diffstat of what changed upstream
>  n,no-stat!         do not show diffstat of what changed upstream
>  verify             allow pre-rebase hook to run
>  rerere-autoupdate  allow rerere to update index with resolved conflicts
>  root!              rebase all reachable commits up to the root(s)
>  autosquash         move commits that begin with squash!/fixup! under -i
> +signoff            add a Signed-off-by: line to each commit
>  committer-date-is-author-date! passed to 'git am'
>  ignore-date!       passed to 'git am'
> -signoff            passed to 'git am'
>  whitespace=!       passed to 'git apply'
>  ignore-whitespace! passed to 'git apply'
>  C=!                passed to 'git apply'

Re: [PATCH v3 3/7] t3422: new testcases for checking when incompatible options passed

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> +	git checkout B &&
> +	# This is indented with HT SP HT.
> +	echo "	 	foo();" >>foo &&

I often wonder, whenever I see a need for a comment like this, if
saying the same thing in code to make it visible is cleaner and less
error prone way to do so, i.e. e.g.

	echo "_ _foo();" | tr "_" "\011" >>foo &&

> +# Rebase has lots of useful options like --whitepsace=fix, which are
> +# actually all built in terms of flags to git-am.  Since neither
> +# --merge nor --interactive (nor any options that imply those two) use
> +# git-am, using them together will result in flags like --whitespace=fix
> +# being ignored.  Make sure rebase warns the user and aborts instead.
> +#
> +
> +test_run_rebase () {
> +	opt=$1
> +	shift
> +	test_expect_failure "$opt incompatible with --merge" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --merge A
> +	"
> +
> +	test_expect_failure "$opt incompatible with --strategy=ours" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --strategy=ours A
> +	"
> +
> +	test_expect_failure "$opt incompatible with --strategy-option=ours" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --strategy=ours A
> +	"
> +
> +	test_expect_failure "$opt incompatible with --interactive" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --interactive A
> +	"
> +
> +	test_expect_failure "$opt incompatible with --exec" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --exec 'true' A
> +	"
> +
> +}
> +
> +test_run_rebase --whitespace=fix
> +test_run_rebase --ignore-whitespace
> +test_run_rebase --committer-date-is-author-date
> +test_run_rebase -C4

I happen to be from old school and "rebase" primarily means
"format-patch piped to am" in my mind, so from that point of view,
"test_run_rebase --OPT" that says "--OPT which is a valid option for
the primary operating mode of rebase does not work with the other
exotic modes of the command" is not all that bad, but I do not think
that worldview holds for many people in general.  Perhaps calling it
something like "test_rebase_am_only" makes the intent clearer?

> +test_expect_success '--preserve-merges incompatible with --signoff' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --preserve-merges --signoff A
> +'
> +
> +test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --preserve-merges --rebase-merges A
> +'
> +
> +test_expect_failure '--rebase-merges incompatible with --strategy' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --rebase-merges -s resolve A
> +'
> +
> +test_expect_failure '--rebase-merges incompatible with --strategy-option' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --rebase-merges -Xignore-space-change A
> +'
> +
> +test_done

Re: [PATCH v3 4/7] git-rebase: error out when incompatible options passed

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> +if test -n "$git_am_opt"; then
> +	incompatible_opts=$(echo "$git_am_opt" | sed -e 's/ -q//')

This is probably just a taste-thing, but I'd probably prefer to see
the "sed" filter out "-q" alone, and make an effort to leave future
things like "-qx" alone, e.g.

    $(echo " $git_am_opt " | sed -e 's/ -q / /g' -e 's/^ \(.*\) $/\1/')

or something like that.

> +	if test -n "$interactive_rebase"
> +	then
> +		if test -n "$incompatible_opts"
> +		then
> +			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
> +		fi
> +	fi
> +	if test -n "$do_merge"; then
> +		if test -n "$incompatible_opts"
> +		then
> +			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
> +		fi
> +	fi
> +fi

Not making --merge and --interactive incompatible as the proposed
log message said makes this hunk at manageable complexity, I guess
;-)

Re: [PATCH v3 1/7] git-rebase.txt: document incompatible options

[Elijah Newren]

On Thu, Jun 21, 2018 at 12:47 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> @@ -280,8 +286,10 @@ other words, the sides are swapped.
>>  +
>>  Because 'git rebase' replays each commit from the working branch
>>  on top of the <upstream> branch using the given strategy, using
>> -the 'ours' strategy simply discards all patches from the <branch>,
>> +the 'ours' strategy simply empties all patches from the <branch>,
>
> I think overall the direction this series takes, and especially what
> this step does---clarify what the current code & design does first
> before attempting to improve the status quo, makes sense, but I had
> trouble justifying this change.  Do you want to emphasize that the
> operation discards the changes but still leaves empty commits, and
> "discards all patches" imply there won't be any commits left on top
> of the "onto" point?

Yes, is there an alternative wording that you'd find more clear?

(Also, I'd like to push to make rebase -i/-m behave the same as
am-based rebase with empty commits, meaning they are just discarded by
default rather than halted at.  But I haven't gotten to that point...)

>> @@ -487,6 +510,52 @@ recreates the topic branch with fresh commits so it can be remerged
>>  successfully without needing to "revert the reversion" (see the
>>  link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
>>
>> +INCOMPATIBLE OPTIONS
>> +--------------------
>> +
>> +git-rebase has many flags that are incompatible with each other,
>> +predominantly due to the fact that it has three different underlying
>> +implementations:
>> +
>> + * one based on linkgit:git-am[1] (the default)
>> + * one based on linkgit:git-merge-recursive[1] (merge backend)
>> + * one based on linkgit:git-cherry-pick[1] (interactive backend)
>> +
>> +Flags only understood by the am backend:
>> +
>> + * --committer-date-is-author-date
>> + * --ignore-date
>> + * --whitespace
>> + * --ignore-whitespace
>> + * -C
>> +
>> +Flags understood by both merge and interactive backends:
>> +
>> + * --merge
>> + * --strategy
>> + * --strategy-option
>> + * --allow-empty-message
>> +
>> +Flags only understood by the interactive backend:
>> +
>> + * --[no-]autosquash
>> + * --rebase-merges
>> + * --preserve-merges
>> + * --interactive
>> + * --exec
>> + * --keep-empty
>> + * --autosquash
>> + * --edit-todo
>> + * --root when used in combination with --onto
>> +
>> +Other incompatible flag pairs:
>> +
>> + * --preserve-merges and --interactive
>> + * --preserve-merges and --signoff
>> + * --preserve-merges and --rebase-merges
>> + * --rebase-merges and --strategy
>> + * --rebase-merges and --strategy-option
>> +
>>  include::merge-strategies.txt[]
>
> It is a bit unclear what these lists want to say.  If --ignore-date
> is only understood by "rebase" and not "rebase -i", wouldn't that
> make "--interactive and --ignore-date" an incompatible flag pair?
> Or do you mean the "Other incompatible" list to enumerate the ones
> that are explicitly rejected, as opposed to getting silently
> ignored?

Yes, --interactive and --ignore-date are an incompatible flag pair but
I didn't want to list all 65 of those pairs explicitly.  Anything that
only works by being passed to git-am is incompatible with anything
that implies the --merge or --interactive backends.

The list becomes simpler once we nuke git-rebase--merge.sh,
implementing it atop git-rebase--interactive.sh (which comes in a
separate series that depends on this one).

> In any case, I'll need to read through the remainder of the series
> to come to any conclusion, but given that -p is getting killed by
> its primary author, with help from others to split out its
> implementation from the --interactive codepath, it might be a bad
> time to do this series.  Thanks for working on this topic.

This series doesn't conflict with ag/rebase-p at all; it merges
cleanly with next and pu.  The follow-on work to nuke
git-rebase--merge.sh, however, does conflict so I'm waiting on
(re-)sending it.

Re: [PATCH v3 2/7] git-rebase.sh: update help messages a bit

[Elijah Newren]

On Thu, Jun 21, 2018 at 12:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> signoff is not specific to the am-backend.  Also, re-order a few options
>> to make like things (e.g. strategy and strategy-option) be near each
>> other.
>> ---
>>  git-rebase.sh | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> Missing sign-off, but otherwise looks good.

Oops.

Signed-off-by: Elijah Newren <newren@gmail.com>

Re: [PATCH v3 3/7] t3422: new testcases for checking when incompatible options passed

[Elijah Newren]

On Thu, Jun 21, 2018 at 1:04 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> +     git checkout B &&
>> +     # This is indented with HT SP HT.
>> +     echo "          foo();" >>foo &&
>
> I often wonder, whenever I see a need for a comment like this, if
> saying the same thing in code to make it visible is cleaner and less
> error prone way to do so, i.e. e.g.
>
>         echo "_ _foo();" | tr "_" "\011" >>foo &&

I can make that change.  Should I make the same change to
t4015-diff-whitespace.sh, where I copied that comment and line of code
from?  (In a different topic submission, of course.)

>> +# Rebase has lots of useful options like --whitepsace=fix, which are
>> +# actually all built in terms of flags to git-am.  Since neither
>> +# --merge nor --interactive (nor any options that imply those two) use
>> +# git-am, using them together will result in flags like --whitespace=fix
>> +# being ignored.  Make sure rebase warns the user and aborts instead.
>> +#
>> +
>> +test_run_rebase () {
>> +     opt=$1
>> +     shift
>> +     test_expect_failure "$opt incompatible with --merge" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --merge A
>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --strategy=ours" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --strategy=ours A
>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --strategy-option=ours" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --strategy=ours A
>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --interactive" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --interactive A
>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --exec" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --exec 'true' A
>> +     "
>> +
>> +}
>> +
>> +test_run_rebase --whitespace=fix
>> +test_run_rebase --ignore-whitespace
>> +test_run_rebase --committer-date-is-author-date
>> +test_run_rebase -C4
>
> I happen to be from old school and "rebase" primarily means
> "format-patch piped to am" in my mind, so from that point of view,
> "test_run_rebase --OPT" that says "--OPT which is a valid option for
> the primary operating mode of rebase does not work with the other
> exotic modes of the command" is not all that bad, but I do not think
> that worldview holds for many people in general.  Perhaps calling it
> something like "test_rebase_am_only" makes the intent clearer?

Well, if that's what rebase means in your mind, then I'm sure you'll
have fun commenting on my follow-on series.  :-)

Anyway, sure, I'm happy to change the function name.

Re: [PATCH v3 4/7] git-rebase: error out when incompatible options passed

[Elijah Newren]

On Thu, Jun 21, 2018 at 1:29 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> +if test -n "$git_am_opt"; then
>> +     incompatible_opts=$(echo "$git_am_opt" | sed -e 's/ -q//')
>
> This is probably just a taste-thing, but I'd probably prefer to see
> the "sed" filter out "-q" alone, and make an effort to leave future
> things like "-qx" alone, e.g.
>
>     $(echo " $git_am_opt " | sed -e 's/ -q / /g' -e 's/^ \(.*\) $/\1/')
>
> or something like that.

Makes sense.

>> +     if test -n "$interactive_rebase"
>> +     then
>> +             if test -n "$incompatible_opts"
>> +             then
>> +                     die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
>> +             fi
>> +     fi
>> +     if test -n "$do_merge"; then
>> +             if test -n "$incompatible_opts"
>> +             then
>> +                     die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
>> +             fi
>> +     fi
>> +fi
>
> Not making --merge and --interactive incompatible as the proposed
> log message said makes this hunk at manageable complexity, I guess
> ;-)

Right, and since --interactive essentially handles --merge just fine
and I'm planning to soon simplify anyway[1], at least once ag/rebase-p
merges to master, I didn't want to make it any more complex.

[1] https://public-inbox.org/git/20180607171344.23331-3-newren@gmail.com/

Re: [PATCH v3 5/7] git-rebase.txt: document behavioral inconsistencies between modes

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> +BEHAVIORAL INCONSISTENCIES
> +--------------------------
> +
> +  * --no-ff vs. --force-rebase

Do we want to `--quote` these?

> +    These options are actually identical, though their description
> +    leads people to believe they might not be.

Perhaps the same bandwidth can be spent on improving their
description, instead of adding this paragraph?

> + * empty commits:
> +
> +    am-based rebase will drop any "empty" commits, whether the
> +    commit started empty (had no changes relative to its parent to
> +    start with) or ended empty (all changes were already applied
> +    upstream in other commits).
> +
> +    merge-based rebase does the same.
> +
> +    interactive-based rebase will by default drop commits that
> +    started empty and halt if it hits a commit that ended up empty.

I think the description is accurate.

WRT a change that ends up being empty (as opposed to a change that
is empty from the beginning), I'd think that the current behaviour
is desireable one.  "am" based rebase is solely to transplant an
existing history and want to stop much less than "interactive" one
whose purpose is to polish a series before making it publishable,
and asking for confirmation ("this has become empty--do you want to
drop it?") is more appropriate from the workflow point of view; so
it may deserve s/inconsistencies/differences/.

> +    The --keep-empty option exists for interactive rebases to allow
> +    it to keep commits that started empty.

On the other hand, lack of --keep-empty on the "am" side is probably
a bug that wants to be fixed.

> +  * empty commit messages:
> +
> +    am-based rebase will silently apply commits with empty commit
> +    messages.
> +
> +    merge-based and interactive-based rebases will by default halt
> +    on any such commits.  The --allow-empty-message option exists to
> +    allow interactive-based rebases to apply such commits without
> +    halting.

Ditto for desirable difference coming from workflow point of view.

> +  * directory rename detection:
> +
> +    merge-based and interactive-based rebases work fine with
> +    directory rename detection.  am-based rebases sometimes do not.

I quite do not get what the big deal is with "directory rename"; it
is merely a degree of magic heuristics employed while detecting file
renames.  It is natural to expect that the less information you make
available to the machinery, the less amount of heuristics gets
exercised to make "magic" happen.  Is the internal implementation
detail described below (omitted) all that interesting to general
readers of "git rebase --help", I wonder?  I would understand it if
this were under Documentation/technical/, though.

Re: [PATCH v3 6/7] git-rebase.txt: address confusion between --no-ff vs --force-rebase

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> rebase was taught the --force-rebase option in commit b2f82e05de ("Teach
> rebase to rebase even if upstream is up to date", 2009-02-13).  This flag
> worked for the am and merge backends, but wasn't a valid option for the
> interactive backend.
> ...
>  INCOMPATIBLE OPTIONS
>  --------------------
>  
> @@ -559,11 +549,6 @@ Other incompatible flag pairs:
>  BEHAVIORAL INCONSISTENCIES
>  --------------------------
>  
> -  * --no-ff vs. --force-rebase
> -
> -    These options are actually identical, though their description
> -    leads people to believe they might not be.

Ah, I should have held off my review of 5/7 before reading this one.
Perhaps we want to do this before the step 5/7 to reduce the churn?

Re: [PATCH v3 3/7] t3422: new testcases for checking when incompatible options passed

[Eric Sunshine]

On Thu, Jun 21, 2018 at 4:05 PM Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
> > +     # This is indented with HT SP HT.
> > +     echo "          foo();" >>foo &&
>
> I often wonder, whenever I see a need for a comment like this, if
> saying the same thing in code to make it visible is cleaner and less
> error prone way to do so, i.e. e.g.
>
>         echo "_ _foo();" | tr "_" "\011" >>foo &&

Or use q_to_tab() from test-lib-functions.h:

    echo "q qfoo();" | q_to_tab

Re: [PATCH v3 6/7] git-rebase.txt: address confusion between --no-ff vs --force-rebase

[Elijah Newren]

On Thu, Jun 21, 2018 at 1:46 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> rebase was taught the --force-rebase option in commit b2f82e05de ("Teach
>> rebase to rebase even if upstream is up to date", 2009-02-13).  This flag
>> worked for the am and merge backends, but wasn't a valid option for the
>> interactive backend.
>> ...
>>  INCOMPATIBLE OPTIONS
>>  --------------------
>>
>> @@ -559,11 +549,6 @@ Other incompatible flag pairs:
>>  BEHAVIORAL INCONSISTENCIES
>>  --------------------------
>>
>> -  * --no-ff vs. --force-rebase
>> -
>> -    These options are actually identical, though their description
>> -    leads people to believe they might not be.
>
> Ah, I should have held off my review of 5/7 before reading this one.
> Perhaps we want to do this before the step 5/7 to reduce the churn?

Sure, I can switch the order around.

Re: [PATCH v3 5/7] git-rebase.txt: document behavioral inconsistencies between modes

[Elijah Newren]

Hi Junio,

On Thu, Jun 21, 2018 at 1:44 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> +BEHAVIORAL INCONSISTENCIES
>> +--------------------------
>> +
>> +  * --no-ff vs. --force-rebase
>
> Do we want to `--quote` these?

Ah, good point.

>> + * empty commits:
>> +
>> +    am-based rebase will drop any "empty" commits, whether the
>> +    commit started empty (had no changes relative to its parent to
>> +    start with) or ended empty (all changes were already applied
>> +    upstream in other commits).
>> +
>> +    merge-based rebase does the same.
>> +
>> +    interactive-based rebase will by default drop commits that
>> +    started empty and halt if it hits a commit that ended up empty.
>
> I think the description is accurate.
>
> WRT a change that ends up being empty (as opposed to a change that
> is empty from the beginning), I'd think that the current behaviour
> is desireable one.  "am" based rebase is solely to transplant an
> existing history and want to stop much less than "interactive" one
> whose purpose is to polish a series before making it publishable,
> and asking for confirmation ("this has become empty--do you want to
> drop it?") is more appropriate from the workflow point of view; so
> it may deserve s/inconsistencies/differences/.

Are you arguing the default for an explicit interactive rebase, or
also an implied one?

I would argue that implied interactive rebases also solely have the
purpose of "transplanting an existing history", as you say.  Thus, if
the user passes --rebase-merges (or --strategy or --strategy-options
after git-rebase--merge becomes part of git-rebase--interactive), then
the default should be the same as for am-based rebase, i.e. just
silently drop the empty commits.

>> +    The --keep-empty option exists for interactive rebases to allow
>> +    it to keep commits that started empty.
>
> On the other hand, lack of --keep-empty on the "am" side is probably
> a bug that wants to be fixed.

I personally don't like --keep-empty; it's too granular.  I'd prefer a
  --empty={drop,halt,keep}
flag, and have it apply to both patches that were empty when the
rebase started as well as patches that became empty after application
on the new base.  I think that'd be useful for both am-based and
interactive-based rebases.

Then we could also make an explicit --interactive flag imply
--empty=halt, if so desired, and otherwise make --empty=drop the
default.

I'm interested in implementing that, if other folks think it's not crazy.

Thoughts?

>> +  * empty commit messages:
>> +
>> +    am-based rebase will silently apply commits with empty commit
>> +    messages.
>> +
>> +    merge-based and interactive-based rebases will by default halt
>> +    on any such commits.  The --allow-empty-message option exists to
>> +    allow interactive-based rebases to apply such commits without
>> +    halting.
>
> Ditto for desirable difference coming from workflow point of view.

I can see the argument for having a difference for rebases if the
interactive rebase was explicit, though I disagree with the argument.
See https://public-inbox.org/git/CABPp-BHrcUHX_zHxpojV5=sxJ1=NoDg9uhxv+NH5BsHsQYavPQ@mail.gmail.com/
for my views on this.

However, I'm curious again -- is your argument only for
interactive_rebase=explicit or also for interactive_rebase=implied?

>> +  * directory rename detection:
>> +
>> +    merge-based and interactive-based rebases work fine with
>> +    directory rename detection.  am-based rebases sometimes do not.
>
> I quite do not get what the big deal is with "directory rename"; it
> is merely a degree of magic heuristics employed while detecting file
> renames.  It is natural to expect that the less information you make
> available to the machinery, the less amount of heuristics gets
> exercised to make "magic" happen.  Is the internal implementation
> detail described below (omitted) all that interesting to general
> readers of "git rebase --help", I wonder?  I would understand it if
> this were under Documentation/technical/, though.

You are right that the last two paragraphs are probably much too
technical and belong elsewhere; I can move them.  However, I am not
sure if you were requesting just those paragraphs be moved, or if you
wanted the whole "directory rename detection" section removed from the
manpage.  Could you clarify?  Also, if the latter...How would users
know how much "information is made available to the underlying
machinery", as you put it?  Or, at a higher level, what is the right
way to communicate to them how they can obtain a behavior they might
want (directory rename detection), if it's not mentioned somewhere in
the manpage?  Is there an alternate place to put this in the document?


Thanks,
Elijah

Re: [PATCH v3 1/7] git-rebase.txt: document incompatible options

[SZEDER Gábor]

> @@ -487,6 +510,52 @@ recreates the topic branch with fresh commits so it can be remerged
>  successfully without needing to "revert the reversion" (see the
>  link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
>  
> +INCOMPATIBLE OPTIONS
> +--------------------
> +
> +git-rebase has many flags that are incompatible with each other,
> +predominantly due to the fact that it has three different underlying
> +implementations:
> +
> + * one based on linkgit:git-am[1] (the default)
> + * one based on linkgit:git-merge-recursive[1] (merge backend)

Referencing a non-existing man page via the 'linkgit' macro causes
'make check-docs' to complain:

      LINT lint-docs
  ./git-rebase.txt:511: no such source: git-merge-recursive[1]

[PATCH v4 0/9] Document/fix/warn about rebase incompatibilities and inconsistencies

[Elijah Newren]

git-rebase has lots of options that are mutually incompatible.  Even among
aspects of its behavior that is common to all rebase types, it has a number
of inconsistencies.  This series tries to document, fix, and/or warn users
about many of these.

Changes since v3 (range-diff included at the end):
  * Fix many small issues mentioned by Junio and SZEDER, including a fix
    suggested by Eric.
  * Add a new patch (#6) which introduces technical documentation on directory
    rename detection, so that...
  * The more technical portion of the docs from patch 7 (used to be patch 5
    in v3) can be moved from git-rebase.txt to the new
    technical/directory-rename-detection.txt, as suggested by Junio.
  * Also adds a new patch (#8) which adds testcases to demonstrate what the
    new technical doc discusses.

Items of particular note for reviewers:
  * I have left patch 9 (used to be patch 7) in RFC state, but
    expanded the commit message with an in-depth usability rationale
    for the change.
  * It sounded like Junio was slightly unclear about the intent of the
    wording in Patch 1.  Not sure if my answer (in email) was sufficient or
    if there are changes I should make to the patch.
  * On Patch 5 of v3 (which is now patch 7 of v4), Junio suggested a course
    of action to take with --keep-empty.  I counter-proposed with a
    --empty={drop,halt,keep} idea.  Is just deferring this issue to a later
    series okay, or are there changes I should make to the current series to
    handle this?  (If so, what changes should those be?)

Elijah Newren (9):
  git-rebase.txt: document incompatible options
  git-rebase.sh: update help messages a bit
  t3422: new testcases for checking when incompatible options passed
  git-rebase: error out when incompatible options passed
  git-rebase.txt: address confusion between --no-ff vs --force-rebase
  directory-rename-detection.txt: technical docs on abilities and
    limitations
  git-rebase.txt: document behavioral differences between modes
  t3401: add directory rename testcases for rebase and am
  git-rebase: make --allow-empty-message the default

 Documentation/git-rebase.txt                  | 135 ++++++++++++++----
 .../technical/directory-rename-detection.txt  | 115 +++++++++++++++
 git-rebase.sh                                 |  43 +++++-
 t/t3401-rebase-and-am-rename.sh               | 105 ++++++++++++++
 t/t3404-rebase-interactive.sh                 |   7 +-
 t/t3405-rebase-malformed.sh                   |  11 +-
 t/t3422-rebase-incompatible-options.sh        |  88 ++++++++++++
 7 files changed, 462 insertions(+), 42 deletions(-)
 create mode 100644 Documentation/technical/directory-rename-detection.txt
 create mode 100755 t/t3401-rebase-and-am-rename.sh
 create mode 100755 t/t3422-rebase-incompatible-options.sh

 1: 4cdf9130cc !  1: 3f454ebc5e git-rebase.txt: document incompatible options
    @@ -163,7 +163,7 @@
     +implementations:
     +
     + * one based on linkgit:git-am[1] (the default)
    -+ * one based on linkgit:git-merge-recursive[1] (merge backend)
    ++ * one based on git-merge-recursive (merge backend)
     + * one based on linkgit:git-cherry-pick[1] (interactive backend)
     +
     +Flags only understood by the am backend:
 2: e336f76c5e !  2: 31a5a071a6 git-rebase.sh: update help messages a bit
    @@ -6,6 +6,8 @@
         to make like things (e.g. strategy and strategy-option) be near each
         other.
     
    +    Signed-off-by: Elijah Newren <newren@gmail.com>
    +
     diff --git a/git-rebase.sh b/git-rebase.sh
     --- a/git-rebase.sh
     +++ b/git-rebase.sh
 3: 4ab38d8a5f !  3: 5a2b5eec79 t3422: new testcases for checking when incompatible options passed
    @@ -34,8 +34,7 @@
     +	git commit -m A &&
     +
     +	git checkout B &&
    -+	# This is indented with HT SP HT.
    -+	echo "	 	foo();" >>foo &&
    ++	echo "q qfoo();" | q_to_tab >>foo &&
     +	git add foo &&
     +	git commit -m B
     +'
    @@ -48,7 +47,7 @@
     +# being ignored.  Make sure rebase warns the user and aborts instead.
     +#
     +
    -+test_run_rebase () {
    ++test_rebase_am_only () {
     +	opt=$1
     +	shift
     +	test_expect_failure "$opt incompatible with --merge" "
    @@ -78,10 +77,10 @@
     +
     +}
     +
    -+test_run_rebase --whitespace=fix
    -+test_run_rebase --ignore-whitespace
    -+test_run_rebase --committer-date-is-author-date
    -+test_run_rebase -C4
    ++test_rebase_am_only --whitespace=fix
    ++test_rebase_am_only --ignore-whitespace
    ++test_rebase_am_only --committer-date-is-author-date
    ++test_rebase_am_only -C4
     +
     +test_expect_success '--preserve-merges incompatible with --signoff' '
     +	git checkout B^0 &&
 4: 5223954caf !  4: 1e1c83724a git-rebase: error out when incompatible options passed
    @@ -37,7 +37,8 @@
      fi
      
     +if test -n "$git_am_opt"; then
    -+	incompatible_opts=$(echo "$git_am_opt" | sed -e 's/ -q//')
    ++	incompatible_opts=$(echo " $git_am_opt " | \
    ++			    sed -e 's/ -q / /g' -e 's/^ \(.*\) $/\1/')
     +	if test -n "$interactive_rebase"
     +	then
     +		if test -n "$incompatible_opts"
    @@ -85,7 +86,7 @@
     --- a/t/t3422-rebase-incompatible-options.sh
     +++ b/t/t3422-rebase-incompatible-options.sh
     @@
    - test_run_rebase () {
    + test_rebase_am_only () {
      	opt=$1
      	shift
     -	test_expect_failure "$opt incompatible with --merge" "
 5: 96f7ba98bc < --: ---------- git-rebase.txt: document behavioral inconsistencies between modes
 6: 7bb7b380ac !  5: 51023269d3 git-rebase.txt: address confusion between --no-ff vs --force-rebase
    @@ -93,15 +93,3 @@
      INCOMPATIBLE OPTIONS
      --------------------
      
    -@@
    - BEHAVIORAL INCONSISTENCIES
    - --------------------------
    - 
    --  * --no-ff vs. --force-rebase
    --
    --    These options are actually identical, though their description
    --    leads people to believe they might not be.
    --
    -  * empty commits:
    - 
    -     am-based rebase will drop any "empty" commits, whether the
--: ---------- >  6: f017d45dd9 directory-rename-detection.txt: technical docs on abilities and limitations
--: ---------- >  7: 0a359df404 git-rebase.txt: document behavioral differences between modes
--: ---------- >  8: beaadceaef t3401: add directory rename testcases for rebase and am
 7: 3ed07548a6 !  9: 431b2c36d5 git-rebase: make --allow-empty-message the default
    @@ -2,13 +2,85 @@
     
         git-rebase: make --allow-empty-message the default
     
    -    All rebase backends should behave the same with empty commit messages, but
    -    currently do not.  am-based rebases already apply commits with an empty
    -    commit message without stopping or requiring the user to specify an extra
    -    flag.  Since am-based rebases are the default rebase type, and since it
    -    appears no one has ever requested a --no-allow-empty-message flag to
    -    change this behavior, make --allow-empty-message the default so that
    -    merge-based and interactive-based rebases will behave the same.
    +    rebase backends currently behave differently with empty commit messages,
    +    largely as a side-effect of the different underlying commands on which
    +    they are based.  am-based rebases apply commits with an empty commit
    +    message without stopping or requiring the user to specify an extra flag.
    +    (It is interesting to note that am-based rebases are the default rebase
    +    type, and no one has ever requested a --no-allow-empty-message flag to
    +    change this behavior.)  merge-based and interactive-based rebases (which
    +    are ultimately based on git-commit), will currently halt on any such
    +    commits and require the user to manually specify what to do with the
    +    commit and continue.
    +
    +    One possible rationale for the difference in behavior is that the purpose
    +    of an "am" based rebase is solely to transplant an existing history, while
    +    an "interactive" rebase is one whose purpose is to polish a series before
    +    making it publishable.  Thus, stopping and asking for confirmation for a
    +    possible problem is more appropriate in the latter case.  However, there
    +    are two problems with this rationale:
    +
    +      1) merge-based rebases are also non-interactive and there are multiple
    +         types of rebases that use the interactive machinery but are not
    +         explicitly interactive (e.g. when either --rebase-merges or
    +         --keep-empty are specified without --interactive).  These rebases are
    +         also used solely to transplant an existing history, and thus also
    +         should default to --allow-empty-message.
    +
    +      2) this rationale only says that the user is more accepting of stopping
    +         in the case of an explicitly interactive rebase, not that stopping
    +         for this particular reason actually makes sense.  Exploring whether
    +         it makes sense, requires backing up and analyzing the underlying
    +         commands...
    +
    +    If git-commit did not error out on empty commits by default, accidental
    +    creation of commits with empty messages would be a very common occurrence
    +    (this check has caught me many times).  Further, nearly all such empty
    +    commit messages would be considered an accidental error (as evidenced by a
    +    huge amount of documentation across version control systems and in various
    +    blog posts explaining how important commit messages are).  A simple check
    +    for what would otherwise be a common error thus made a lot of sense, and
    +    git-commit gained an --allow-empty-message flag for special case
    +    overrides.  This has made commits with empty messages very rare.
    +
    +    There are two sources for commits with empty messages for rebase (and
    +    cherry-pick): (a) commits created in git where the user previously
    +    specified --allow-empty-message to git-commit, and (b) commits imported
    +    into git from other version control systems.  In case (a), the user has
    +    already explicitly specified that there is something special about this
    +    commit that makes them not want to specify a commit message; forcing them
    +    to re-specify with every cherry-pick or rebase seems more likely to be
    +    infuriating than helpful.  In case (b), the commit is highly unlikely to
    +    have been authored by the person who has imported the history and is doing
    +    the rebase or cherry-pick, and thus the user is unlikely to be the
    +    appropriate person to write a commit message for it.  Stopping and
    +    expecting the user to modify the commit before proceeding thus seems
    +    counter-productive.
    +
    +    Further, note that while empty commit messages was a common error case for
    +    git-commit to deal with, it is a rare case for rebase (or cherry-pick).
    +    The fact that it is rare raises the question of why it would be worth
    +    checking and stopping on this particular condition and not others.  For
    +    example, why doesn't an interactive rebase automatically stop if the
    +    commit message's first line is 2000 columns long, or is missing a blank
    +    line after the first line, or has every line indented with five spaces, or
    +    any number of other myriad problems?
    +
    +    Finally, note that if a user doing an interactive rebase does have the
    +    necessary knowledge to add a message for any such commit and wants to do
    +    so, it is rather simple for them to change the appropriate line from
    +    'pick' to 'reword'.  The fact that the subject is empty in the todo list
    +    that the user edits should even serve as a way to notify them.
    +
    +    As far as I can tell, the fact that merge-based and interactive-based
    +    rebases stop on commits with empty commit messages is solely a by-product
    +    of having been based on git-commit.  It went without notice for a long
    +    time precisely because such cases are rare.  The rareness of this
    +    situation made it difficult to reason about, so when folks did eventually
    +    notice this behavior, they assumed it was there for a good reason and just
    +    added an --allow-empty-message flag.  In my opinion, stopping on such
    +    messages not desirable in any of these cases, even the (explicitly)
    +    interactive case.
     
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
    @@ -16,7 +88,7 @@
     --- a/Documentation/git-rebase.txt
     +++ b/Documentation/git-rebase.txt
     @@
    -     The --keep-empty option exists for interactive rebases to allow
    +     The `--keep-empty` option exists for interactive rebases to allow
          it to keep commits that started empty.
      
     -  * empty commit messages:
    @@ -25,7 +97,7 @@
     -    messages.
     -
     -    merge-based and interactive-based rebases will by default halt
    --    on any such commits.  The --allow-empty-message option exists to
    +-    on any such commits.  The `--allow-empty-message` option exists to
     -    allow interactive-based rebases to apply such commits without
     -    halting.
     -

-- 
2.18.0.9.g678597d97e

[PATCH v4 1/9] git-rebase.txt: document incompatible options

[Elijah Newren]

git rebase has many options that only work with one of its three backends.
It also has a few other pairs of incompatible options.  Document these.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 85 ++++++++++++++++++++++++++++++++----
 1 file changed, 77 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 0e20a66e73..b2d95e3fb9 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -243,11 +243,15 @@ leave out at most one of A and B, in which case it defaults to HEAD.
 --keep-empty::
 	Keep the commits that do not change anything from its
 	parents in the result.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --allow-empty-message::
 	By default, rebasing commits with an empty message will fail.
 	This option overrides that behavior, allowing commits with empty
 	messages to be rebased.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --skip::
 	Restart the rebasing process by skipping the current patch.
@@ -271,6 +275,8 @@ branch on top of the <upstream> branch.  Because of this, when a merge
 conflict happens, the side reported as 'ours' is the so-far rebased
 series, starting with <upstream>, and 'theirs' is the working branch.  In
 other words, the sides are swapped.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -s <strategy>::
 --strategy=<strategy>::
@@ -280,8 +286,10 @@ other words, the sides are swapped.
 +
 Because 'git rebase' replays each commit from the working branch
 on top of the <upstream> branch using the given strategy, using
-the 'ours' strategy simply discards all patches from the <branch>,
+the 'ours' strategy simply empties all patches from the <branch>,
 which makes little sense.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -X <strategy-option>::
 --strategy-option=<strategy-option>::
@@ -289,6 +297,8 @@ which makes little sense.
 	This implies `--merge` and, if no strategy has been
 	specified, `-s recursive`.  Note the reversal of 'ours' and
 	'theirs' as noted above for the `-m` option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -S[<keyid>]::
 --gpg-sign[=<keyid>]::
@@ -324,6 +334,8 @@ which makes little sense.
 	and after each change.  When fewer lines of surrounding
 	context exist they all must match.  By default no context is
 	ever ignored.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -f::
 --force-rebase::
@@ -355,19 +367,22 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 --whitespace=<option>::
 	These flag are passed to the 'git apply' program
 	(see linkgit:git-apply[1]) that applies the patch.
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --committer-date-is-author-date::
 --ignore-date::
 	These flags are passed to 'git am' to easily change the dates
 	of the rebased commits (see linkgit:git-am[1]).
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --signoff::
 	Add a Signed-off-by: trailer to all the rebased commits. Note
 	that if `--interactive` is given then only commits marked to be
-	picked, edited or reworded will have the trailer added. Incompatible
-	with the `--preserve-merges` option.
+	picked, edited or reworded will have the trailer added.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -i::
 --interactive::
@@ -378,6 +393,8 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -r::
 --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
@@ -404,7 +421,7 @@ It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
 explicit `exec git merge -s <strategy> [...]` commands.
 +
-See also REBASING MERGES below.
+See also REBASING MERGES and INCOMPATIBLE OPTIONS below.
 
 -p::
 --preserve-merges::
@@ -415,6 +432,8 @@ See also REBASING MERGES below.
 This uses the `--interactive` machinery internally, but combining it
 with the `--interactive` option explicitly is generally not a good
 idea unless you know what you are doing (see BUGS below).
++
+See also INCOMPATIBLE OPTIONS below.
 
 -x <cmd>::
 --exec <cmd>::
@@ -437,6 +456,8 @@ squash/fixup series.
 +
 This uses the `--interactive` machinery internally, but it can be run
 without an explicit `--interactive`.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --root::
 	Rebase all commits reachable from <branch>, instead of
@@ -447,6 +468,8 @@ without an explicit `--interactive`.
 	When used together with both --onto and --preserve-merges,
 	'all' root commits will be rewritten to have <newbase> as parent
 	instead.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autosquash::
 --no-autosquash::
@@ -461,11 +484,11 @@ without an explicit `--interactive`.
 	too.  The recommended way to create fixup/squash commits is by using
 	the `--fixup`/`--squash` options of linkgit:git-commit[1].
 +
-This option is only valid when the `--interactive` option is used.
-+
 If the `--autosquash` option is enabled by default using the
 configuration variable `rebase.autoSquash`, this option can be
 used to override and disable this setting.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autostash::
 --no-autostash::
@@ -487,6 +510,52 @@ recreates the topic branch with fresh commits so it can be remerged
 successfully without needing to "revert the reversion" (see the
 link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
 
+INCOMPATIBLE OPTIONS
+--------------------
+
+git-rebase has many flags that are incompatible with each other,
+predominantly due to the fact that it has three different underlying
+implementations:
+
+ * one based on linkgit:git-am[1] (the default)
+ * one based on git-merge-recursive (merge backend)
+ * one based on linkgit:git-cherry-pick[1] (interactive backend)
+
+Flags only understood by the am backend:
+
+ * --committer-date-is-author-date
+ * --ignore-date
+ * --whitespace
+ * --ignore-whitespace
+ * -C
+
+Flags understood by both merge and interactive backends:
+
+ * --merge
+ * --strategy
+ * --strategy-option
+ * --allow-empty-message
+
+Flags only understood by the interactive backend:
+
+ * --[no-]autosquash
+ * --rebase-merges
+ * --preserve-merges
+ * --interactive
+ * --exec
+ * --keep-empty
+ * --autosquash
+ * --edit-todo
+ * --root when used in combination with --onto
+
+Other incompatible flag pairs:
+
+ * --preserve-merges and --interactive
+ * --preserve-merges and --signoff
+ * --preserve-merges and --rebase-merges
+ * --rebase-merges and --strategy
+ * --rebase-merges and --strategy-option
+
 include::merge-strategies.txt[]
 
 NOTES
-- 
2.18.0.9.g678597d97e

[PATCH v4 2/9] git-rebase.sh: update help messages a bit

[Elijah Newren]

signoff is not specific to the am-backend.  Also, re-order a few options
to make like things (e.g. strategy and strategy-option) be near each
other.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 40be59ecc4..bf71b7fa20 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -20,23 +20,23 @@ onto=!             rebase onto given branch instead of upstream
 r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
+X,strategy-option=! pass the argument through to the merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
+f,force-rebase!    cherry-pick all commits, even if unchanged
 m,merge!           use merging strategies to rebase
 i,interactive!     let the user edit the list of commits to rebase
 x,exec=!           add exec lines after each commit of the editable list
 k,keep-empty	   preserve empty commits during rebase
 allow-empty-message allow rebasing commits with empty messages
-f,force-rebase!    force rebase even if branch is up to date
-X,strategy-option=! pass the argument through to the merge strategy
 stat!              display a diffstat of what changed upstream
 n,no-stat!         do not show diffstat of what changed upstream
 verify             allow pre-rebase hook to run
 rerere-autoupdate  allow rerere to update index with resolved conflicts
 root!              rebase all reachable commits up to the root(s)
 autosquash         move commits that begin with squash!/fixup! under -i
+signoff            add a Signed-off-by: line to each commit
 committer-date-is-author-date! passed to 'git am'
 ignore-date!       passed to 'git am'
-signoff            passed to 'git am'
 whitespace=!       passed to 'git apply'
 ignore-whitespace! passed to 'git apply'
 C=!                passed to 'git apply'
-- 
2.18.0.9.g678597d97e

[PATCH v4 3/9] t3422: new testcases for checking when incompatible options passed

[Elijah Newren]

git rebase is split into three types: am, merge, and interactive.  Various
options imply different types, and which mode we are using determine which
sub-script (git-rebase--$type) is executed to finish the work.  Not all
options work with all types, so add tests for combinations where we expect
to receive an error rather than having options be silently ignored.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3422-rebase-incompatible-options.sh | 88 ++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
 create mode 100755 t/t3422-rebase-incompatible-options.sh

diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
new file mode 100755
index 0000000000..2687a85254
--- /dev/null
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -0,0 +1,88 @@
+#!/bin/sh
+
+test_description='test if rebase detects and aborts on incompatible options'
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	test_seq 2 9 >foo &&
+	git add foo &&
+	git commit -m orig &&
+
+	git branch A &&
+	git branch B &&
+
+	git checkout A &&
+	test_seq 1 9 >foo &&
+	git add foo &&
+	git commit -m A &&
+
+	git checkout B &&
+	echo "q qfoo();" | q_to_tab >>foo &&
+	git add foo &&
+	git commit -m B
+'
+
+#
+# Rebase has lots of useful options like --whitepsace=fix, which are
+# actually all built in terms of flags to git-am.  Since neither
+# --merge nor --interactive (nor any options that imply those two) use
+# git-am, using them together will result in flags like --whitespace=fix
+# being ignored.  Make sure rebase warns the user and aborts instead.
+#
+
+test_rebase_am_only () {
+	opt=$1
+	shift
+	test_expect_failure "$opt incompatible with --merge" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --merge A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --interactive" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --interactive A
+	"
+
+	test_expect_failure "$opt incompatible with --exec" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --exec 'true' A
+	"
+
+}
+
+test_rebase_am_only --whitespace=fix
+test_rebase_am_only --ignore-whitespace
+test_rebase_am_only --committer-date-is-author-date
+test_rebase_am_only -C4
+
+test_expect_success '--preserve-merges incompatible with --signoff' '
+	git checkout B^0 &&
+	test_must_fail git rebase --preserve-merges --signoff A
+'
+
+test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
+	git checkout B^0 &&
+	test_must_fail git rebase --preserve-merges --rebase-merges A
+'
+
+test_expect_failure '--rebase-merges incompatible with --strategy' '
+	git checkout B^0 &&
+	test_must_fail git rebase --rebase-merges -s resolve A
+'
+
+test_expect_failure '--rebase-merges incompatible with --strategy-option' '
+	git checkout B^0 &&
+	test_must_fail git rebase --rebase-merges -Xignore-space-change A
+'
+
+test_done
-- 
2.18.0.9.g678597d97e

[PATCH v4 4/9] git-rebase: error out when incompatible options passed

[Elijah Newren]

git rebase has three different types: am, merge, and interactive, all of
which are implemented in terms of separate scripts.  am builds on git-am,
merge builds on git-merge-recursive, and interactive builds on
git-cherry-pick.  We make use of features in those lower-level commands in
the different rebase types, but those features don't exist in all of the
lower level commands so we have a range of incompatibilities.  Previously,
we just accepted nearly any argument and silently ignored whichever ones
weren't implemented for the type of rebase specified.  Change this so the
incompatibilities are documented, included in the testsuite, and tested
for at runtime with an appropriate error message shown.

Some exceptions I left out:

  * --merge and --interactive are technically incompatible since they are
    supposed to run different underlying scripts, but with a few small
    changes, --interactive can do everything that --merge can.  In fact,
    I'll shortly be sending another patch to remove git-rebase--merge and
    reimplement it on top of git-rebase--interactive.

  * One could argue that --interactive and --quiet are incompatible since
    --interactive doesn't implement a --quiet mode (perhaps since
    cherry-pick itself does not implement one).  However, the interactive
    mode is more quiet than the other modes in general with progress
    messages, so one could argue that it's already quiet.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh                          | 35 ++++++++++++++++++++++++++
 t/t3422-rebase-incompatible-options.sh | 16 ++++++------
 2 files changed, 43 insertions(+), 8 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index bf71b7fa20..18ac8226c4 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -503,6 +503,24 @@ then
 	git_format_patch_opt="$git_format_patch_opt --progress"
 fi
 
+if test -n "$git_am_opt"; then
+	incompatible_opts=$(echo " $git_am_opt " | \
+			    sed -e 's/ -q / /g' -e 's/^ \(.*\) $/\1/')
+	if test -n "$interactive_rebase"
+	then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
+		fi
+	fi
+	if test -n "$do_merge"; then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
+		fi
+	fi
+fi
+
 if test -n "$signoff"
 then
 	test -n "$preserve_merges" &&
@@ -511,6 +529,23 @@ then
 	force_rebase=t
 fi
 
+if test -n "$preserve_merges"
+then
+	# Note: incompatibility with --signoff handled in signoff block above
+	# Note: incompatibility with --interactive is just a strong warning;
+	#       git-rebase.txt caveats with "unless you know what you are doing"
+	test -n "$rebase_merges" &&
+		die "$(gettext "error: cannot combine '--preserve_merges' with '--rebase-merges'")"
+fi
+
+if test -n "$rebase_merges"
+then
+	test -n "$strategy_opts" &&
+		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy-option'")"
+	test -n "$strategy" &&
+		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy'")"
+fi
+
 if test -z "$rebase_root"
 then
 	case "$#" in
diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
index 2687a85254..fc0ce150d1 100755
--- a/t/t3422-rebase-incompatible-options.sh
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -33,27 +33,27 @@ test_expect_success 'setup' '
 test_rebase_am_only () {
 	opt=$1
 	shift
-	test_expect_failure "$opt incompatible with --merge" "
+	test_expect_success "$opt incompatible with --merge" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --merge A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy=ours" "
+	test_expect_success "$opt incompatible with --strategy=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+	test_expect_success "$opt incompatible with --strategy-option=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --interactive" "
+	test_expect_success "$opt incompatible with --interactive" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --interactive A
 	"
 
-	test_expect_failure "$opt incompatible with --exec" "
+	test_expect_success "$opt incompatible with --exec" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --exec 'true' A
 	"
@@ -70,17 +70,17 @@ test_expect_success '--preserve-merges incompatible with --signoff' '
 	test_must_fail git rebase --preserve-merges --signoff A
 '
 
-test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
+test_expect_success '--preserve-merges incompatible with --rebase-merges' '
 	git checkout B^0 &&
 	test_must_fail git rebase --preserve-merges --rebase-merges A
 '
 
-test_expect_failure '--rebase-merges incompatible with --strategy' '
+test_expect_success '--rebase-merges incompatible with --strategy' '
 	git checkout B^0 &&
 	test_must_fail git rebase --rebase-merges -s resolve A
 '
 
-test_expect_failure '--rebase-merges incompatible with --strategy-option' '
+test_expect_success '--rebase-merges incompatible with --strategy-option' '
 	git checkout B^0 &&
 	test_must_fail git rebase --rebase-merges -Xignore-space-change A
 '
-- 
2.18.0.9.g678597d97e

[PATCH v4 5/9] git-rebase.txt: address confusion between --no-ff vs --force-rebase

[Elijah Newren]

rebase was taught the --force-rebase option in commit b2f82e05de ("Teach
rebase to rebase even if upstream is up to date", 2009-02-13).  This flag
worked for the am and merge backends, but wasn't a valid option for the
interactive backend.

rebase was taught the --no-ff option for interactive rebases in commit
b499549401cb ("Teach rebase the --no-ff option.", 2010-03-24), to do the
exact same thing as --force-rebase does for non-interactive rebases.  This
commit explicitly documented the fact that --force-rebase was incompatible
with --interactive, though it made --no-ff a synonym for --force-rebase
for non-interactive rebases.  The choice of a new option was based on the
fact that "force rebase" didn't sound like an appropriate term for the
interactive machinery.

In commit 6bb4e485cff8 ("rebase: align variable names", 2011-02-06), the
separate parsing of command line options in the different rebase scripts
was removed, and whether on accident or because the author noticed that
these options did the same thing, the options became synonyms and both
were accepted by all three rebase types.

In commit 2d26d533a012 ("Documentation/git-rebase.txt: -f forces a rebase
that would otherwise be a no-op", 2014-08-12), which reworded the
description of the --force-rebase option, the (no-longer correct) sentence
stating that --force-rebase was incompatible with --interactive was
finally removed.

Finally, as explained at
https://public-inbox.org/git/98279912-0f52-969d-44a6-22242039387f@xiplink.com

    In the original discussion around this option [1], at one point I
    proposed teaching rebase--interactive to respect --force-rebase
    instead of adding a new option [2].  Ultimately --no-ff was chosen as
    the better user interface design [3], because an interactive rebase
    can't be "forced" to run.

We have accepted both --no-ff and --force-rebase as full synonyms for all
three rebase types for over seven years.  Documenting them differently
and in ways that suggest they might not be quite synonyms simply leads to
confusion.  Adjust the documentation to match reality.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 30 ++++++++++--------------------
 1 file changed, 10 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index b2d95e3fb9..2f47495a4d 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -337,16 +337,18 @@ See also INCOMPATIBLE OPTIONS below.
 +
 See also INCOMPATIBLE OPTIONS below.
 
--f::
+--no-ff::
 --force-rebase::
-	Force a rebase even if the current branch is up to date and
-	the command without `--force` would return without doing anything.
+-f::
+	Individually replay all rebased commits instead of fast-forwarding
+	over the unchanged ones.  This ensures that the entire history of
+	the rebased branch is composed of new commits.
 +
-You may find this (or --no-ff with an interactive rebase) helpful after
-reverting a topic branch merge, as this option recreates the topic branch with
-fresh commits so it can be remerged successfully without needing to "revert
-the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
+You may find this helpful after reverting a topic branch merge, as this option
+recreates the topic branch with fresh commits so it can be remerged
+successfully without needing to "revert the reversion" (see the
+link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for
+details).
 
 --fork-point::
 --no-fork-point::
@@ -498,18 +500,6 @@ See also INCOMPATIBLE OPTIONS below.
 	with care: the final stash application after a successful
 	rebase might result in non-trivial conflicts.
 
---no-ff::
-	With --interactive, cherry-pick all rebased commits instead of
-	fast-forwarding over the unchanged ones.  This ensures that the
-	entire history of the rebased branch is composed of new commits.
-+
-Without --interactive, this is a synonym for --force-rebase.
-+
-You may find this helpful after reverting a topic branch merge, as this option
-recreates the topic branch with fresh commits so it can be remerged
-successfully without needing to "revert the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
-
 INCOMPATIBLE OPTIONS
 --------------------
 
-- 
2.18.0.9.g678597d97e

[PATCH v4 6/9] directory-rename-detection.txt: technical docs on abilities and limitations

[Elijah Newren]

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 .../technical/directory-rename-detection.txt  | 92 +++++++++++++++++++
 1 file changed, 92 insertions(+)
 create mode 100644 Documentation/technical/directory-rename-detection.txt

diff --git a/Documentation/technical/directory-rename-detection.txt b/Documentation/technical/directory-rename-detection.txt
new file mode 100644
index 0000000000..6e22920a39
--- /dev/null
+++ b/Documentation/technical/directory-rename-detection.txt
@@ -0,0 +1,92 @@
+Directory rename detection
+==========================
+
+Rename detection logic in diffcore-rename that checks for renames of
+individual files is aggregated and analyzed in merge-recursive for cases
+where combinations of renames indicate that a full directory has been
+renamed.
+
+Scope of abilities
+------------------
+
+It is perhaps easiest to start with an example:
+
+  * When all of x/a, x/b and x/c have moved to z/a, z/b and z/c, it is
+    likely that x/d added in the meantime would also want to move to z/d by
+    taking the hint that the entire directory 'x' moved to 'z'.
+
+More interesting possibilities exist, though, such as:
+
+  * one side of history renames x -> z, and the other renames some file to
+    x/e, causing the need for the merge to do a transitive rename.
+
+  * one side of history renames x -> z, but also renames all files within
+    x.  For example, x/a -> z/alpha, x/b -> z/bravo, etc.
+
+  * both 'x' and 'y' being merged into a single directory 'z', with a
+    directory rename being detected for both x->z and y->z.
+
+  * not all files in a directory being renamed to the same location;
+    i.e. perhaps most the files in 'x' are now found under 'z', but a few
+    are found under 'w'.
+
+  * a directory being renamed, which also contained a subdirectory that was
+    renamed to some entirely different location.  (And perhaps the inner
+    directory itself contained inner directories that were renamed to yet
+    other locations).
+
+  * combinations of the above; see t/t6043-merge-rename-directories.sh for
+    various interesting cases.
+
+Limitations -- applicability of directory renames
+-------------------------------------------------
+
+In order to prevent edge and corner cases resulting in either conflicts
+that cannot be represented in the index or which might be too complex for
+users to try to understand and resolve, a couple basic rules limit when
+directory rename detection applies:
+
+  1) If a given directory still exists on both sides of a merge, we do
+     not consider it to have been renamed.
+
+  2) If a subset of to-be-renamed files have a file or directory in the
+     way (or would be in the way of each other), "turn off" the directory
+     rename for those specific sub-paths and report the conflict to the
+     user.
+
+  3) If the other side of history did a directory rename to a path that
+     your side of history renamed away, then ignore that particular
+     rename from the other side of history for any implicit directory
+     renames (but warn the user).
+
+Limitations -- detailed rules and testcases
+-------------------------------------------
+
+t/t6043-merge-rename-directories.sh contains extensive tests and commentary
+which generate and explore the rules listed above.  It also lists a few
+additional rules:
+
+  a) If renames split a directory into two or more others, the directory
+     with the most renames, "wins".
+
+  b) Avoid directory-rename-detection for a path, if that path is the
+     source of a rename on either side of a merge.
+
+  c) Only apply implicit directory renames to directories if the other side
+     of history is the one doing the renaming.
+
+Limitations -- support in different commands
+--------------------------------------------
+
+Directory rename detection is supported by 'merge' and 'cherry-pick'.
+Other git commands which users might be surprised to see limited or no
+directory rename detection support in:
+
+  * diff
+
+    Folks have requested in the past that `git diff` detect directory
+    renames and somehow simplify its output.  It is not clear whether this
+    would be desirable or how the output should be simplified, so this was
+    simply not implemented.  Further, to implement this, directory rename
+    detection logic would need to move from merge-recursive to
+    diffcore-rename.
-- 
2.18.0.9.g678597d97e

[PATCH v4 7/9] git-rebase.txt: document behavioral differences between modes

[Elijah Newren]

There are a variety of aspects that are common to all rebases regardless
of which backend is in use; however, the behavior for these different
aspects varies in ways that could surprise users.  (In fact, it's not
clear -- to me at least -- that these differences were even desirable or
intentional.)  Document these differences.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt                  | 32 +++++++++++++++++++
 .../technical/directory-rename-detection.txt  | 23 +++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 2f47495a4d..a67df4caba 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -546,6 +546,38 @@ Other incompatible flag pairs:
  * --rebase-merges and --strategy
  * --rebase-merges and --strategy-option
 
+BEHAVIORAL DIFFERENCES
+-----------------------
+
+ * empty commits:
+
+    am-based rebase will drop any "empty" commits, whether the
+    commit started empty (had no changes relative to its parent to
+    start with) or ended empty (all changes were already applied
+    upstream in other commits).
+
+    merge-based rebase does the same.
+
+    interactive-based rebase will by default drop commits that
+    started empty and halt if it hits a commit that ended up empty.
+    The `--keep-empty` option exists for interactive rebases to allow
+    it to keep commits that started empty.
+
+  * empty commit messages:
+
+    am-based rebase will silently apply commits with empty commit
+    messages.
+
+    merge-based and interactive-based rebases will by default halt
+    on any such commits.  The `--allow-empty-message` option exists to
+    allow interactive-based rebases to apply such commits without
+    halting.
+
+  * directory rename detection:
+
+    merge-based and interactive-based rebases work fine with
+    directory rename detection.  am-based rebases sometimes do not.
+
 include::merge-strategies.txt[]
 
 NOTES
diff --git a/Documentation/technical/directory-rename-detection.txt b/Documentation/technical/directory-rename-detection.txt
index 6e22920a39..1c0086e287 100644
--- a/Documentation/technical/directory-rename-detection.txt
+++ b/Documentation/technical/directory-rename-detection.txt
@@ -90,3 +90,26 @@ directory rename detection support in:
     simply not implemented.  Further, to implement this, directory rename
     detection logic would need to move from merge-recursive to
     diffcore-rename.
+
+  * am
+
+    git-am tries to avoid a full three way merge, instead calling
+    git-apply.  That prevents us from detecting renames at all, which may
+    defeat the directory rename detection.  There is a fallback, though; if
+    the initial git-apply fails and the user has specified the -3 option,
+    git-am will fall back to a three way merge.  However, git-am lacks the
+    necessary information to do a "real" three way merge.  Instead, it has
+    to use build_fake_ancestor() to get a merge base that is missing files
+    whose rename may have been important to detect for directory rename
+    detection to function.
+
+  * rebase
+
+    Since am-based rebases work by first generating a bunch of patches
+    (which no longer record what the original commits were and thus don't
+    have the necessary info from which we can find a real merge-base), and
+    then calling git-am, this implies that am-based rebases will not always
+    successfully detect directory renames either (see the 'am' section
+    above).  merged-based rebases (rebase -m) and cherry-pick-based rebases
+    (rebase -i) are not affected by this shortcoming, and fully support
+    directory rename detection.
-- 
2.18.0.9.g678597d97e

[PATCH v4 8/9] t3401: add directory rename testcases for rebase and am

[Elijah Newren]

Add a simple directory rename testcase, in conjunction with each of the
types of rebases:
  git-rebase--interactive
  git-rebase--am
  git-rebase--merge
and also use the same testcase for
  git am --3way

This demonstrates a difference in behavior between the different rebase
backends in regards to directory rename detection.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3401-rebase-and-am-rename.sh | 105 ++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100755 t/t3401-rebase-and-am-rename.sh

diff --git a/t/t3401-rebase-and-am-rename.sh b/t/t3401-rebase-and-am-rename.sh
new file mode 100755
index 0000000000..8f832957fc
--- /dev/null
+++ b/t/t3401-rebase-and-am-rename.sh
@@ -0,0 +1,105 @@
+#!/bin/sh
+
+test_description='git rebase + directory rename tests'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup testcase' '
+	test_create_repo dir-rename &&
+	(
+		cd dir-rename &&
+
+		mkdir x &&
+		test_seq  1 10 >x/a &&
+		test_seq 11 20 >x/b &&
+		test_seq 21 30 >x/c &&
+		test_write_lines a b c d e f g h i >l &&
+		git add x l &&
+		git commit -m "Initial" &&
+
+		git branch O &&
+		git branch A &&
+		git branch B &&
+
+		git checkout A &&
+		git mv x y &&
+		git mv l letters &&
+		git commit -m "Rename x to y, l to letters" &&
+
+		git checkout B &&
+		echo j >>l &&
+		test_seq 31 40 >x/d &&
+		git add l x/d &&
+		git commit -m "Modify l, add x/d"
+	)
+'
+
+test_expect_success 'rebase --interactive: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		set_fake_editor &&
+		FAKE_LINES="1" git rebase --interactive A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_failure 'rebase (am): directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		git rebase A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_success 'rebase --merge: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		git rebase --merge A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_failure 'am: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout A^0 &&
+
+		git format-patch -1 B &&
+
+		git am --3way 0001*.patch &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_done
-- 
2.18.0.9.g678597d97e

[RFC PATCH v4 9/9] git-rebase: make --allow-empty-message the default

[Elijah Newren]

rebase backends currently behave differently with empty commit messages,
largely as a side-effect of the different underlying commands on which
they are based.  am-based rebases apply commits with an empty commit
message without stopping or requiring the user to specify an extra flag.
(It is interesting to note that am-based rebases are the default rebase
type, and no one has ever requested a --no-allow-empty-message flag to
change this behavior.)  merge-based and interactive-based rebases (which
are ultimately based on git-commit), will currently halt on any such
commits and require the user to manually specify what to do with the
commit and continue.

One possible rationale for the difference in behavior is that the purpose
of an "am" based rebase is solely to transplant an existing history, while
an "interactive" rebase is one whose purpose is to polish a series before
making it publishable.  Thus, stopping and asking for confirmation for a
possible problem is more appropriate in the latter case.  However, there
are two problems with this rationale:

  1) merge-based rebases are also non-interactive and there are multiple
     types of rebases that use the interactive machinery but are not
     explicitly interactive (e.g. when either --rebase-merges or
     --keep-empty are specified without --interactive).  These rebases are
     also used solely to transplant an existing history, and thus also
     should default to --allow-empty-message.

  2) this rationale only says that the user is more accepting of stopping
     in the case of an explicitly interactive rebase, not that stopping
     for this particular reason actually makes sense.  Exploring whether
     it makes sense, requires backing up and analyzing the underlying
     commands...

If git-commit did not error out on empty commits by default, accidental
creation of commits with empty messages would be a very common occurrence
(this check has caught me many times).  Further, nearly all such empty
commit messages would be considered an accidental error (as evidenced by a
huge amount of documentation across version control systems and in various
blog posts explaining how important commit messages are).  A simple check
for what would otherwise be a common error thus made a lot of sense, and
git-commit gained an --allow-empty-message flag for special case
overrides.  This has made commits with empty messages very rare.

There are two sources for commits with empty messages for rebase (and
cherry-pick): (a) commits created in git where the user previously
specified --allow-empty-message to git-commit, and (b) commits imported
into git from other version control systems.  In case (a), the user has
already explicitly specified that there is something special about this
commit that makes them not want to specify a commit message; forcing them
to re-specify with every cherry-pick or rebase seems more likely to be
infuriating than helpful.  In case (b), the commit is highly unlikely to
have been authored by the person who has imported the history and is doing
the rebase or cherry-pick, and thus the user is unlikely to be the
appropriate person to write a commit message for it.  Stopping and
expecting the user to modify the commit before proceeding thus seems
counter-productive.

Further, note that while empty commit messages was a common error case for
git-commit to deal with, it is a rare case for rebase (or cherry-pick).
The fact that it is rare raises the question of why it would be worth
checking and stopping on this particular condition and not others.  For
example, why doesn't an interactive rebase automatically stop if the
commit message's first line is 2000 columns long, or is missing a blank
line after the first line, or has every line indented with five spaces, or
any number of other myriad problems?

Finally, note that if a user doing an interactive rebase does have the
necessary knowledge to add a message for any such commit and wants to do
so, it is rather simple for them to change the appropriate line from
'pick' to 'reword'.  The fact that the subject is empty in the todo list
that the user edits should even serve as a way to notify them.

As far as I can tell, the fact that merge-based and interactive-based
rebases stop on commits with empty commit messages is solely a by-product
of having been based on git-commit.  It went without notice for a long
time precisely because such cases are rare.  The rareness of this
situation made it difficult to reason about, so when folks did eventually
notice this behavior, they assumed it was there for a good reason and just
added an --allow-empty-message flag.  In my opinion, stopping on such
messages not desirable in any of these cases, even the (explicitly)
interactive case.

Signed-off-by: Elijah Newren <newren@gmail.com>
---

My commit messsage seems like one of those things that someone else will
instantly see how to shrink to less than a quarter of its size while still
retaining all essential reasoning.  I can't seem to find the simple way to
state it, though.

 Documentation/git-rebase.txt  | 10 ----------
 git-rebase.sh                 |  2 +-
 t/t3404-rebase-interactive.sh |  7 ++++---
 t/t3405-rebase-malformed.sh   | 11 +++--------
 4 files changed, 8 insertions(+), 22 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index a67df4caba..9e136ee16e 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -563,16 +563,6 @@ BEHAVIORAL DIFFERENCES
     The `--keep-empty` option exists for interactive rebases to allow
     it to keep commits that started empty.
 
-  * empty commit messages:
-
-    am-based rebase will silently apply commits with empty commit
-    messages.
-
-    merge-based and interactive-based rebases will by default halt
-    on any such commits.  The `--allow-empty-message` option exists to
-    allow interactive-based rebases to apply such commits without
-    halting.
-
   * directory rename detection:
 
     merge-based and interactive-based rebases work fine with
diff --git a/git-rebase.sh b/git-rebase.sh
index 18ac8226c4..031dbd2ec8 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -95,7 +95,7 @@ rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
-allow_empty_message=
+allow_empty_message=--allow-empty-message
 signoff=
 test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
 case "$(git config --bool commit.gpgsign)" in
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 352a52e59d..81ce9fe7f9 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -553,15 +553,16 @@ test_expect_success '--continue tries to commit, even for "edit"' '
 '
 
 test_expect_success 'aborted --continue does not squash commits after "edit"' '
+	test_when_finished "git rebase --abort" &&
 	old=$(git rev-parse HEAD) &&
 	test_tick &&
 	set_fake_editor &&
 	FAKE_LINES="edit 1" git rebase -i HEAD^ &&
 	echo "edited again" > file7 &&
 	git add file7 &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " git rebase --continue &&
-	test $old = $(git rev-parse HEAD) &&
-	git rebase --abort
+	echo all the things >>conflict &&
+	test_must_fail git rebase --continue &&
+	test $old = $(git rev-parse HEAD)
 '
 
 test_expect_success 'auto-amend only edited commits after "edit"' '
diff --git a/t/t3405-rebase-malformed.sh b/t/t3405-rebase-malformed.sh
index cb7c6de84a..da94dddc86 100755
--- a/t/t3405-rebase-malformed.sh
+++ b/t/t3405-rebase-malformed.sh
@@ -77,19 +77,14 @@ test_expect_success 'rebase commit with diff in message' '
 '
 
 test_expect_success 'rebase -m commit with empty message' '
-	test_must_fail git rebase -m master empty-message-merge &&
-	git rebase --abort &&
-	git rebase -m --allow-empty-message master empty-message-merge
+	git rebase -m master empty-message-merge
 '
 
 test_expect_success 'rebase -i commit with empty message' '
 	git checkout diff-in-message &&
 	set_fake_editor &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i HEAD^ &&
-	git rebase --abort &&
-	FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i --allow-empty-message HEAD^
+	env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
+		git rebase -i HEAD^
 '
 
 test_done
-- 
2.18.0.9.g678597d97e

Re: [PATCH] t3401: add directory rename testcases for rebase and am

[Elijah Newren]

On Wed, Jun 6, 2018 at 10:04 PM, Elijah Newren <newren@gmail.com> wrote:
> Add a simple directory rename testcase, in conjunction with each of the
> types of rebases:
>   git-rebase--interactive
>   git-rebase--am
>   git-rebase--merge
> and also use the same testcase for
>   git am --3way
>
> This demonstrates an inconsistency between the different rebase backends
> and which can detect the directory rename.

Due to changes in the en/rebase-consistency series, I'm no longer
keeping this patch as a separate submission but have pulled it into v4
of that series[1].


[1] https://public-inbox.org/git/20180625161300.26060-9-newren@gmail.com/

Re: [PATCH v4 3/9] t3422: new testcases for checking when incompatible options passed

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> +# Rebase has lots of useful options like --whitepsace=fix, which are
> +# actually all built in terms of flags to git-am.  Since neither
> +# --merge nor --interactive (nor any options that imply those two) use
> +# git-am, using them together will result in flags like --whitespace=fix
> +# being ignored.  Make sure rebase warns the user and aborts instead.
> +#
> +
> +test_rebase_am_only () {
> +	opt=$1
> +	shift
> +	test_expect_failure "$opt incompatible with --merge" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --merge A
> +	"
> +
> +	test_expect_failure "$opt incompatible with --strategy=ours" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --strategy=ours A
> +	"
> +
> +	test_expect_failure "$opt incompatible with --strategy-option=ours" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --strategy=ours A

This line is broken and it is carried over to later patches.  It
needs to be -Xours (or --strategy-option=ours, if we really want ot
be verbose).

> +	"
> +
> +	test_expect_failure "$opt incompatible with --interactive" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --interactive A
> +	"
> +
> +	test_expect_failure "$opt incompatible with --exec" "
> +		git checkout B^0 &&
> +		test_must_fail git rebase $opt --exec 'true' A
> +	"
> +
> +}

> +
> +test_rebase_am_only --whitespace=fix
> +test_rebase_am_only --ignore-whitespace
> +test_rebase_am_only --committer-date-is-author-date
> +test_rebase_am_only -C4

I was hesitant to hardcode what I perceive as limitations of non-am
rebase implementations with a test like this, but once somebody
fixes "rebase -i" for example to be capable of --whitespace=fix for
example, then we can just drop one line from the above four (and
write other tests for "rebase -i --whitespace=fix").  The
test_rebase_am_only is to help us make sure what is (still) not
supported by non-am rebases gets diagnosed as an error.

So my worry is totally unfounded, which is good.

> +test_expect_success '--preserve-merges incompatible with --signoff' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --preserve-merges --signoff A
> +'
> +
> +test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --preserve-merges --rebase-merges A
> +'
> +
> +test_expect_failure '--rebase-merges incompatible with --strategy' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --rebase-merges -s resolve A
> +'
> +
> +test_expect_failure '--rebase-merges incompatible with --strategy-option' '
> +	git checkout B^0 &&
> +	test_must_fail git rebase --rebase-merges -Xignore-space-change A
> +'
> +
> +test_done

Re: [PATCH v4 3/9] t3422: new testcases for checking when incompatible options passed

[Elijah Newren]

On Tue, Jun 26, 2018 at 11:17 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Elijah Newren <newren@gmail.com> writes:
>
>> +# Rebase has lots of useful options like --whitepsace=fix, which are
>> +# actually all built in terms of flags to git-am.  Since neither
>> +# --merge nor --interactive (nor any options that imply those two) use
>> +# git-am, using them together will result in flags like --whitespace=fix
>> +# being ignored.  Make sure rebase warns the user and aborts instead.
>> +#
>> +
>> +test_rebase_am_only () {
>> +     opt=$1
>> +     shift
>> +     test_expect_failure "$opt incompatible with --merge" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --merge A
>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --strategy=ours" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --strategy=ours A
>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --strategy-option=ours" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --strategy=ours A
>
> This line is broken and it is carried over to later patches.  It
> needs to be -Xours (or --strategy-option=ours, if we really want ot
> be verbose).

Indeed; I'll fix that up and resubmit.  Thanks for catching it.

>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --interactive" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --interactive A
>> +     "
>> +
>> +     test_expect_failure "$opt incompatible with --exec" "
>> +             git checkout B^0 &&
>> +             test_must_fail git rebase $opt --exec 'true' A
>> +     "
>> +
>> +}
>
>> +
>> +test_rebase_am_only --whitespace=fix
>> +test_rebase_am_only --ignore-whitespace
>> +test_rebase_am_only --committer-date-is-author-date
>> +test_rebase_am_only -C4
>
> I was hesitant to hardcode what I perceive as limitations of non-am
> rebase implementations with a test like this, but once somebody
> fixes "rebase -i" for example to be capable of --whitespace=fix for
> example, then we can just drop one line from the above four (and
> write other tests for "rebase -i --whitespace=fix").  The
> test_rebase_am_only is to help us make sure what is (still) not
> supported by non-am rebases gets diagnosed as an error.

Yes, and I was thinking in particular that we could start by teaching
rebase -i to understand --ignore-space-change/--ignore-whitespace by
just transliterating it into -Xignore-space-change.  That is, after
https://public-inbox.org/git/20180607050845.19779-2-newren@gmail.com/
is picked up and eventually merged down.  Speaking of which, I need to
resubmit that.

> So my worry is totally unfounded, which is good.

[PATCH v5 0/9] Document/fix/warn about rebase incompatibilities and inconsistencies

[Elijah Newren]

git-rebase has lots of options that are mutually incompatible.  Even among
aspects of its behavior that is common to all rebase types, it has a number
of inconsistencies.  This series tries to document, fix, and/or warn users
about many of these.

Changes since v4 (short branch-diff at the end):
  - Fixed --strategy vs. --strategy-option (in patch 3, carries over to
    context region in later commit)

v4 didn't get a lot of feedback (though it was picked up by Junio), so I'll
repeat the areas that would be of most note for reviewers since v3:

  * I have left patch 9 in RFC state; since v3 it has an expanded the
    commit message with an in-depth usability rationale for the
    change.
  * It sounded like Junio was slightly unclear about the intent of the
    wording in Patch 1.  Not sure if my answer (in email) was sufficient or
    if there are wording improvements others might suggest.
  * I'm assuming the --keep-empty and/or --empty={drop,halt,keep} (see
    comments on patch 5 of v3) can be resolved in a later series.

Elijah Newren (9):
  git-rebase.txt: document incompatible options
  git-rebase.sh: update help messages a bit
  t3422: new testcases for checking when incompatible options passed
  git-rebase: error out when incompatible options passed
  git-rebase.txt: address confusion between --no-ff vs --force-rebase
  directory-rename-detection.txt: technical docs on abilities and
    limitations
  git-rebase.txt: document behavioral differences between modes
  t3401: add directory rename testcases for rebase and am
  git-rebase: make --allow-empty-message the default

 Documentation/git-rebase.txt                  | 135 ++++++++++++++----
 .../technical/directory-rename-detection.txt  | 115 +++++++++++++++
 git-rebase.sh                                 |  43 +++++-
 t/t3401-rebase-and-am-rename.sh               | 105 ++++++++++++++
 t/t3404-rebase-interactive.sh                 |   7 +-
 t/t3405-rebase-malformed.sh                   |  11 +-
 t/t3422-rebase-incompatible-options.sh        |  88 ++++++++++++
 7 files changed, 462 insertions(+), 42 deletions(-)
 create mode 100644 Documentation/technical/directory-rename-detection.txt
 create mode 100755 t/t3401-rebase-and-am-rename.sh
 create mode 100755 t/t3422-rebase-incompatible-options.sh

 1: 3f454ebc5e =  1: 3f454ebc5e git-rebase.txt: document incompatible options
 2: 31a5a071a6 =  2: 31a5a071a6 git-rebase.sh: update help messages a bit
 3: 5a2b5eec79 !  3: bc3a5a3f95 t3422: new testcases for checking when incompatible options passed
    @@ -62,7 +62,7 @@
     +
     +	test_expect_failure "$opt incompatible with --strategy-option=ours" "
     +		git checkout B^0 &&
    -+		test_must_fail git rebase $opt --strategy=ours A
    ++		test_must_fail git rebase $opt --strategy-option=ours A
     +	"
     +
     +	test_expect_failure "$opt incompatible with --interactive" "
 4: 1e1c83724a !  4: ca3b8327f7 git-rebase: error out when incompatible options passed
    @@ -104,7 +104,7 @@
     -	test_expect_failure "$opt incompatible with --strategy-option=ours" "
     +	test_expect_success "$opt incompatible with --strategy-option=ours" "
      		git checkout B^0 &&
    - 		test_must_fail git rebase $opt --strategy=ours A
    + 		test_must_fail git rebase $opt --strategy-option=ours A
      	"
      
     -	test_expect_failure "$opt incompatible with --interactive" "
 5: 51023269d3 =  5: 6ac359359e git-rebase.txt: address confusion between --no-ff vs --force-rebase
 6: f017d45dd9 =  6: e5c5db9110 directory-rename-detection.txt: technical docs on abilities and limitations
 7: 0a359df404 =  7: e330437305 git-rebase.txt: document behavioral differences between modes
 8: beaadceaef =  8: f704f7eee8 t3401: add directory rename testcases for rebase and am
 9: 431b2c36d5 =  9: 436f597487 git-rebase: make --allow-empty-message the default

-- 
2.18.0.9.g431b2c36d5

[PATCH v5 1/9] git-rebase.txt: document incompatible options

[Elijah Newren]

git rebase has many options that only work with one of its three backends.
It also has a few other pairs of incompatible options.  Document these.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 85 ++++++++++++++++++++++++++++++++----
 1 file changed, 77 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 0e20a66e73..b2d95e3fb9 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -243,11 +243,15 @@ leave out at most one of A and B, in which case it defaults to HEAD.
 --keep-empty::
 	Keep the commits that do not change anything from its
 	parents in the result.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --allow-empty-message::
 	By default, rebasing commits with an empty message will fail.
 	This option overrides that behavior, allowing commits with empty
 	messages to be rebased.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --skip::
 	Restart the rebasing process by skipping the current patch.
@@ -271,6 +275,8 @@ branch on top of the <upstream> branch.  Because of this, when a merge
 conflict happens, the side reported as 'ours' is the so-far rebased
 series, starting with <upstream>, and 'theirs' is the working branch.  In
 other words, the sides are swapped.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -s <strategy>::
 --strategy=<strategy>::
@@ -280,8 +286,10 @@ other words, the sides are swapped.
 +
 Because 'git rebase' replays each commit from the working branch
 on top of the <upstream> branch using the given strategy, using
-the 'ours' strategy simply discards all patches from the <branch>,
+the 'ours' strategy simply empties all patches from the <branch>,
 which makes little sense.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -X <strategy-option>::
 --strategy-option=<strategy-option>::
@@ -289,6 +297,8 @@ which makes little sense.
 	This implies `--merge` and, if no strategy has been
 	specified, `-s recursive`.  Note the reversal of 'ours' and
 	'theirs' as noted above for the `-m` option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -S[<keyid>]::
 --gpg-sign[=<keyid>]::
@@ -324,6 +334,8 @@ which makes little sense.
 	and after each change.  When fewer lines of surrounding
 	context exist they all must match.  By default no context is
 	ever ignored.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -f::
 --force-rebase::
@@ -355,19 +367,22 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 --whitespace=<option>::
 	These flag are passed to the 'git apply' program
 	(see linkgit:git-apply[1]) that applies the patch.
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --committer-date-is-author-date::
 --ignore-date::
 	These flags are passed to 'git am' to easily change the dates
 	of the rebased commits (see linkgit:git-am[1]).
-	Incompatible with the --interactive option.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --signoff::
 	Add a Signed-off-by: trailer to all the rebased commits. Note
 	that if `--interactive` is given then only commits marked to be
-	picked, edited or reworded will have the trailer added. Incompatible
-	with the `--preserve-merges` option.
+	picked, edited or reworded will have the trailer added.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -i::
 --interactive::
@@ -378,6 +393,8 @@ default is `--no-fork-point`, otherwise the default is `--fork-point`.
 The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
++
+See also INCOMPATIBLE OPTIONS below.
 
 -r::
 --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
@@ -404,7 +421,7 @@ It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
 explicit `exec git merge -s <strategy> [...]` commands.
 +
-See also REBASING MERGES below.
+See also REBASING MERGES and INCOMPATIBLE OPTIONS below.
 
 -p::
 --preserve-merges::
@@ -415,6 +432,8 @@ See also REBASING MERGES below.
 This uses the `--interactive` machinery internally, but combining it
 with the `--interactive` option explicitly is generally not a good
 idea unless you know what you are doing (see BUGS below).
++
+See also INCOMPATIBLE OPTIONS below.
 
 -x <cmd>::
 --exec <cmd>::
@@ -437,6 +456,8 @@ squash/fixup series.
 +
 This uses the `--interactive` machinery internally, but it can be run
 without an explicit `--interactive`.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --root::
 	Rebase all commits reachable from <branch>, instead of
@@ -447,6 +468,8 @@ without an explicit `--interactive`.
 	When used together with both --onto and --preserve-merges,
 	'all' root commits will be rewritten to have <newbase> as parent
 	instead.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autosquash::
 --no-autosquash::
@@ -461,11 +484,11 @@ without an explicit `--interactive`.
 	too.  The recommended way to create fixup/squash commits is by using
 	the `--fixup`/`--squash` options of linkgit:git-commit[1].
 +
-This option is only valid when the `--interactive` option is used.
-+
 If the `--autosquash` option is enabled by default using the
 configuration variable `rebase.autoSquash`, this option can be
 used to override and disable this setting.
++
+See also INCOMPATIBLE OPTIONS below.
 
 --autostash::
 --no-autostash::
@@ -487,6 +510,52 @@ recreates the topic branch with fresh commits so it can be remerged
 successfully without needing to "revert the reversion" (see the
 link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
 
+INCOMPATIBLE OPTIONS
+--------------------
+
+git-rebase has many flags that are incompatible with each other,
+predominantly due to the fact that it has three different underlying
+implementations:
+
+ * one based on linkgit:git-am[1] (the default)
+ * one based on git-merge-recursive (merge backend)
+ * one based on linkgit:git-cherry-pick[1] (interactive backend)
+
+Flags only understood by the am backend:
+
+ * --committer-date-is-author-date
+ * --ignore-date
+ * --whitespace
+ * --ignore-whitespace
+ * -C
+
+Flags understood by both merge and interactive backends:
+
+ * --merge
+ * --strategy
+ * --strategy-option
+ * --allow-empty-message
+
+Flags only understood by the interactive backend:
+
+ * --[no-]autosquash
+ * --rebase-merges
+ * --preserve-merges
+ * --interactive
+ * --exec
+ * --keep-empty
+ * --autosquash
+ * --edit-todo
+ * --root when used in combination with --onto
+
+Other incompatible flag pairs:
+
+ * --preserve-merges and --interactive
+ * --preserve-merges and --signoff
+ * --preserve-merges and --rebase-merges
+ * --rebase-merges and --strategy
+ * --rebase-merges and --strategy-option
+
 include::merge-strategies.txt[]
 
 NOTES
-- 
2.18.0.9.g431b2c36d5

[PATCH v5 2/9] git-rebase.sh: update help messages a bit

[Elijah Newren]

signoff is not specific to the am-backend.  Also, re-order a few options
to make like things (e.g. strategy and strategy-option) be near each
other.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 40be59ecc4..bf71b7fa20 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -20,23 +20,23 @@ onto=!             rebase onto given branch instead of upstream
 r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
+X,strategy-option=! pass the argument through to the merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
+f,force-rebase!    cherry-pick all commits, even if unchanged
 m,merge!           use merging strategies to rebase
 i,interactive!     let the user edit the list of commits to rebase
 x,exec=!           add exec lines after each commit of the editable list
 k,keep-empty	   preserve empty commits during rebase
 allow-empty-message allow rebasing commits with empty messages
-f,force-rebase!    force rebase even if branch is up to date
-X,strategy-option=! pass the argument through to the merge strategy
 stat!              display a diffstat of what changed upstream
 n,no-stat!         do not show diffstat of what changed upstream
 verify             allow pre-rebase hook to run
 rerere-autoupdate  allow rerere to update index with resolved conflicts
 root!              rebase all reachable commits up to the root(s)
 autosquash         move commits that begin with squash!/fixup! under -i
+signoff            add a Signed-off-by: line to each commit
 committer-date-is-author-date! passed to 'git am'
 ignore-date!       passed to 'git am'
-signoff            passed to 'git am'
 whitespace=!       passed to 'git apply'
 ignore-whitespace! passed to 'git apply'
 C=!                passed to 'git apply'
-- 
2.18.0.9.g431b2c36d5

[PATCH v5 3/9] t3422: new testcases for checking when incompatible options passed

[Elijah Newren]

git rebase is split into three types: am, merge, and interactive.  Various
options imply different types, and which mode we are using determine which
sub-script (git-rebase--$type) is executed to finish the work.  Not all
options work with all types, so add tests for combinations where we expect
to receive an error rather than having options be silently ignored.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3422-rebase-incompatible-options.sh | 88 ++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
 create mode 100755 t/t3422-rebase-incompatible-options.sh

diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
new file mode 100755
index 0000000000..b007a15eba
--- /dev/null
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -0,0 +1,88 @@
+#!/bin/sh
+
+test_description='test if rebase detects and aborts on incompatible options'
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	test_seq 2 9 >foo &&
+	git add foo &&
+	git commit -m orig &&
+
+	git branch A &&
+	git branch B &&
+
+	git checkout A &&
+	test_seq 1 9 >foo &&
+	git add foo &&
+	git commit -m A &&
+
+	git checkout B &&
+	echo "q qfoo();" | q_to_tab >>foo &&
+	git add foo &&
+	git commit -m B
+'
+
+#
+# Rebase has lots of useful options like --whitepsace=fix, which are
+# actually all built in terms of flags to git-am.  Since neither
+# --merge nor --interactive (nor any options that imply those two) use
+# git-am, using them together will result in flags like --whitespace=fix
+# being ignored.  Make sure rebase warns the user and aborts instead.
+#
+
+test_rebase_am_only () {
+	opt=$1
+	shift
+	test_expect_failure "$opt incompatible with --merge" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --merge A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --strategy-option=ours A
+	"
+
+	test_expect_failure "$opt incompatible with --interactive" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --interactive A
+	"
+
+	test_expect_failure "$opt incompatible with --exec" "
+		git checkout B^0 &&
+		test_must_fail git rebase $opt --exec 'true' A
+	"
+
+}
+
+test_rebase_am_only --whitespace=fix
+test_rebase_am_only --ignore-whitespace
+test_rebase_am_only --committer-date-is-author-date
+test_rebase_am_only -C4
+
+test_expect_success '--preserve-merges incompatible with --signoff' '
+	git checkout B^0 &&
+	test_must_fail git rebase --preserve-merges --signoff A
+'
+
+test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
+	git checkout B^0 &&
+	test_must_fail git rebase --preserve-merges --rebase-merges A
+'
+
+test_expect_failure '--rebase-merges incompatible with --strategy' '
+	git checkout B^0 &&
+	test_must_fail git rebase --rebase-merges -s resolve A
+'
+
+test_expect_failure '--rebase-merges incompatible with --strategy-option' '
+	git checkout B^0 &&
+	test_must_fail git rebase --rebase-merges -Xignore-space-change A
+'
+
+test_done
-- 
2.18.0.9.g431b2c36d5

[PATCH v5 4/9] git-rebase: error out when incompatible options passed

[Elijah Newren]

git rebase has three different types: am, merge, and interactive, all of
which are implemented in terms of separate scripts.  am builds on git-am,
merge builds on git-merge-recursive, and interactive builds on
git-cherry-pick.  We make use of features in those lower-level commands in
the different rebase types, but those features don't exist in all of the
lower level commands so we have a range of incompatibilities.  Previously,
we just accepted nearly any argument and silently ignored whichever ones
weren't implemented for the type of rebase specified.  Change this so the
incompatibilities are documented, included in the testsuite, and tested
for at runtime with an appropriate error message shown.

Some exceptions I left out:

  * --merge and --interactive are technically incompatible since they are
    supposed to run different underlying scripts, but with a few small
    changes, --interactive can do everything that --merge can.  In fact,
    I'll shortly be sending another patch to remove git-rebase--merge and
    reimplement it on top of git-rebase--interactive.

  * One could argue that --interactive and --quiet are incompatible since
    --interactive doesn't implement a --quiet mode (perhaps since
    cherry-pick itself does not implement one).  However, the interactive
    mode is more quiet than the other modes in general with progress
    messages, so one could argue that it's already quiet.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh                          | 35 ++++++++++++++++++++++++++
 t/t3422-rebase-incompatible-options.sh | 16 ++++++------
 2 files changed, 43 insertions(+), 8 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index bf71b7fa20..18ac8226c4 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -503,6 +503,24 @@ then
 	git_format_patch_opt="$git_format_patch_opt --progress"
 fi
 
+if test -n "$git_am_opt"; then
+	incompatible_opts=$(echo " $git_am_opt " | \
+			    sed -e 's/ -q / /g' -e 's/^ \(.*\) $/\1/')
+	if test -n "$interactive_rebase"
+	then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine interactive options (--interactive, --exec, --rebase-merges, --preserve-merges, --keep-empty, --root + --onto) with am options ($incompatible_opts)")"
+		fi
+	fi
+	if test -n "$do_merge"; then
+		if test -n "$incompatible_opts"
+		then
+			die "$(gettext "error: cannot combine merge options (--merge, --strategy, --strategy-option) with am options ($incompatible_opts)")"
+		fi
+	fi
+fi
+
 if test -n "$signoff"
 then
 	test -n "$preserve_merges" &&
@@ -511,6 +529,23 @@ then
 	force_rebase=t
 fi
 
+if test -n "$preserve_merges"
+then
+	# Note: incompatibility with --signoff handled in signoff block above
+	# Note: incompatibility with --interactive is just a strong warning;
+	#       git-rebase.txt caveats with "unless you know what you are doing"
+	test -n "$rebase_merges" &&
+		die "$(gettext "error: cannot combine '--preserve_merges' with '--rebase-merges'")"
+fi
+
+if test -n "$rebase_merges"
+then
+	test -n "$strategy_opts" &&
+		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy-option'")"
+	test -n "$strategy" &&
+		die "$(gettext "error: cannot combine '--rebase_merges' with '--strategy'")"
+fi
+
 if test -z "$rebase_root"
 then
 	case "$#" in
diff --git a/t/t3422-rebase-incompatible-options.sh b/t/t3422-rebase-incompatible-options.sh
index b007a15eba..bb78a6ec86 100755
--- a/t/t3422-rebase-incompatible-options.sh
+++ b/t/t3422-rebase-incompatible-options.sh
@@ -33,27 +33,27 @@ test_expect_success 'setup' '
 test_rebase_am_only () {
 	opt=$1
 	shift
-	test_expect_failure "$opt incompatible with --merge" "
+	test_expect_success "$opt incompatible with --merge" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --merge A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy=ours" "
+	test_expect_success "$opt incompatible with --strategy=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --strategy-option=ours" "
+	test_expect_success "$opt incompatible with --strategy-option=ours" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --strategy-option=ours A
 	"
 
-	test_expect_failure "$opt incompatible with --interactive" "
+	test_expect_success "$opt incompatible with --interactive" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --interactive A
 	"
 
-	test_expect_failure "$opt incompatible with --exec" "
+	test_expect_success "$opt incompatible with --exec" "
 		git checkout B^0 &&
 		test_must_fail git rebase $opt --exec 'true' A
 	"
@@ -70,17 +70,17 @@ test_expect_success '--preserve-merges incompatible with --signoff' '
 	test_must_fail git rebase --preserve-merges --signoff A
 '
 
-test_expect_failure '--preserve-merges incompatible with --rebase-merges' '
+test_expect_success '--preserve-merges incompatible with --rebase-merges' '
 	git checkout B^0 &&
 	test_must_fail git rebase --preserve-merges --rebase-merges A
 '
 
-test_expect_failure '--rebase-merges incompatible with --strategy' '
+test_expect_success '--rebase-merges incompatible with --strategy' '
 	git checkout B^0 &&
 	test_must_fail git rebase --rebase-merges -s resolve A
 '
 
-test_expect_failure '--rebase-merges incompatible with --strategy-option' '
+test_expect_success '--rebase-merges incompatible with --strategy-option' '
 	git checkout B^0 &&
 	test_must_fail git rebase --rebase-merges -Xignore-space-change A
 '
-- 
2.18.0.9.g431b2c36d5

[PATCH v5 5/9] git-rebase.txt: address confusion between --no-ff vs --force-rebase

[Elijah Newren]

rebase was taught the --force-rebase option in commit b2f82e05de ("Teach
rebase to rebase even if upstream is up to date", 2009-02-13).  This flag
worked for the am and merge backends, but wasn't a valid option for the
interactive backend.

rebase was taught the --no-ff option for interactive rebases in commit
b499549401cb ("Teach rebase the --no-ff option.", 2010-03-24), to do the
exact same thing as --force-rebase does for non-interactive rebases.  This
commit explicitly documented the fact that --force-rebase was incompatible
with --interactive, though it made --no-ff a synonym for --force-rebase
for non-interactive rebases.  The choice of a new option was based on the
fact that "force rebase" didn't sound like an appropriate term for the
interactive machinery.

In commit 6bb4e485cff8 ("rebase: align variable names", 2011-02-06), the
separate parsing of command line options in the different rebase scripts
was removed, and whether on accident or because the author noticed that
these options did the same thing, the options became synonyms and both
were accepted by all three rebase types.

In commit 2d26d533a012 ("Documentation/git-rebase.txt: -f forces a rebase
that would otherwise be a no-op", 2014-08-12), which reworded the
description of the --force-rebase option, the (no-longer correct) sentence
stating that --force-rebase was incompatible with --interactive was
finally removed.

Finally, as explained at
https://public-inbox.org/git/98279912-0f52-969d-44a6-22242039387f@xiplink.com

    In the original discussion around this option [1], at one point I
    proposed teaching rebase--interactive to respect --force-rebase
    instead of adding a new option [2].  Ultimately --no-ff was chosen as
    the better user interface design [3], because an interactive rebase
    can't be "forced" to run.

We have accepted both --no-ff and --force-rebase as full synonyms for all
three rebase types for over seven years.  Documenting them differently
and in ways that suggest they might not be quite synonyms simply leads to
confusion.  Adjust the documentation to match reality.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt | 30 ++++++++++--------------------
 1 file changed, 10 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index b2d95e3fb9..2f47495a4d 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -337,16 +337,18 @@ See also INCOMPATIBLE OPTIONS below.
 +
 See also INCOMPATIBLE OPTIONS below.
 
--f::
+--no-ff::
 --force-rebase::
-	Force a rebase even if the current branch is up to date and
-	the command without `--force` would return without doing anything.
+-f::
+	Individually replay all rebased commits instead of fast-forwarding
+	over the unchanged ones.  This ensures that the entire history of
+	the rebased branch is composed of new commits.
 +
-You may find this (or --no-ff with an interactive rebase) helpful after
-reverting a topic branch merge, as this option recreates the topic branch with
-fresh commits so it can be remerged successfully without needing to "revert
-the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
+You may find this helpful after reverting a topic branch merge, as this option
+recreates the topic branch with fresh commits so it can be remerged
+successfully without needing to "revert the reversion" (see the
+link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for
+details).
 
 --fork-point::
 --no-fork-point::
@@ -498,18 +500,6 @@ See also INCOMPATIBLE OPTIONS below.
 	with care: the final stash application after a successful
 	rebase might result in non-trivial conflicts.
 
---no-ff::
-	With --interactive, cherry-pick all rebased commits instead of
-	fast-forwarding over the unchanged ones.  This ensures that the
-	entire history of the rebased branch is composed of new commits.
-+
-Without --interactive, this is a synonym for --force-rebase.
-+
-You may find this helpful after reverting a topic branch merge, as this option
-recreates the topic branch with fresh commits so it can be remerged
-successfully without needing to "revert the reversion" (see the
-link:howto/revert-a-faulty-merge.html[revert-a-faulty-merge How-To] for details).
-
 INCOMPATIBLE OPTIONS
 --------------------
 
-- 
2.18.0.9.g431b2c36d5

[PATCH v5 6/9] directory-rename-detection.txt: technical docs on abilities and limitations

[Elijah Newren]

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 .../technical/directory-rename-detection.txt  | 92 +++++++++++++++++++
 1 file changed, 92 insertions(+)
 create mode 100644 Documentation/technical/directory-rename-detection.txt

diff --git a/Documentation/technical/directory-rename-detection.txt b/Documentation/technical/directory-rename-detection.txt
new file mode 100644
index 0000000000..6e22920a39
--- /dev/null
+++ b/Documentation/technical/directory-rename-detection.txt
@@ -0,0 +1,92 @@
+Directory rename detection
+==========================
+
+Rename detection logic in diffcore-rename that checks for renames of
+individual files is aggregated and analyzed in merge-recursive for cases
+where combinations of renames indicate that a full directory has been
+renamed.
+
+Scope of abilities
+------------------
+
+It is perhaps easiest to start with an example:
+
+  * When all of x/a, x/b and x/c have moved to z/a, z/b and z/c, it is
+    likely that x/d added in the meantime would also want to move to z/d by
+    taking the hint that the entire directory 'x' moved to 'z'.
+
+More interesting possibilities exist, though, such as:
+
+  * one side of history renames x -> z, and the other renames some file to
+    x/e, causing the need for the merge to do a transitive rename.
+
+  * one side of history renames x -> z, but also renames all files within
+    x.  For example, x/a -> z/alpha, x/b -> z/bravo, etc.
+
+  * both 'x' and 'y' being merged into a single directory 'z', with a
+    directory rename being detected for both x->z and y->z.
+
+  * not all files in a directory being renamed to the same location;
+    i.e. perhaps most the files in 'x' are now found under 'z', but a few
+    are found under 'w'.
+
+  * a directory being renamed, which also contained a subdirectory that was
+    renamed to some entirely different location.  (And perhaps the inner
+    directory itself contained inner directories that were renamed to yet
+    other locations).
+
+  * combinations of the above; see t/t6043-merge-rename-directories.sh for
+    various interesting cases.
+
+Limitations -- applicability of directory renames
+-------------------------------------------------
+
+In order to prevent edge and corner cases resulting in either conflicts
+that cannot be represented in the index or which might be too complex for
+users to try to understand and resolve, a couple basic rules limit when
+directory rename detection applies:
+
+  1) If a given directory still exists on both sides of a merge, we do
+     not consider it to have been renamed.
+
+  2) If a subset of to-be-renamed files have a file or directory in the
+     way (or would be in the way of each other), "turn off" the directory
+     rename for those specific sub-paths and report the conflict to the
+     user.
+
+  3) If the other side of history did a directory rename to a path that
+     your side of history renamed away, then ignore that particular
+     rename from the other side of history for any implicit directory
+     renames (but warn the user).
+
+Limitations -- detailed rules and testcases
+-------------------------------------------
+
+t/t6043-merge-rename-directories.sh contains extensive tests and commentary
+which generate and explore the rules listed above.  It also lists a few
+additional rules:
+
+  a) If renames split a directory into two or more others, the directory
+     with the most renames, "wins".
+
+  b) Avoid directory-rename-detection for a path, if that path is the
+     source of a rename on either side of a merge.
+
+  c) Only apply implicit directory renames to directories if the other side
+     of history is the one doing the renaming.
+
+Limitations -- support in different commands
+--------------------------------------------
+
+Directory rename detection is supported by 'merge' and 'cherry-pick'.
+Other git commands which users might be surprised to see limited or no
+directory rename detection support in:
+
+  * diff
+
+    Folks have requested in the past that `git diff` detect directory
+    renames and somehow simplify its output.  It is not clear whether this
+    would be desirable or how the output should be simplified, so this was
+    simply not implemented.  Further, to implement this, directory rename
+    detection logic would need to move from merge-recursive to
+    diffcore-rename.
-- 
2.18.0.9.g431b2c36d5

[PATCH v5 7/9] git-rebase.txt: document behavioral differences between modes

[Elijah Newren]

There are a variety of aspects that are common to all rebases regardless
of which backend is in use; however, the behavior for these different
aspects varies in ways that could surprise users.  (In fact, it's not
clear -- to me at least -- that these differences were even desirable or
intentional.)  Document these differences.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-rebase.txt                  | 32 +++++++++++++++++++
 .../technical/directory-rename-detection.txt  | 23 +++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 2f47495a4d..a67df4caba 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -546,6 +546,38 @@ Other incompatible flag pairs:
  * --rebase-merges and --strategy
  * --rebase-merges and --strategy-option
 
+BEHAVIORAL DIFFERENCES
+-----------------------
+
+ * empty commits:
+
+    am-based rebase will drop any "empty" commits, whether the
+    commit started empty (had no changes relative to its parent to
+    start with) or ended empty (all changes were already applied
+    upstream in other commits).
+
+    merge-based rebase does the same.
+
+    interactive-based rebase will by default drop commits that
+    started empty and halt if it hits a commit that ended up empty.
+    The `--keep-empty` option exists for interactive rebases to allow
+    it to keep commits that started empty.
+
+  * empty commit messages:
+
+    am-based rebase will silently apply commits with empty commit
+    messages.
+
+    merge-based and interactive-based rebases will by default halt
+    on any such commits.  The `--allow-empty-message` option exists to
+    allow interactive-based rebases to apply such commits without
+    halting.
+
+  * directory rename detection:
+
+    merge-based and interactive-based rebases work fine with
+    directory rename detection.  am-based rebases sometimes do not.
+
 include::merge-strategies.txt[]
 
 NOTES
diff --git a/Documentation/technical/directory-rename-detection.txt b/Documentation/technical/directory-rename-detection.txt
index 6e22920a39..1c0086e287 100644
--- a/Documentation/technical/directory-rename-detection.txt
+++ b/Documentation/technical/directory-rename-detection.txt
@@ -90,3 +90,26 @@ directory rename detection support in:
     simply not implemented.  Further, to implement this, directory rename
     detection logic would need to move from merge-recursive to
     diffcore-rename.
+
+  * am
+
+    git-am tries to avoid a full three way merge, instead calling
+    git-apply.  That prevents us from detecting renames at all, which may
+    defeat the directory rename detection.  There is a fallback, though; if
+    the initial git-apply fails and the user has specified the -3 option,
+    git-am will fall back to a three way merge.  However, git-am lacks the
+    necessary information to do a "real" three way merge.  Instead, it has
+    to use build_fake_ancestor() to get a merge base that is missing files
+    whose rename may have been important to detect for directory rename
+    detection to function.
+
+  * rebase
+
+    Since am-based rebases work by first generating a bunch of patches
+    (which no longer record what the original commits were and thus don't
+    have the necessary info from which we can find a real merge-base), and
+    then calling git-am, this implies that am-based rebases will not always
+    successfully detect directory renames either (see the 'am' section
+    above).  merged-based rebases (rebase -m) and cherry-pick-based rebases
+    (rebase -i) are not affected by this shortcoming, and fully support
+    directory rename detection.
-- 
2.18.0.9.g431b2c36d5

[PATCH v5 8/9] t3401: add directory rename testcases for rebase and am

[Elijah Newren]

Add a simple directory rename testcase, in conjunction with each of the
types of rebases:
  git-rebase--interactive
  git-rebase--am
  git-rebase--merge
and also use the same testcase for
  git am --3way

This demonstrates a difference in behavior between the different rebase
backends in regards to directory rename detection.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3401-rebase-and-am-rename.sh | 105 ++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100755 t/t3401-rebase-and-am-rename.sh

diff --git a/t/t3401-rebase-and-am-rename.sh b/t/t3401-rebase-and-am-rename.sh
new file mode 100755
index 0000000000..8f832957fc
--- /dev/null
+++ b/t/t3401-rebase-and-am-rename.sh
@@ -0,0 +1,105 @@
+#!/bin/sh
+
+test_description='git rebase + directory rename tests'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup testcase' '
+	test_create_repo dir-rename &&
+	(
+		cd dir-rename &&
+
+		mkdir x &&
+		test_seq  1 10 >x/a &&
+		test_seq 11 20 >x/b &&
+		test_seq 21 30 >x/c &&
+		test_write_lines a b c d e f g h i >l &&
+		git add x l &&
+		git commit -m "Initial" &&
+
+		git branch O &&
+		git branch A &&
+		git branch B &&
+
+		git checkout A &&
+		git mv x y &&
+		git mv l letters &&
+		git commit -m "Rename x to y, l to letters" &&
+
+		git checkout B &&
+		echo j >>l &&
+		test_seq 31 40 >x/d &&
+		git add l x/d &&
+		git commit -m "Modify l, add x/d"
+	)
+'
+
+test_expect_success 'rebase --interactive: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		set_fake_editor &&
+		FAKE_LINES="1" git rebase --interactive A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_failure 'rebase (am): directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		git rebase A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_success 'rebase --merge: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout B^0 &&
+
+		git rebase --merge A &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_expect_failure 'am: directory rename detected' '
+	(
+		cd dir-rename &&
+
+		git checkout A^0 &&
+
+		git format-patch -1 B &&
+
+		git am --3way 0001*.patch &&
+
+		git ls-files -s >out &&
+		test_line_count = 5 out &&
+
+		test_path_is_file y/d &&
+		test_path_is_missing x/d
+	)
+'
+
+test_done
-- 
2.18.0.9.g431b2c36d5

[RFC PATCH v5 9/9] git-rebase: make --allow-empty-message the default

[Elijah Newren]

rebase backends currently behave differently with empty commit messages,
largely as a side-effect of the different underlying commands on which
they are based.  am-based rebases apply commits with an empty commit
message without stopping or requiring the user to specify an extra flag.
(It is interesting to note that am-based rebases are the default rebase
type, and no one has ever requested a --no-allow-empty-message flag to
change this behavior.)  merge-based and interactive-based rebases (which
are ultimately based on git-commit), will currently halt on any such
commits and require the user to manually specify what to do with the
commit and continue.

One possible rationale for the difference in behavior is that the purpose
of an "am" based rebase is solely to transplant an existing history, while
an "interactive" rebase is one whose purpose is to polish a series before
making it publishable.  Thus, stopping and asking for confirmation for a
possible problem is more appropriate in the latter case.  However, there
are two problems with this rationale:

  1) merge-based rebases are also non-interactive and there are multiple
     types of rebases that use the interactive machinery but are not
     explicitly interactive (e.g. when either --rebase-merges or
     --keep-empty are specified without --interactive).  These rebases are
     also used solely to transplant an existing history, and thus also
     should default to --allow-empty-message.

  2) this rationale only says that the user is more accepting of stopping
     in the case of an explicitly interactive rebase, not that stopping
     for this particular reason actually makes sense.  Exploring whether
     it makes sense, requires backing up and analyzing the underlying
     commands...

If git-commit did not error out on empty commits by default, accidental
creation of commits with empty messages would be a very common occurrence
(this check has caught me many times).  Further, nearly all such empty
commit messages would be considered an accidental error (as evidenced by a
huge amount of documentation across version control systems and in various
blog posts explaining how important commit messages are).  A simple check
for what would otherwise be a common error thus made a lot of sense, and
git-commit gained an --allow-empty-message flag for special case
overrides.  This has made commits with empty messages very rare.

There are two sources for commits with empty messages for rebase (and
cherry-pick): (a) commits created in git where the user previously
specified --allow-empty-message to git-commit, and (b) commits imported
into git from other version control systems.  In case (a), the user has
already explicitly specified that there is something special about this
commit that makes them not want to specify a commit message; forcing them
to re-specify with every cherry-pick or rebase seems more likely to be
infuriating than helpful.  In case (b), the commit is highly unlikely to
have been authored by the person who has imported the history and is doing
the rebase or cherry-pick, and thus the user is unlikely to be the
appropriate person to write a commit message for it.  Stopping and
expecting the user to modify the commit before proceeding thus seems
counter-productive.

Further, note that while empty commit messages was a common error case for
git-commit to deal with, it is a rare case for rebase (or cherry-pick).
The fact that it is rare raises the question of why it would be worth
checking and stopping on this particular condition and not others.  For
example, why doesn't an interactive rebase automatically stop if the
commit message's first line is 2000 columns long, or is missing a blank
line after the first line, or has every line indented with five spaces, or
any number of other myriad problems?

Finally, note that if a user doing an interactive rebase does have the
necessary knowledge to add a message for any such commit and wants to do
so, it is rather simple for them to change the appropriate line from
'pick' to 'reword'.  The fact that the subject is empty in the todo list
that the user edits should even serve as a way to notify them.

As far as I can tell, the fact that merge-based and interactive-based
rebases stop on commits with empty commit messages is solely a by-product
of having been based on git-commit.  It went without notice for a long
time precisely because such cases are rare.  The rareness of this
situation made it difficult to reason about, so when folks did eventually
notice this behavior, they assumed it was there for a good reason and just
added an --allow-empty-message flag.  In my opinion, stopping on such
messages not desirable in any of these cases, even the (explicitly)
interactive case.

Signed-off-by: Elijah Newren <newren@gmail.com>
---

My commit messsage seems like one of those things that someone else will
instantly see how to shrink to less than a quarter of its size while still
retaining all essential reasoning.  I can't seem to find the simple way to
state it, though.

 Documentation/git-rebase.txt  | 10 ----------
 git-rebase.sh                 |  2 +-
 t/t3404-rebase-interactive.sh |  7 ++++---
 t/t3405-rebase-malformed.sh   | 11 +++--------
 4 files changed, 8 insertions(+), 22 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index a67df4caba..9e136ee16e 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -563,16 +563,6 @@ BEHAVIORAL DIFFERENCES
     The `--keep-empty` option exists for interactive rebases to allow
     it to keep commits that started empty.
 
-  * empty commit messages:
-
-    am-based rebase will silently apply commits with empty commit
-    messages.
-
-    merge-based and interactive-based rebases will by default halt
-    on any such commits.  The `--allow-empty-message` option exists to
-    allow interactive-based rebases to apply such commits without
-    halting.
-
   * directory rename detection:
 
     merge-based and interactive-based rebases work fine with
diff --git a/git-rebase.sh b/git-rebase.sh
index 18ac8226c4..031dbd2ec8 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -95,7 +95,7 @@ rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
-allow_empty_message=
+allow_empty_message=--allow-empty-message
 signoff=
 test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
 case "$(git config --bool commit.gpgsign)" in
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 352a52e59d..81ce9fe7f9 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -553,15 +553,16 @@ test_expect_success '--continue tries to commit, even for "edit"' '
 '
 
 test_expect_success 'aborted --continue does not squash commits after "edit"' '
+	test_when_finished "git rebase --abort" &&
 	old=$(git rev-parse HEAD) &&
 	test_tick &&
 	set_fake_editor &&
 	FAKE_LINES="edit 1" git rebase -i HEAD^ &&
 	echo "edited again" > file7 &&
 	git add file7 &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " git rebase --continue &&
-	test $old = $(git rev-parse HEAD) &&
-	git rebase --abort
+	echo all the things >>conflict &&
+	test_must_fail git rebase --continue &&
+	test $old = $(git rev-parse HEAD)
 '
 
 test_expect_success 'auto-amend only edited commits after "edit"' '
diff --git a/t/t3405-rebase-malformed.sh b/t/t3405-rebase-malformed.sh
index cb7c6de84a..da94dddc86 100755
--- a/t/t3405-rebase-malformed.sh
+++ b/t/t3405-rebase-malformed.sh
@@ -77,19 +77,14 @@ test_expect_success 'rebase commit with diff in message' '
 '
 
 test_expect_success 'rebase -m commit with empty message' '
-	test_must_fail git rebase -m master empty-message-merge &&
-	git rebase --abort &&
-	git rebase -m --allow-empty-message master empty-message-merge
+	git rebase -m master empty-message-merge
 '
 
 test_expect_success 'rebase -i commit with empty message' '
 	git checkout diff-in-message &&
 	set_fake_editor &&
-	test_must_fail env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i HEAD^ &&
-	git rebase --abort &&
-	FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
-		git rebase -i --allow-empty-message HEAD^
+	env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
+		git rebase -i HEAD^
 '
 
 test_done
-- 
2.18.0.9.g431b2c36d5

[PATCH v2 0/2] Fix use of strategy options with interactive rebases

[Elijah Newren]

The interactive machinery for git rebase can accept special merge
strategies or strategy options, but has a bug in its handling of
strategy options.  This short series patches that.

Changes since v1:
  - Small change to the second commit message
  - Addition of a cover letter.  :-)

Elijah Newren (2):
  t3418: add testcase showing problems with rebase -i and strategy
    options
  Fix use of strategy options with interactive rebases

 git-rebase.sh              |  2 +-
 sequencer.c                |  7 ++++++-
 t/t3418-rebase-continue.sh | 32 ++++++++++++++++++++++++++++++++
 3 files changed, 39 insertions(+), 2 deletions(-)

 1: 68cad084cf =  1: 43b9ac5a63 t3418: add testcase showing problems with rebase -i and strategy options
 2: 8eda0e7c1e !  2: d345eb96d5 Fix use of strategy options with interactive rebases
    @@ -2,9 +2,10 @@
     
         Fix use of strategy options with interactive rebases
     
    -    git-rebase.sh wrote stuff like
    +    git-rebase.sh wrote strategy options to .git/rebase/merge/strategy_opts
    +    in the following format:
           '--ours'  '--renormalize'
    -    to .git/rebase-merge/strategy_opts.  Note the double spaces.
    +    Note the double spaces.
     
         git-rebase--interactive uses sequencer.c to parse that file, and
         sequencer.c used split_cmdline() to get the individual strategy options.
    @@ -15,10 +16,11 @@
         So, when it ended up calling try_merge_strategy(), that in turn would run
           git merge-$strategy -- --ours -- --renormalize $merge_base -- $head $remote
     
    -    instead of the expected
    +    instead of the expected/desired
           git merge-$strategy --ours --renormalize $merge_base -- $head $remote
     
    -    Remove the extra spaces so that split_cmdline() will work as expected.
    +    Remove the extra spaces so that when it goes through split_cmdline() we end
    +    up with the desired command line.
     
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
-- 
2.18.0.9.g431b2c36d5

[PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Elijah Newren]

We are not passing the same args to merge strategies when we are doing an
--interactive rebase as we do with a --merge rebase.  The merge strategy
should not need to be aware of which type of rebase is in effect.  Add a
testcase which checks for the appropriate args.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3418-rebase-continue.sh | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 03bf1b8a3b..872022106f 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
 	test -f funny.was.run
 '
 
+test_expect_failure 'rebase -i --continue handles merge strategy and options' '
+	rm -fr .git/rebase-* &&
+	git reset --hard commit-new-file-F2-on-topic-branch &&
+	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
+	test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
+	mkdir test-bin &&
+	cat >test-bin/git-merge-funny <<-EOF &&
+	#!$SHELL_PATH
+	echo "\$@" >>funny.args
+	case "\$1" in --opt) ;; *) exit 2 ;; esac
+	case "\$2" in --foo) ;; *) exit 2 ;; esac
+	case "\$4" in --) ;; *) exit 2 ;; esac
+	shift 2 &&
+	>funny.was.run &&
+	exec git merge-recursive "\$@"
+	EOF
+	chmod +x test-bin/git-merge-funny &&
+	(
+		PATH=./test-bin:$PATH
+		test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
+	) &&
+	test -f funny.was.run &&
+	rm funny.was.run &&
+	echo "Resolved" >F2 &&
+	git add F2 &&
+	(
+		PATH=./test-bin:$PATH
+		git rebase --continue
+	) &&
+	test -f funny.was.run
+'
+
 test_expect_success 'rebase passes merge strategy options correctly' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F3-on-topic-branch &&
-- 
2.18.0.9.g431b2c36d5

[PATCH v2 2/2] Fix use of strategy options with interactive rebases

[Elijah Newren]

git-rebase.sh wrote strategy options to .git/rebase/merge/strategy_opts
in the following format:
  '--ours'  '--renormalize'
Note the double spaces.

git-rebase--interactive uses sequencer.c to parse that file, and
sequencer.c used split_cmdline() to get the individual strategy options.
After splitting, sequencer.c prefixed each "option" with a double dash,
so, concatenating all its options would result in:
  -- --ours -- --renormalize

So, when it ended up calling try_merge_strategy(), that in turn would run
  git merge-$strategy -- --ours -- --renormalize $merge_base -- $head $remote

instead of the expected/desired
  git merge-$strategy --ours --renormalize $merge_base -- $head $remote

Remove the extra spaces so that when it goes through split_cmdline() we end
up with the desired command line.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh              | 2 +-
 sequencer.c                | 7 ++++++-
 t/t3418-rebase-continue.sh | 2 +-
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 19bdebb480..f3b10c7f62 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -328,7 +328,7 @@ do
 		do_merge=t
 		;;
 	--strategy-option=*)
-		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}")"
+		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}" | sed -e s/^.//)"
 		do_merge=t
 		test -z "$strategy" && strategy=recursive
 		;;
diff --git a/sequencer.c b/sequencer.c
index 5354d4d51e..ef9237c814 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2206,6 +2206,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 {
 	int i;
+	char *strategy_opts_string;
 
 	strbuf_reset(buf);
 	if (!read_oneliner(buf, rebase_path_strategy(), 0))
@@ -2214,7 +2215,11 @@ static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
 		return;
 
-	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
+	strategy_opts_string = buf->buf;
+	if (*strategy_opts_string == ' ')
+		strategy_opts_string++;
+	opts->xopts_nr = split_cmdline(strategy_opts_string,
+				       (const char ***)&opts->xopts);
 	for (i = 0; i < opts->xopts_nr; i++) {
 		const char *arg = opts->xopts[i];
 
diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 872022106f..7ca6cbc415 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -74,7 +74,7 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
 	test -f funny.was.run
 '
 
-test_expect_failure 'rebase -i --continue handles merge strategy and options' '
+test_expect_success 'rebase -i --continue handles merge strategy and options' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F2-on-topic-branch &&
 	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
-- 
2.18.0.9.g431b2c36d5

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Eric Sunshine]

On Wed, Jun 27, 2018 at 3:36 AM Elijah Newren <newren@gmail.com> wrote:
> We are not passing the same args to merge strategies when we are doing an
> --interactive rebase as we do with a --merge rebase.  The merge strategy
> should not need to be aware of which type of rebase is in effect.  Add a
> testcase which checks for the appropriate args.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
> diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
> @@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
> +test_expect_failure 'rebase -i --continue handles merge strategy and options' '
> +       rm -fr .git/rebase-* &&
> +       git reset --hard commit-new-file-F2-on-topic-branch &&
> +       test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
> +       test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
> +       mkdir test-bin &&
> +       cat >test-bin/git-merge-funny <<-EOF &&
> +       #!$SHELL_PATH
> +       echo "\$@" >>funny.args
> +       case "\$1" in --opt) ;; *) exit 2 ;; esac
> +       case "\$2" in --foo) ;; *) exit 2 ;; esac
> +       case "\$4" in --) ;; *) exit 2 ;; esac
> +       shift 2 &&
> +       >funny.was.run &&
> +       exec git merge-recursive "\$@"
> +       EOF
> +       chmod +x test-bin/git-merge-funny &&
> +       (
> +               PATH=./test-bin:$PATH

Broken &&-chain (in subshell).

> +               test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
> +       ) &&
> +       test -f funny.was.run &&
> +       rm funny.was.run &&
> +       echo "Resolved" >F2 &&
> +       git add F2 &&
> +       (
> +               PATH=./test-bin:$PATH

Ditto.

> +               git rebase --continue
> +       ) &&
> +       test -f funny.was.run
> +'

[PATCH v2] git-rebase--merge: modernize "git-$cmd" to "git $cmd"

[Elijah Newren]

Signed-off-by: Elijah Newren <newren@gmail.com>
---

Changes since v1:
  - Fixed up commit message (move below comment to below diffstat as
    originally intended)

Long term I just want to make git-rebase--merge go away, so this patch
will eventually be obsoleted.  But since I'm waiting for multiple
topics to merge down before re-submitting that series, and since that
series has some open questions as well, I figure it's worth
(re-)submitting this simple fix in the mean time.

 git-rebase--merge.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-rebase--merge.sh b/git-rebase--merge.sh
index cf4c042214..aa2f2f0872 100644
--- a/git-rebase--merge.sh
+++ b/git-rebase--merge.sh
@@ -71,7 +71,7 @@ call_merge () {
 	test -z "$strategy" && strategy=recursive
 	# If cmt doesn't have a parent, don't include it as a base
 	base=$(git rev-parse --verify --quiet $cmt^)
-	eval 'git-merge-$strategy' $strategy_opts $base ' -- "$hd" "$cmt"'
+	eval 'git merge-$strategy' $strategy_opts $base ' -- "$hd" "$cmt"'
 	rv=$?
 	case "$rv" in
 	0)
@@ -88,7 +88,7 @@ call_merge () {
 		;;
 	*)
 		die "Unknown exit code ($rv) from command:" \
-			"git-merge-$strategy $cmt^ -- HEAD $cmt"
+			"git merge-$strategy $cmt^ -- HEAD $cmt"
 		;;
 	esac
 }
-- 
2.18.0.9.g431b2c36d5

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Elijah Newren]

On Wed, Jun 27, 2018 at 12:45 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Wed, Jun 27, 2018 at 3:36 AM Elijah Newren <newren@gmail.com> wrote:
>> We are not passing the same args to merge strategies when we are doing an
>> --interactive rebase as we do with a --merge rebase.  The merge strategy
>> should not need to be aware of which type of rebase is in effect.  Add a
>> testcase which checks for the appropriate args.
>>
>> Signed-off-by: Elijah Newren <newren@gmail.com>
>> ---
>> diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
>> @@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
>> +test_expect_failure 'rebase -i --continue handles merge strategy and options' '
>> +       rm -fr .git/rebase-* &&
>> +       git reset --hard commit-new-file-F2-on-topic-branch &&
>> +       test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
>> +       test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
>> +       mkdir test-bin &&
>> +       cat >test-bin/git-merge-funny <<-EOF &&
>> +       #!$SHELL_PATH
>> +       echo "\$@" >>funny.args
>> +       case "\$1" in --opt) ;; *) exit 2 ;; esac
>> +       case "\$2" in --foo) ;; *) exit 2 ;; esac
>> +       case "\$4" in --) ;; *) exit 2 ;; esac
>> +       shift 2 &&
>> +       >funny.was.run &&
>> +       exec git merge-recursive "\$@"
>> +       EOF
>> +       chmod +x test-bin/git-merge-funny &&
>> +       (
>> +               PATH=./test-bin:$PATH
>
> Broken &&-chain (in subshell).
>
>> +               test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
>> +       ) &&
>> +       test -f funny.was.run &&
>> +       rm funny.was.run &&
>> +       echo "Resolved" >F2 &&
>> +       git add F2 &&
>> +       (
>> +               PATH=./test-bin:$PATH
>
> Ditto.
>

I'm just trying to prove how important your other patch series is.  ;-)

Doh, sorry for the mistake.  Again.  I'll fix it up.

[PATCH v3 0/2] Fix use of strategy options with interactive rebases

[Elijah Newren]

The interactive machinery for git rebase can accept special merge
strategies or strategy options, but has a bug in its handling of
strategy options.  This short series patches that.

Changes since v2:
  - Fix broken &&-chaining (Thanks to Eric for spotting)

Elijah Newren (2):
  t3418: add testcase showing problems with rebase -i and strategy
    options
  Fix use of strategy options with interactive rebases

 git-rebase.sh              |  2 +-
 sequencer.c                |  7 ++++++-
 t/t3418-rebase-continue.sh | 32 ++++++++++++++++++++++++++++++++
 3 files changed, 39 insertions(+), 2 deletions(-)

 1: 43b9ac5a63 !  1: f8a5df9ef1 t3418: add testcase showing problems with rebase -i and strategy options
    @@ -34,7 +34,7 @@
     +	EOF
     +	chmod +x test-bin/git-merge-funny &&
     +	(
    -+		PATH=./test-bin:$PATH
    ++		PATH=./test-bin:$PATH &&
     +		test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
     +	) &&
     +	test -f funny.was.run &&
    @@ -42,7 +42,7 @@
     +	echo "Resolved" >F2 &&
     +	git add F2 &&
     +	(
    -+		PATH=./test-bin:$PATH
    ++		PATH=./test-bin:$PATH &&
     +		git rebase --continue
     +	) &&
     +	test -f funny.was.run
 2: d345eb96d5 =  2: b7e4971e66 Fix use of strategy options with interactive rebases

-- 
2.18.0.9.g431b2c36d5

[PATCH v3 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Elijah Newren]

We are not passing the same args to merge strategies when we are doing an
--interactive rebase as we do with a --merge rebase.  The merge strategy
should not need to be aware of which type of rebase is in effect.  Add a
testcase which checks for the appropriate args.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3418-rebase-continue.sh | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 03bf1b8a3b..11546d6e14 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
 	test -f funny.was.run
 '
 
+test_expect_failure 'rebase -i --continue handles merge strategy and options' '
+	rm -fr .git/rebase-* &&
+	git reset --hard commit-new-file-F2-on-topic-branch &&
+	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
+	test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
+	mkdir test-bin &&
+	cat >test-bin/git-merge-funny <<-EOF &&
+	#!$SHELL_PATH
+	echo "\$@" >>funny.args
+	case "\$1" in --opt) ;; *) exit 2 ;; esac
+	case "\$2" in --foo) ;; *) exit 2 ;; esac
+	case "\$4" in --) ;; *) exit 2 ;; esac
+	shift 2 &&
+	>funny.was.run &&
+	exec git merge-recursive "\$@"
+	EOF
+	chmod +x test-bin/git-merge-funny &&
+	(
+		PATH=./test-bin:$PATH &&
+		test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
+	) &&
+	test -f funny.was.run &&
+	rm funny.was.run &&
+	echo "Resolved" >F2 &&
+	git add F2 &&
+	(
+		PATH=./test-bin:$PATH &&
+		git rebase --continue
+	) &&
+	test -f funny.was.run
+'
+
 test_expect_success 'rebase passes merge strategy options correctly' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F3-on-topic-branch &&
-- 
2.18.0.9.g431b2c36d5

[PATCH v3 2/2] Fix use of strategy options with interactive rebases

[Elijah Newren]

git-rebase.sh wrote strategy options to .git/rebase/merge/strategy_opts
in the following format:
  '--ours'  '--renormalize'
Note the double spaces.

git-rebase--interactive uses sequencer.c to parse that file, and
sequencer.c used split_cmdline() to get the individual strategy options.
After splitting, sequencer.c prefixed each "option" with a double dash,
so, concatenating all its options would result in:
  -- --ours -- --renormalize

So, when it ended up calling try_merge_strategy(), that in turn would run
  git merge-$strategy -- --ours -- --renormalize $merge_base -- $head $remote

instead of the expected/desired
  git merge-$strategy --ours --renormalize $merge_base -- $head $remote

Remove the extra spaces so that when it goes through split_cmdline() we end
up with the desired command line.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh              | 2 +-
 sequencer.c                | 7 ++++++-
 t/t3418-rebase-continue.sh | 2 +-
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index 19bdebb480..f3b10c7f62 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -328,7 +328,7 @@ do
 		do_merge=t
 		;;
 	--strategy-option=*)
-		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}")"
+		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}" | sed -e s/^.//)"
 		do_merge=t
 		test -z "$strategy" && strategy=recursive
 		;;
diff --git a/sequencer.c b/sequencer.c
index 5354d4d51e..ef9237c814 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2206,6 +2206,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 {
 	int i;
+	char *strategy_opts_string;
 
 	strbuf_reset(buf);
 	if (!read_oneliner(buf, rebase_path_strategy(), 0))
@@ -2214,7 +2215,11 @@ static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
 		return;
 
-	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
+	strategy_opts_string = buf->buf;
+	if (*strategy_opts_string == ' ')
+		strategy_opts_string++;
+	opts->xopts_nr = split_cmdline(strategy_opts_string,
+				       (const char ***)&opts->xopts);
 	for (i = 0; i < opts->xopts_nr; i++) {
 		const char *arg = opts->xopts[i];
 
diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 11546d6e14..c145dbac38 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -74,7 +74,7 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
 	test -f funny.was.run
 '
 
-test_expect_failure 'rebase -i --continue handles merge strategy and options' '
+test_expect_success 'rebase -i --continue handles merge strategy and options' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F2-on-topic-branch &&
 	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
-- 
2.18.0.9.g431b2c36d5

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

>>> +       chmod +x test-bin/git-merge-funny &&
>>> +       (
>>> +               PATH=./test-bin:$PATH
>>
>> Broken &&-chain (in subshell).
>>
>>> +               test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
>>> +       ) &&
>>> +       test -f funny.was.run &&
>>> +       rm funny.was.run &&
>>> +       echo "Resolved" >F2 &&
>>> +       git add F2 &&
>>> +       (
>>> +               PATH=./test-bin:$PATH
>>
>> Ditto.
>>
>
> I'm just trying to prove how important your other patch series is.  ;-)

Actually, this shows why the other patch series, especially its last
step, is problematic a bit.  The above assignments are followed by a
single command, i.e.

+	(
+		PATH=./test-bin:$PATH
+		test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
+	) &&

+	(
+		PATH=./test-bin:$PATH
+		git rebase --continue
+	) &&

and it would be reasonable to assume that these variable assignments
would not fail.  IOW, there is no BUG in breaking &&-chain at these
two places.  It is merely that the automated mechanism introduced by
step 29/29 of that other series does not understand that (which is
expected).  It is not wrong to have && at the end of the assignment,
though.

Having said that, it would be simpler for at least the latter to
write it using a single-shot environment assignment, perhaps?  I.e.

	PATH=./test-bin:$PATH git rebase --continue &&

without running in a subshell?

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Elijah Newren]

On Wed, Jun 27, 2018 at 9:54 AM, Junio C Hamano <gitster@pobox.com> wrote:

> +       (
> +               PATH=./test-bin:$PATH
> +               test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
> +       ) &&
>
> +       (
> +               PATH=./test-bin:$PATH
> +               git rebase --continue
> +       ) &&
>
> and it would be reasonable to assume that these variable assignments
> would not fail.  IOW, there is no BUG in breaking &&-chain at these
> two places.  It is merely that the automated mechanism introduced by
> step 29/29 of that other series does not understand that (which is
> expected).  It is not wrong to have && at the end of the assignment,
> though.
>
> Having said that, it would be simpler for at least the latter to
> write it using a single-shot environment assignment, perhaps?  I.e.
>
>         PATH=./test-bin:$PATH git rebase --continue &&
>
> without running in a subshell?

Seems reasonable.  Since these tests were essentially copies of other
tests within the same file, just for rebase -i instead of -m, should I
also add another patch to the series fixing up the rebase -m testcase
to also replace the subshell with a single-shot environment
assignment?

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Johannes Sixt]

Am 27.06.2018 um 19:27 schrieb Elijah Newren:
> On Wed, Jun 27, 2018 at 9:54 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Having said that, it would be simpler for at least the latter to
>> write it using a single-shot environment assignment, perhaps?  I.e.
>>
>>          PATH=./test-bin:$PATH git rebase --continue &&
>>
>> without running in a subshell?
> 
> Seems reasonable.  Since these tests were essentially copies of other
> tests within the same file, just for rebase -i instead of -m, should I
> also add another patch to the series fixing up the rebase -m testcase
> to also replace the subshell with a single-shot environment
> assignment?

Pitfalls ahead!

	PATH=... git rebase ...

is OK, but

	PATH=... test_must_fail git rebase ...

is not; the latter requires the subshell, otherwise the modified PATH 
variable survives the command because test_must_fail is a shell 
function. Yes, it's silly, but that's how it is.

-- Hannes

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Eric Sunshine]

On Wed, Jun 27, 2018 at 2:17 PM Johannes Sixt <j6t@kdbg.org> wrote:
> Pitfalls ahead!
>
>         PATH=... git rebase ...
>
> is OK, but
>
>         PATH=... test_must_fail git rebase ...
>
> is not; the latter requires the subshell, otherwise the modified PATH
> variable survives the command because test_must_fail is a shell
> function. Yes, it's silly, but that's how it is.

As an alternative to the subshell, some test use 'env':

    test_must_fail env PATH=... git rebase ...

Re: [PATCH v3 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[SZEDER Gábor]

> diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
> index 03bf1b8a3b..11546d6e14 100755
> --- a/t/t3418-rebase-continue.sh
> +++ b/t/t3418-rebase-continue.sh
> @@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
>  	test -f funny.was.run
>  '
>  
> +test_expect_failure 'rebase -i --continue handles merge strategy and options' '
> +	rm -fr .git/rebase-* &&
> +	git reset --hard commit-new-file-F2-on-topic-branch &&
> +	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
> +	test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
> +	mkdir test-bin &&
> +	cat >test-bin/git-merge-funny <<-EOF &&
> +	#!$SHELL_PATH
> +	echo "\$@" >>funny.args
> +	case "\$1" in --opt) ;; *) exit 2 ;; esac
> +	case "\$2" in --foo) ;; *) exit 2 ;; esac
> +	case "\$4" in --) ;; *) exit 2 ;; esac
> +	shift 2 &&
> +	>funny.was.run &&
> +	exec git merge-recursive "\$@"
> +	EOF
> +	chmod +x test-bin/git-merge-funny &&

You could use the 'write_script' helper function here.

> +	(
> +		PATH=./test-bin:$PATH &&
> +		test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
> +	) &&
> +	test -f funny.was.run &&
> +	rm funny.was.run &&
> +	echo "Resolved" >F2 &&
> +	git add F2 &&
> +	(
> +		PATH=./test-bin:$PATH &&
> +		git rebase --continue
> +	) &&
> +	test -f funny.was.run
> +'
> +
>  test_expect_success 'rebase passes merge strategy options correctly' '
>  	rm -fr .git/rebase-* &&
>  	git reset --hard commit-new-file-F3-on-topic-branch &&
> -- 
> 2.18.0.9.g431b2c36d5
> 
> 

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase

[SZEDER Gábor]

> Am 27.06.2018 um 19:27 schrieb Elijah Newren:
> > On Wed, Jun 27, 2018 at 9:54 AM, Junio C Hamano <gitster@pobox.com> wrote:
> >> Having said that, it would be simpler for at least the latter to
> >> write it using a single-shot environment assignment, perhaps?  I.e.
> >>
> >>          PATH=./test-bin:$PATH git rebase --continue &&
> >>
> >> without running in a subshell?
> > 
> > Seems reasonable.  Since these tests were essentially copies of other
> > tests within the same file, just for rebase -i instead of -m, should I
> > also add another patch to the series fixing up the rebase -m testcase
> > to also replace the subshell with a single-shot environment
> > assignment?
> 
> Pitfalls ahead!
> 
> 	PATH=... git rebase ...
> 
> is OK, but
> 
> 	PATH=... test_must_fail git rebase ...
> 
> is not; the latter requires the subshell, otherwise the modified PATH 
> variable survives the command because test_must_fail is a shell 
> function. Yes, it's silly, but that's how it is.

We have the 'test_env' helper function for this case:

  test_env PATH=./test-bin:$PATH test_must_fail git rebase --opts

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Pitfalls ahead!

Thanks.  I said "at least the latter" for this exact reason ;-).

> 	PATH=... git rebase ...
>
> is OK, but
>
> 	PATH=... test_must_fail git rebase ...
>
> is not; the latter requires the subshell, otherwise the modified PATH
> variable survives the command because test_must_fail is a shell
> function. Yes, it's silly, but that's how it is.

Or you could do

	test_must_fail env VAR=VAL git rebase ...

which I am not particularly fond of, but seems to be used quite
often.

Re: [PATCH v2 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Seems reasonable.  Since these tests were essentially copies of other
> tests within the same file, just for rebase -i instead of -m, should I
> also add another patch to the series fixing up the rebase -m testcase
> to also replace the subshell with a single-shot environment
> assignment?

I personally would think it would be best to leave that to a later
day, and take your v3 that properly &&-chains these two assignments.

It may be a good clean-up, but is an overkill if done as a
preparatory clean-up in the context of these two small fixes.

Re: [PATCH v5 0/9] Document/fix/warn about rebase incompatibilities and inconsistencies

[Phillip Wood]

Hi Elijah

On 27/06/18 08:23, Elijah Newren wrote:
> git-rebase has lots of options that are mutually incompatible.  Even among
> aspects of its behavior that is common to all rebase types, it has a number
> of inconsistencies.  This series tries to document, fix, and/or warn users
> about many of these.
> 
> Changes since v4 (short branch-diff at the end):
>   - Fixed --strategy vs. --strategy-option (in patch 3, carries over to
>     context region in later commit)
> 
> v4 didn't get a lot of feedback (though it was picked up by Junio), so I'll
> repeat the areas that would be of most note for reviewers since v3:
> 
>   * I have left patch 9 in RFC state; since v3 it has an expanded the
>     commit message with an in-depth usability rationale for the
>     change.
>   * It sounded like Junio was slightly unclear about the intent of the
>     wording in Patch 1.  Not sure if my answer (in email) was sufficient or
>     if there are wording improvements others might suggest.
>   * I'm assuming the --keep-empty and/or --empty={drop,halt,keep} (see
>     comments on patch 5 of v3) can be resolved in a later series.

I'm sorry I've not got round to looking at the last couple of versions.
Unfortunately I'm about to go off-line for a couple of weeks so I just
wanted to let you know I wasn't ignoring you!. If they haven't been
merged when I get back on-line I'll have a look then

Best Wishes

Phillip

> Elijah Newren (9):
>   git-rebase.txt: document incompatible options
>   git-rebase.sh: update help messages a bit
>   t3422: new testcases for checking when incompatible options passed
>   git-rebase: error out when incompatible options passed
>   git-rebase.txt: address confusion between --no-ff vs --force-rebase
>   directory-rename-detection.txt: technical docs on abilities and
>     limitations
>   git-rebase.txt: document behavioral differences between modes
>   t3401: add directory rename testcases for rebase and am
>   git-rebase: make --allow-empty-message the default
> 
>  Documentation/git-rebase.txt                  | 135 ++++++++++++++----
>  .../technical/directory-rename-detection.txt  | 115 +++++++++++++++
>  git-rebase.sh                                 |  43 +++++-
>  t/t3401-rebase-and-am-rename.sh               | 105 ++++++++++++++
>  t/t3404-rebase-interactive.sh                 |   7 +-
>  t/t3405-rebase-malformed.sh                   |  11 +-
>  t/t3422-rebase-incompatible-options.sh        |  88 ++++++++++++
>  7 files changed, 462 insertions(+), 42 deletions(-)
>  create mode 100644 Documentation/technical/directory-rename-detection.txt
>  create mode 100755 t/t3401-rebase-and-am-rename.sh
>  create mode 100755 t/t3422-rebase-incompatible-options.sh
> 
>  1: 3f454ebc5e =  1: 3f454ebc5e git-rebase.txt: document incompatible options
>  2: 31a5a071a6 =  2: 31a5a071a6 git-rebase.sh: update help messages a bit
>  3: 5a2b5eec79 !  3: bc3a5a3f95 t3422: new testcases for checking when incompatible options passed
>     @@ -62,7 +62,7 @@
>      +
>      +	test_expect_failure "$opt incompatible with --strategy-option=ours" "
>      +		git checkout B^0 &&
>     -+		test_must_fail git rebase $opt --strategy=ours A
>     ++		test_must_fail git rebase $opt --strategy-option=ours A
>      +	"
>      +
>      +	test_expect_failure "$opt incompatible with --interactive" "
>  4: 1e1c83724a !  4: ca3b8327f7 git-rebase: error out when incompatible options passed
>     @@ -104,7 +104,7 @@
>      -	test_expect_failure "$opt incompatible with --strategy-option=ours" "
>      +	test_expect_success "$opt incompatible with --strategy-option=ours" "
>       		git checkout B^0 &&
>     - 		test_must_fail git rebase $opt --strategy=ours A
>     + 		test_must_fail git rebase $opt --strategy-option=ours A
>       	"
>       
>      -	test_expect_failure "$opt incompatible with --interactive" "
>  5: 51023269d3 =  5: 6ac359359e git-rebase.txt: address confusion between --no-ff vs --force-rebase
>  6: f017d45dd9 =  6: e5c5db9110 directory-rename-detection.txt: technical docs on abilities and limitations
>  7: 0a359df404 =  7: e330437305 git-rebase.txt: document behavioral differences between modes
>  8: beaadceaef =  8: f704f7eee8 t3401: add directory rename testcases for rebase and am
>  9: 431b2c36d5 =  9: 436f597487 git-rebase: make --allow-empty-message the default
> 

Re: [PATCH v3 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[SZEDER Gábor]

On Wed, Jun 27, 2018 at 8:28 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>
> > diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
> > index 03bf1b8a3b..11546d6e14 100755
> > --- a/t/t3418-rebase-continue.sh
> > +++ b/t/t3418-rebase-continue.sh
> > @@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
> >       test -f funny.was.run
> >  '
> >
> > +test_expect_failure 'rebase -i --continue handles merge strategy and options' '
> > +     rm -fr .git/rebase-* &&
> > +     git reset --hard commit-new-file-F2-on-topic-branch &&
> > +     test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
> > +     test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
> > +     mkdir test-bin &&
> > +     cat >test-bin/git-merge-funny <<-EOF &&
> > +     #!$SHELL_PATH
> > +     echo "\$@" >>funny.args
> > +     case "\$1" in --opt) ;; *) exit 2 ;; esac
> > +     case "\$2" in --foo) ;; *) exit 2 ;; esac
> > +     case "\$4" in --) ;; *) exit 2 ;; esac
> > +     shift 2 &&
> > +     >funny.was.run &&
> > +     exec git merge-recursive "\$@"
> > +     EOF
> > +     chmod +x test-bin/git-merge-funny &&
>
> You could use the 'write_script' helper function here.
>
> > +     (
> > +             PATH=./test-bin:$PATH &&
> > +             test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
> > +     ) &&
> > +     test -f funny.was.run &&

And please use 'test_path_is_file' here ...

> > +     rm funny.was.run &&
> > +     echo "Resolved" >F2 &&
> > +     git add F2 &&
> > +     (
> > +             PATH=./test-bin:$PATH &&
> > +             git rebase --continue
> > +     ) &&
> > +     test -f funny.was.run

... and here.

> > +'
> > +
> >  test_expect_success 'rebase passes merge strategy options correctly' '
> >       rm -fr .git/rebase-* &&
> >       git reset --hard commit-new-file-F3-on-topic-branch &&
> > --
> > 2.18.0.9.g431b2c36d5
> >
> >

Re: [PATCH v3 1/2] t3418: add testcase showing problems with rebase -i and strategy options

[Elijah Newren]

HI SZEDER,

On Wed, Jul 11, 2018 at 3:56 AM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
> On Wed, Jun 27, 2018 at 8:28 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>>
>> > diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
>> > index 03bf1b8a3b..11546d6e14 100755
>> > --- a/t/t3418-rebase-continue.sh
>> > +++ b/t/t3418-rebase-continue.sh
>> > @@ -74,6 +74,38 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
>> >       test -f funny.was.run
>> >  '
>> >
>> > +test_expect_failure 'rebase -i --continue handles merge strategy and options' '
>> > +     rm -fr .git/rebase-* &&
>> > +     git reset --hard commit-new-file-F2-on-topic-branch &&
>> > +     test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&
>> > +     test_when_finished "rm -fr test-bin funny.was.run funny.args" &&
>> > +     mkdir test-bin &&
>> > +     cat >test-bin/git-merge-funny <<-EOF &&
>> > +     #!$SHELL_PATH
>> > +     echo "\$@" >>funny.args
>> > +     case "\$1" in --opt) ;; *) exit 2 ;; esac
>> > +     case "\$2" in --foo) ;; *) exit 2 ;; esac
>> > +     case "\$4" in --) ;; *) exit 2 ;; esac
>> > +     shift 2 &&
>> > +     >funny.was.run &&
>> > +     exec git merge-recursive "\$@"
>> > +     EOF
>> > +     chmod +x test-bin/git-merge-funny &&
>>
>> You could use the 'write_script' helper function here.
>>
>> > +     (
>> > +             PATH=./test-bin:$PATH &&
>> > +             test_must_fail git rebase -i -s funny -Xopt -Xfoo master topic
>> > +     ) &&
>> > +     test -f funny.was.run &&
>
> And please use 'test_path_is_file' here ...
>
>> > +     rm funny.was.run &&
>> > +     echo "Resolved" >F2 &&
>> > +     git add F2 &&
>> > +     (
>> > +             PATH=./test-bin:$PATH &&
>> > +             git rebase --continue
>> > +     ) &&
>> > +     test -f funny.was.run
>
> ... and here.
>

Thanks for looking over things.  This particular test was a copy of
another from the same file, just for rebase -i instead of rebase -m.
I think it'd be nice to keep the two tests looking similar and either
fix up the rebase -m case with a preparatory patch added to this
series, or wait for this series to merge and let a future series clean
up both testcases.  For other suggested fixups to the same test, Junio
suggested just taking the latter approach[1], so I'm assuming that's
how I should handle these as well.

[1] https://public-inbox.org/git/xmqqmuvgdods.fsf@gitster-ct.c.googlers.com/

Re: [PATCH v3 2/2] Fix use of strategy options with interactive rebases

[Johannes Schindelin]

Hi Elijah,

On Wed, 27 Jun 2018, Elijah Newren wrote:

> git-rebase.sh wrote strategy options to .git/rebase/merge/strategy_opts
> in the following format:
>   '--ours'  '--renormalize'
> Note the double spaces.
> 
> git-rebase--interactive uses sequencer.c to parse that file, and
> sequencer.c used split_cmdline() to get the individual strategy options.
> After splitting, sequencer.c prefixed each "option" with a double dash,
> so, concatenating all its options would result in:
>   -- --ours -- --renormalize
> 
> So, when it ended up calling try_merge_strategy(), that in turn would run
>   git merge-$strategy -- --ours -- --renormalize $merge_base -- $head $remote
> 
> instead of the expected/desired
>   git merge-$strategy --ours --renormalize $merge_base -- $head $remote
> 
> Remove the extra spaces so that when it goes through split_cmdline() we end
> up with the desired command line.
> 
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  git-rebase.sh              | 2 +-
>  sequencer.c                | 7 ++++++-
>  t/t3418-rebase-continue.sh | 2 +-
>  3 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 19bdebb480..f3b10c7f62 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -328,7 +328,7 @@ do
>  		do_merge=t
>  		;;
>  	--strategy-option=*)
> -		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}")"
> +		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}" | sed -e s/^.//)"

Didn't you mean to use "s/^ *//" instead?

>  		do_merge=t
>  		test -z "$strategy" && strategy=recursive
>  		;;
> diff --git a/sequencer.c b/sequencer.c
> index 5354d4d51e..ef9237c814 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -2206,6 +2206,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
>  static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
>  {
>  	int i;
> +	char *strategy_opts_string;
>  
>  	strbuf_reset(buf);
>  	if (!read_oneliner(buf, rebase_path_strategy(), 0))
> @@ -2214,7 +2215,11 @@ static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
>  	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
>  		return;
>  
> -	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
> +	strategy_opts_string = buf->buf;
> +	if (*strategy_opts_string == ' ')

I think that this would ideally even be a `while` instead of an `if`.

> +		strategy_opts_string++;
> +	opts->xopts_nr = split_cmdline(strategy_opts_string,
> +				       (const char ***)&opts->xopts);
>  	for (i = 0; i < opts->xopts_nr; i++) {
>  		const char *arg = opts->xopts[i];
>  
> diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
> index 11546d6e14..c145dbac38 100755
> --- a/t/t3418-rebase-continue.sh
> +++ b/t/t3418-rebase-continue.sh
> @@ -74,7 +74,7 @@ test_expect_success 'rebase --continue remembers merge strategy and options' '
>  	test -f funny.was.run
>  '
>  
> -test_expect_failure 'rebase -i --continue handles merge strategy and options' '
> +test_expect_success 'rebase -i --continue handles merge strategy and options' '
>  	rm -fr .git/rebase-* &&
>  	git reset --hard commit-new-file-F2-on-topic-branch &&
>  	test_commit "commit-new-file-F3-on-topic-branch-for-dash-i" F3 32 &&

Thanks!
Dscho

Re: [PATCH v2] git-rebase--merge: modernize "git-$cmd" to "git $cmd"

[Johannes Schindelin]

Hi Elijah,

On Wed, 27 Jun 2018, Elijah Newren wrote:

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
> 
> Changes since v1:
>   - Fixed up commit message (move below comment to below diffstat as
>     originally intended)
> 
> Long term I just want to make git-rebase--merge go away, so this patch
> will eventually be obsoleted.  But since I'm waiting for multiple
> topics to merge down before re-submitting that series, and since that
> series has some open questions as well, I figure it's worth
> (re-)submitting this simple fix in the mean time.

I carry essentially the same patch in Git for Windows for a while now
(more than a year, to be a little preciser):

https://github.com/git-for-windows/git/commit/42c6f1c943a

(but it seems that I either missed one when I wrote that commit, or I
missed when it was introduced)

There are more dashed forms in Git's code base, still, see e.g.

https://github.com/git-for-windows/git/commit/4b3fc41b117
https://github.com/git-for-windows/git/commit/c47a29c373c

I would *love* to see those go away.

FWIW I had originally also "undashed" the use of `git-receive-pack`, but
that breaks things, as the dashed form was unfortunately baked into the
protocol (which is of course a design mistake even if Linus still denies
it).

It would go a long way to help with platforms and packaging methods where
hardlinks are simply inconvenient. Because we could then finally get rid
of (almost) all those hardlinked builtins.

Ciao,
Dscho

Re: [PATCH v3 2/2] Fix use of strategy options with interactive rebases

[Elijah Newren]

Hi Dscho,

On Thu, Jul 12, 2018 at 8:41 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi Elijah,
>
> On Wed, 27 Jun 2018, Elijah Newren wrote:
>
...
>> diff --git a/git-rebase.sh b/git-rebase.sh
>> index 19bdebb480..f3b10c7f62 100755
>> --- a/git-rebase.sh
>> +++ b/git-rebase.sh
>> @@ -328,7 +328,7 @@ do
>>               do_merge=t
>>               ;;
>>       --strategy-option=*)
>> -             strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}")"
>> +             strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}" | sed -e s/^.//)"
>
> Didn't you mean to use "s/^ *//" instead?
>
>>               do_merge=t
>>               test -z "$strategy" && strategy=recursive
>>               ;;
>> diff --git a/sequencer.c b/sequencer.c
>> index 5354d4d51e..ef9237c814 100644
>> --- a/sequencer.c
>> +++ b/sequencer.c
>> @@ -2206,6 +2206,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
>>  static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
>>  {
>>       int i;
>> +     char *strategy_opts_string;
>>
>>       strbuf_reset(buf);
>>       if (!read_oneliner(buf, rebase_path_strategy(), 0))
>> @@ -2214,7 +2215,11 @@ static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
>>       if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
>>               return;
>>
>> -     opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
>> +     strategy_opts_string = buf->buf;
>> +     if (*strategy_opts_string == ' ')
>
> I think that this would ideally even be a `while` instead of an `if`.

Thanks for taking a look; both sound like good suggestions.  Since the
patch in question has already reached next, here's a patch on top of
en/rebase-i-microfixes to make these two changes:

-- 8< --
Subject: [PATCH] Whitespace handling improvements with interactive rebase
 strategy options

In commit 0060041df ("Fix use of strategy options with interactive
rebases", 2018-06-27), extra whitespace was removed from a generated
string to fix up parsing.  Instead of assuming one extra space, though, we
can just remove all leading whitespace and make the code slightly more
robust.

Suggested-by: Johanness Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-rebase.sh | 2 +-
 sequencer.c   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-rebase.sh b/git-rebase.sh
index f3b10c7f62..e572980bbc 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -328,7 +328,7 @@ do
 		do_merge=t
 		;;
 	--strategy-option=*)
-		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}" | sed -e s/^.//)"
+		strategy_opts="$strategy_opts $(git rev-parse --sq-quote "--${1#--strategy-option=}" | sed -e "s/ *//")"
 		do_merge=t
 		test -z "$strategy" && strategy=recursive
 		;;
diff --git a/sequencer.c b/sequencer.c
index ef9237c814..3f780f8f50 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2216,7 +2216,7 @@ static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 		return;
 
 	strategy_opts_string = buf->buf;
-	if (*strategy_opts_string == ' ')
+	while (*strategy_opts_string == ' ')
 		strategy_opts_string++;
 	opts->xopts_nr = split_cmdline(strategy_opts_string,
 				       (const char ***)&opts->xopts);
-- 
2.18.0.645.g72fe132ec2

Re: [PATCH v2] git-rebase--merge: modernize "git-$cmd" to "git $cmd"

[Elijah Newren]

Hi Dscho,

On Thu, Jul 12, 2018 at 8:49 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Elijah,
>
> On Wed, 27 Jun 2018, Elijah Newren wrote:
>
>> Signed-off-by: Elijah Newren <newren@gmail.com>
>> ---
>>
>> Changes since v1:
>>   - Fixed up commit message (move below comment to below diffstat as
>>     originally intended)
>>
>> Long term I just want to make git-rebase--merge go away, so this patch
>> will eventually be obsoleted.  But since I'm waiting for multiple
>> topics to merge down before re-submitting that series, and since that
>> series has some open questions as well, I figure it's worth
>> (re-)submitting this simple fix in the mean time.
>
> I carry essentially the same patch in Git for Windows for a while now
> (more than a year, to be a little preciser):
>
> https://github.com/git-for-windows/git/commit/42c6f1c943a
>
> (but it seems that I either missed one when I wrote that commit, or I
> missed when it was introduced)

So...I helped you get your work upstream without knowing it?  :-)

> There are more dashed forms in Git's code base, still, see e.g.
>
> https://github.com/git-for-windows/git/commit/4b3fc41b117
> https://github.com/git-for-windows/git/commit/c47a29c373c
>
> I would *love* to see those go away.

Are there blockers or more known work needed to get these ready for
submission, or is it more a case of you just haven't had time to
submit upstream?

> FWIW I had originally also "undashed" the use of `git-receive-pack`, but
> that breaks things, as the dashed form was unfortunately baked into the
> protocol (which is of course a design mistake even if Linus still denies
> it).
>
> It would go a long way to help with platforms and packaging methods where
> hardlinks are simply inconvenient. Because we could then finally get rid
> of (almost) all those hardlinked builtins.

I thought they were symlinked rather than hardlinked, but yeah I've
always found them slightly annoying.

Re: [PATCH v2] git-rebase--merge: modernize "git-$cmd" to "git $cmd"

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2579 bytes --]

Hi Elijah,

On Fri, 13 Jul 2018, Elijah Newren wrote:

> On Thu, Jul 12, 2018 at 8:49 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Wed, 27 Jun 2018, Elijah Newren wrote:
> >
> >> Signed-off-by: Elijah Newren <newren@gmail.com>
> >> ---
> >>
> >> Changes since v1:
> >>   - Fixed up commit message (move below comment to below diffstat as
> >>     originally intended)
> >>
> >> Long term I just want to make git-rebase--merge go away, so this patch
> >> will eventually be obsoleted.  But since I'm waiting for multiple
> >> topics to merge down before re-submitting that series, and since that
> >> series has some open questions as well, I figure it's worth
> >> (re-)submitting this simple fix in the mean time.
> >
> > I carry essentially the same patch in Git for Windows for a while now
> > (more than a year, to be a little preciser):
> >
> > https://github.com/git-for-windows/git/commit/42c6f1c943a
> >
> > (but it seems that I either missed one when I wrote that commit, or I
> > missed when it was introduced)
> 
> So...I helped you get your work upstream without knowing it?  :-)

Yes. Thank you.

> > There are more dashed forms in Git's code base, still, see e.g.
> >
> > https://github.com/git-for-windows/git/commit/4b3fc41b117
> > https://github.com/git-for-windows/git/commit/c47a29c373c
> >
> > I would *love* to see those go away.
> 
> Are there blockers or more known work needed to get these ready for
> submission, or is it more a case of you just haven't had time to
> submit upstream?

Time is the main problem.

I also meant to accompany those patches with a commit (that I did not
manage to write yet) that optoinally skips hard-linking the builtins
(except `git-receive-pack`, of course).

> > FWIW I had originally also "undashed" the use of `git-receive-pack`, but
> > that breaks things, as the dashed form was unfortunately baked into the
> > protocol (which is of course a design mistake even if Linus still denies
> > it).
> >
> > It would go a long way to help with platforms and packaging methods where
> > hardlinks are simply inconvenient. Because we could then finally get rid
> > of (almost) all those hardlinked builtins.
> 
> I thought they were symlinked rather than hardlinked, but yeah I've
> always found them slightly annoying.

They are hardlinked, with a newly-introduced option to symlink instead
that Ævar came up with IIRC.

Symlinks, of course, are just as impossible to handle portably in .zip
files as hardlinks.

Ciao,
Dscho

2.19.0 regression: leaving editor with empty commit message doesn't stop rebase [was: Re: [RFC PATCH v5 9/9] git-rebase: make --allow-empty-message the default]

[SZEDER Gábor]

Lately I noticed that occasionally I ended up with an empty commit
message after an interactive rebase.  However, since I didn't do
anything that I haven't already done countless times before, I thought
that one of my local patches touching the area where 'git commit'
calls launch_editor() got messed up.  Unfortunately, it turned out
that my patch is fine, and those empty commit messages are the
consequence of a regression in v2.19.0, which bisects down to this
patch.

To reproduce the issue, just start an interactive rebase, choose a
commit to reword, save, delete all the commit message, save, and BAM!
there is the commit with the empty message.


On Wed, Jun 27, 2018 at 12:23:19AM -0700, Elijah Newren wrote:
> rebase backends currently behave differently with empty commit messages,
> largely as a side-effect of the different underlying commands on which
> they are based.  am-based rebases apply commits with an empty commit
> message without stopping or requiring the user to specify an extra flag.
> (It is interesting to note that am-based rebases are the default rebase
> type, and no one has ever requested a --no-allow-empty-message flag to
> change this behavior.)  merge-based and interactive-based rebases (which
> are ultimately based on git-commit), will currently halt on any such
> commits and require the user to manually specify what to do with the
> commit and continue.
> 
> One possible rationale for the difference in behavior is that the purpose
> of an "am" based rebase is solely to transplant an existing history, while
> an "interactive" rebase is one whose purpose is to polish a series before
> making it publishable.  Thus, stopping and asking for confirmation for a
> possible problem is more appropriate in the latter case.  However, there
> are two problems with this rationale:
> 
>   1) merge-based rebases are also non-interactive and there are multiple
>      types of rebases that use the interactive machinery but are not
>      explicitly interactive (e.g. when either --rebase-merges or
>      --keep-empty are specified without --interactive).  These rebases are
>      also used solely to transplant an existing history, and thus also
>      should default to --allow-empty-message.
> 
>   2) this rationale only says that the user is more accepting of stopping
>      in the case of an explicitly interactive rebase, not that stopping
>      for this particular reason actually makes sense.  Exploring whether
>      it makes sense, requires backing up and analyzing the underlying
>      commands...
> 
> If git-commit did not error out on empty commits by default, accidental
> creation of commits with empty messages would be a very common occurrence
> (this check has caught me many times).  Further, nearly all such empty
> commit messages would be considered an accidental error (as evidenced by a
> huge amount of documentation across version control systems and in various
> blog posts explaining how important commit messages are).  A simple check
> for what would otherwise be a common error thus made a lot of sense, and
> git-commit gained an --allow-empty-message flag for special case
> overrides.  This has made commits with empty messages very rare.
> 
> There are two sources for commits with empty messages for rebase (and
> cherry-pick): (a) commits created in git where the user previously
> specified --allow-empty-message to git-commit, and (b) commits imported
> into git from other version control systems.  In case (a), the user has
> already explicitly specified that there is something special about this
> commit that makes them not want to specify a commit message; forcing them
> to re-specify with every cherry-pick or rebase seems more likely to be
> infuriating than helpful.  In case (b), the commit is highly unlikely to
> have been authored by the person who has imported the history and is doing
> the rebase or cherry-pick, and thus the user is unlikely to be the
> appropriate person to write a commit message for it.  Stopping and
> expecting the user to modify the commit before proceeding thus seems
> counter-productive.

I agree that this is an issue that should be addressed, and also agree
that it's reasonable to accept an empty commit message, if the
original commit already had an empty commit message.  (Though perhaps
not completely silently, but with a warning?  Dunno.)

However, this should only be the case for existing commit messages
that are taken verbatim, but never in the case when the user is asked
for a commit message.

> Further, note that while empty commit messages was a common error case for
> git-commit to deal with, it is a rare case for rebase (or cherry-pick).
> The fact that it is rare raises the question of why it would be worth
> checking and stopping on this particular condition and not others.  For
> example, why doesn't an interactive rebase automatically stop if the
> commit message's first line is 2000 columns long, or is missing a blank
> line after the first line, or has every line indented with five spaces, or
> any number of other myriad problems?

Because leaving the editor with an empty commit message is and has
always been the established way for aborting 'git commit' and thus
stopping whatever operation it was part of.

And because the instructions in the commit message template, which is
shown for 'reword' and 'squash', too, still say so:

  # Please enter the commit message for your changes. Lines starting
  # with '#' will be ignored, and an empty message aborts the commit.

> Finally, note that if a user doing an interactive rebase does have the
> necessary knowledge to add a message for any such commit and wants to do
> so, it is rather simple for them to change the appropriate line from
> 'pick' to 'reword'.  The fact that the subject is empty in the todo list
> that the user edits should even serve as a way to notify them.
> 
> As far as I can tell, the fact that merge-based and interactive-based
> rebases stop on commits with empty commit messages is solely a by-product
> of having been based on git-commit.  It went without notice for a long
> time precisely because such cases are rare.  The rareness of this
> situation made it difficult to reason about, so when folks did eventually
> notice this behavior, they assumed it was there for a good reason and just
> added an --allow-empty-message flag.  In my opinion, stopping on such
> messages not desirable in any of these cases, even the (explicitly)
> interactive case.

It is desired, and for a very good reason.

Let's suppose you start an interactive rebase, choose a commit to
squash, save the instruction sheet, rebase fires up your editor, and
then you notice that you mistakenly chose the wrong commit to squash.
What do you do, how do you abort?

Before this patch you could clear the commit message, exit the editor,
and then rebase would say "Aborting commit due to empty commit
message.", and you get to run 'git rebase --abort', and start over.

But with this patch?  I don't know...  Just saving the commit message
as is would let rebase continue and create a bunch of unnecessary
objects, and then you would have to use the reflog to return to the
pre-rebase state.


> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
> 
> My commit messsage seems like one of those things that someone else will
> instantly see how to shrink to less than a quarter of its size while still
> retaining all essential reasoning.  I can't seem to find the simple way to
> state it, though.
> 
>  Documentation/git-rebase.txt  | 10 ----------
>  git-rebase.sh                 |  2 +-
>  t/t3404-rebase-interactive.sh |  7 ++++---
>  t/t3405-rebase-malformed.sh   | 11 +++--------
>  4 files changed, 8 insertions(+), 22 deletions(-)
> 
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index a67df4caba..9e136ee16e 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -563,16 +563,6 @@ BEHAVIORAL DIFFERENCES
>      The `--keep-empty` option exists for interactive rebases to allow
>      it to keep commits that started empty.
>  
> -  * empty commit messages:
> -
> -    am-based rebase will silently apply commits with empty commit
> -    messages.
> -
> -    merge-based and interactive-based rebases will by default halt
> -    on any such commits.  The `--allow-empty-message` option exists to
> -    allow interactive-based rebases to apply such commits without
> -    halting.
> -
>    * directory rename detection:
>  
>      merge-based and interactive-based rebases work fine with
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 18ac8226c4..031dbd2ec8 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -95,7 +95,7 @@ rebase_cousins=
>  preserve_merges=
>  autosquash=
>  keep_empty=
> -allow_empty_message=
> +allow_empty_message=--allow-empty-message
>  signoff=
>  test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
>  case "$(git config --bool commit.gpgsign)" in
> diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
> index 352a52e59d..81ce9fe7f9 100755
> --- a/t/t3404-rebase-interactive.sh
> +++ b/t/t3404-rebase-interactive.sh
> @@ -553,15 +553,16 @@ test_expect_success '--continue tries to commit, even for "edit"' '
>  '
>  
>  test_expect_success 'aborted --continue does not squash commits after "edit"' '
> +	test_when_finished "git rebase --abort" &&
>  	old=$(git rev-parse HEAD) &&
>  	test_tick &&
>  	set_fake_editor &&
>  	FAKE_LINES="edit 1" git rebase -i HEAD^ &&
>  	echo "edited again" > file7 &&
>  	git add file7 &&
> -	test_must_fail env FAKE_COMMIT_MESSAGE=" " git rebase --continue &&
> -	test $old = $(git rev-parse HEAD) &&
> -	git rebase --abort
> +	echo all the things >>conflict &&
> +	test_must_fail git rebase --continue &&
> +	test $old = $(git rev-parse HEAD)
>  '
>  
>  test_expect_success 'auto-amend only edited commits after "edit"' '
> diff --git a/t/t3405-rebase-malformed.sh b/t/t3405-rebase-malformed.sh
> index cb7c6de84a..da94dddc86 100755
> --- a/t/t3405-rebase-malformed.sh
> +++ b/t/t3405-rebase-malformed.sh
> @@ -77,19 +77,14 @@ test_expect_success 'rebase commit with diff in message' '
>  '
>  
>  test_expect_success 'rebase -m commit with empty message' '
> -	test_must_fail git rebase -m master empty-message-merge &&
> -	git rebase --abort &&
> -	git rebase -m --allow-empty-message master empty-message-merge
> +	git rebase -m master empty-message-merge
>  '
>  
>  test_expect_success 'rebase -i commit with empty message' '
>  	git checkout diff-in-message &&
>  	set_fake_editor &&
> -	test_must_fail env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
> -		git rebase -i HEAD^ &&
> -	git rebase --abort &&
> -	FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
> -		git rebase -i --allow-empty-message HEAD^
> +	env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
> +		git rebase -i HEAD^
>  '
>  
>  test_done
> -- 
> 2.18.0.9.g431b2c36d5
> 

Re: 2.19.0 regression: leaving editor with empty commit message doesn't stop rebase [was: Re: [RFC PATCH v5 9/9] git-rebase: make --allow-empty-message the default]

[Elijah Newren]

Hi,

On Tue, Sep 11, 2018 at 7:42 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> Lately I noticed that occasionally I ended up with an empty commit
> message after an interactive rebase...those empty commit messages are the
> consequence of a regression in v2.19.0, which bisects down to this
> patch.
>
> To reproduce the issue, just start an interactive rebase, choose a
> commit to reword, save, delete all the commit message, save, and BAM!
> there is the commit with the empty message.
>
>
> On Wed, Jun 27, 2018 at 12:23:19AM -0700, Elijah Newren wrote:
> > rebase backends currently behave differently with empty commit messages,
> > largely as a side-effect of the different underlying commands on which
> > they are based.  am-based rebases apply commits with an empty commit
...
> I agree that this is an issue that should be addressed, and also agree
> that it's reasonable to accept an empty commit message, if the
> original commit already had an empty commit message.  (Though perhaps
> not completely silently, but with a warning?  Dunno.)
>
> However, this should only be the case for existing commit messages
> that are taken verbatim, but never in the case when the user is asked
> for a commit message.
...
>   # Please enter the commit message for your changes. Lines starting
>   # with '#' will be ignored, and an empty message aborts the commit.
...
> Let's suppose you start an interactive rebase, choose a commit to
> squash, save the instruction sheet, rebase fires up your editor, and
> then you notice that you mistakenly chose the wrong commit to squash.
> What do you do, how do you abort?

All sound like reasonable arguments to me to differentiate between
commit messages that started empty (which I admit was what I had in
mind) vs. ones where we asked for user input and it came out empty.

Are you cooking up a patch?  I might be able to find a little time to
dig into this tomorrow if no one else is working on it already by
then.  It'll probably conflict a bunch with all the
rewrite-rebase-in-C topics in flight in pu.  :-(

[PATCH] sequencer: fix --allow-empty-message behavior, make it smarter

[Elijah Newren]

In commit b00bf1c9a8dd ("git-rebase: make --allow-empty-message the
default", 2018-06-27), several arguments were given for transplanting
empty commits without halting and asking the user for confirmation on
each commit.  These arguments were incomplete because the logic clearly
assumed the only cases under consideration were transplanting of commits
with empty messages (see the comment about "There are two sources for
commits with empty messages).  It didn't discuss or even consider
rewords, squashes, etc. where the user is explicitly asked for a new
commit message and provides an empty one.  (My bad, I totally should
have thought about that at the time, but just didn't.)

Rewords and squashes are significantly different, though, as described
by SZEDER:

    Let's suppose you start an interactive rebase, choose a commit to
    squash, save the instruction sheet, rebase fires up your editor, and
    then you notice that you mistakenly chose the wrong commit to
    squash.  What do you do, how do you abort?

    Before [that commit] you could clear the commit message, exit the
    editor, and then rebase would say "Aborting commit due to empty
    commit message.", and you get to run 'git rebase --abort', and start
    over.

    But [since that commit, ...] saving the commit message as is would
    let rebase continue and create a bunch of unnecessary objects, and
    then you would have to use the reflog to return to the pre-rebase
    state.

Also, he states:

    The instructions in the commit message template, which is shown for
    'reword' and 'squash', too, still say...

    # Please enter the commit message for your changes. Lines starting
    # with '#' will be ignored, and an empty message aborts the commit.

These are sound arguments that when editing commit messages during a
sequencer operation, that if the commit message is empty then the
operation should halt and ask the user to correct.  The arguments in
commit b00bf1c9a8dd (referenced above) still apply when transplanting
previously created commits with empty commit messages, so the sequencer
should not halt for those.

Furthermore, all rationale so far applies equally for cherry-pick as for
rebase.  Therefore, make the code default to --allow-empty-message when
transplanting an existing commit, and to default to halting when the
user is asked to edit a commit message and provides an empty one -- for
both rebase and cherry-pick.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
This patch cleanly applies to both 2.19.0 and pu.  There are some related
code cleanups that I'd like to make, but doing that cleanup conflicts with
the various rewrite-rebase-in-C topics sitting in pu; since those are
fairly lengthy, I really don't want to cause problems there, but I think
SZEDER really wants this 2.19.0 regression fix before 2.20.0 and thus
before those other topics.


 sequencer.c                   |  4 ++--
 t/t3404-rebase-interactive.sh |  7 +++----
 t/t3405-rebase-malformed.sh   |  2 +-
 t/t3505-cherry-pick-empty.sh  | 18 ++++++++----------
 4 files changed, 14 insertions(+), 17 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index dc2c58d464..638ee151f2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -899,7 +899,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if ((flags & ALLOW_EMPTY))
 		argv_array_push(&cmd.args, "--allow-empty");
 
-	if (opts->allow_empty_message)
+	if (!(flags & EDIT_MSG))
 		argv_array_push(&cmd.args, "--allow-empty-message");
 
 	if (cmd.err == -1) {
@@ -1313,7 +1313,7 @@ static int try_to_commit(struct strbuf *msg, const char *author,
 
 	if (cleanup != COMMIT_MSG_CLEANUP_NONE)
 		strbuf_stripspace(msg, cleanup == COMMIT_MSG_CLEANUP_ALL);
-	if (!opts->allow_empty_message && message_is_empty(msg, cleanup)) {
+	if ((flags & EDIT_MSG) && message_is_empty(msg, cleanup)) {
 		res = 1; /* run 'git commit' to display error message */
 		goto out;
 	}
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 86bba5ed7c..ff89b6341a 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -569,16 +569,15 @@ test_expect_success '--continue tries to commit, even for "edit"' '
 '
 
 test_expect_success 'aborted --continue does not squash commits after "edit"' '
-	test_when_finished "git rebase --abort" &&
 	old=$(git rev-parse HEAD) &&
 	test_tick &&
 	set_fake_editor &&
 	FAKE_LINES="edit 1" git rebase -i HEAD^ &&
 	echo "edited again" > file7 &&
 	git add file7 &&
-	echo all the things >>conflict &&
-	test_must_fail git rebase --continue &&
-	test $old = $(git rev-parse HEAD)
+	test_must_fail env FAKE_COMMIT_MESSAGE=" " git rebase --continue &&
+	test $old = $(git rev-parse HEAD) &&
+	git rebase --abort
 '
 
 test_expect_success 'auto-amend only edited commits after "edit"' '
diff --git a/t/t3405-rebase-malformed.sh b/t/t3405-rebase-malformed.sh
index da94dddc86..860e63e444 100755
--- a/t/t3405-rebase-malformed.sh
+++ b/t/t3405-rebase-malformed.sh
@@ -83,7 +83,7 @@ test_expect_success 'rebase -m commit with empty message' '
 test_expect_success 'rebase -i commit with empty message' '
 	git checkout diff-in-message &&
 	set_fake_editor &&
-	env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
+	test_must_fail env FAKE_COMMIT_MESSAGE=" " FAKE_LINES="reword 1" \
 		git rebase -i HEAD^
 '
 
diff --git a/t/t3505-cherry-pick-empty.sh b/t/t3505-cherry-pick-empty.sh
index fbdc47cfbd..5f911bb529 100755
--- a/t/t3505-cherry-pick-empty.sh
+++ b/t/t3505-cherry-pick-empty.sh
@@ -11,17 +11,14 @@ test_expect_success setup '
 	test_tick &&
 	git commit -m "first" &&
 
-	git checkout -b empty-branch &&
-	test_tick &&
-	git commit --allow-empty -m "empty" &&
-
+	git checkout -b empty-message-branch &&
 	echo third >> file1 &&
 	git add file1 &&
 	test_tick &&
 	git commit --allow-empty-message -m "" &&
 
 	git checkout master &&
-	git checkout -b empty-branch2 &&
+	git checkout -b empty-change-branch &&
 	test_tick &&
 	git commit --allow-empty -m "empty"
 
@@ -29,7 +26,7 @@ test_expect_success setup '
 
 test_expect_success 'cherry-pick an empty commit' '
 	git checkout master &&
-	test_expect_code 1 git cherry-pick empty-branch^
+	test_expect_code 1 git cherry-pick empty-change-branch
 '
 
 test_expect_success 'index lockfile was removed' '
@@ -37,8 +34,9 @@ test_expect_success 'index lockfile was removed' '
 '
 
 test_expect_success 'cherry-pick a commit with an empty message' '
+	test_when_finished "git reset --hard empty-message-branch~1" &&
 	git checkout master &&
-	test_expect_code 1 git cherry-pick empty-branch
+	git cherry-pick empty-message-branch
 '
 
 test_expect_success 'index lockfile was removed' '
@@ -47,7 +45,7 @@ test_expect_success 'index lockfile was removed' '
 
 test_expect_success 'cherry-pick a commit with an empty message with --allow-empty-message' '
 	git checkout -f master &&
-	git cherry-pick --allow-empty-message empty-branch
+	git cherry-pick --allow-empty-message empty-message-branch
 '
 
 test_expect_success 'cherry pick an empty non-ff commit without --allow-empty' '
@@ -55,12 +53,12 @@ test_expect_success 'cherry pick an empty non-ff commit without --allow-empty' '
 	echo fourth >>file2 &&
 	git add file2 &&
 	git commit -m "fourth" &&
-	test_must_fail git cherry-pick empty-branch2
+	test_must_fail git cherry-pick empty-change-branch
 '
 
 test_expect_success 'cherry pick an empty non-ff commit with --allow-empty' '
 	git checkout master &&
-	git cherry-pick --allow-empty empty-branch2
+	git cherry-pick --allow-empty empty-change-branch
 '
 
 test_expect_success 'cherry pick with --keep-redundant-commits' '
-- 
2.19.0.612.g68a16e0245

Re: [PATCH] sequencer: fix --allow-empty-message behavior, make it smarter

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> This patch cleanly applies to both 2.19.0 and pu.  There are some related
> code cleanups that I'd like to make, but doing that cleanup conflicts with
> the various rewrite-rebase-in-C topics sitting in pu; since those are
> fairly lengthy, I really don't want to cause problems there, but I think
> SZEDER really wants this 2.19.0 regression fix before 2.20.0 and thus
> before those other topics.

Oh absolutely.  Materials for 2.19.x maintenance track can and
should jump over other topics for 2.20 and later.  Thanks for being
considerate.

> @@ -899,7 +899,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
>  	if ((flags & ALLOW_EMPTY))
>  		argv_array_push(&cmd.args, "--allow-empty");
>  
> -	if (opts->allow_empty_message)
> +	if (!(flags & EDIT_MSG))
>  		argv_array_push(&cmd.args, "--allow-empty-message");

Hmph.  I briefly wondered if an alternative logic would be better:

	If and only the original commit being rewritten is empty,
	then we allow the result to be empty.

But looking at EDIT_MSG would be more explicit and probably is a
better idea.  That would allow you to abort a reword of a commit
whose message is empty.

The reason why I thought about the alt logic is because I am worried
about a use case where

	$ GIT_EDITOR=: git rebase|cherry-pick ...

is used to say "I do not want you to go interactive, when the only
interaction needed from me is to edit the message---I am perfectly
happy with the messages of commits being replayed or ones you come
up with as the default".  Because "--allow-empty-message" tends to
make things _less_ interactive, the worry is unfounded.

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Johannes Schindelin]

Hi Elijah,

while looking for the patches I promised to review much earlier, I
realized that I never answered to this, even older mail. Sorry about that.

[On a positive note: it only has been, what, exactly 7 months since you
sent it. My record is, I believe, somewhat around 5 years before I
responded to a mail.]

On Thu, 21 Jun 2018, Elijah Newren wrote:

> Thanks for all the food for thought.  This is awesome.

:-)

> On Thu, Jun 21, 2018 at 3:57 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > On Wed, 20 Jun 2018, Elijah Newren wrote:
> >> On Sun, Jun 17, 2018 at 2:44 PM, Johannes Schindelin
> >> <Johannes.Schindelin@gmx.de> wrote:
> >>
> >> > I was really referring to speed. But I have to admit that I do not have
> >> > any current numbers.
> >> >
> >> > Another issue just hit me, though: rebase --am does not need to look at as
> >> > many Git objects as rebase --merge or rebase -i. Therefore, GVFS users
> >> > will still want to use --am wherever possible, to avoid "hydrating"
> >> > many objects during their rebase.
> >>
> >> What is it that makes rebase --am need fewer Git objects than rebase
> >> --merge or rebase -i?
> >
> > Multiple things. The most obvious thing: --cherry-pick. When you call `git
> > rebase --am`, it does not try to exclude the patches by looking at
> > `rev-list --cherry-pick --right-only <upstream>..<head>`
> >
> > This really bit us in GVFS Git because we have several thousand developers
> > working on the same code base, and you probably read the numbers elsewhere
> > how many commits are introduced while a developer goes on so much as a
> > week-long vacation.
> 
> Ooh, that's interesting.  Here's a crazy idea: I'm curious if this is
> essentially a workaround that we could expunge.

That is indeed a very, very compelling idea.

> In particular, am-based rebases will drop empty commits (that is, either
> commits that are empty to start, or that after being applied are empty).
> It never grew a --no-allow-empty option either.  If rebase -i/-m were to
> behave the same, for consistency sake, we could drop the `rev-list
> --cherry-pick ...` call.  Granted, that would result in a slight
> difference in behavior (it could result in conflicts in some cases
> instead of an automatically empty commit), but may well be worth it for
> consistencies' sake between rebase backends.  As a side effect of making
> the backends consistent, it may also provide a nice performance boost
> for the GVFS case by removing the need to do the --cherry-pick thing.

I very much like the consistency thing. I am slightly worried about the
ramifications, though.

For example, when I did not use GitGitGadget yet to submit patches, I used
one of my machines to keep a record of still-in-flight branches, and to
verify whether a branch had been merged, I simply used `git rebase -i
upstream/master`. If that showed a `noop`, I was sure that it was merged.

While I no longer have *this* particular use case, I take it as an
indication that Git users out there are probably using the interactive
rebase in similar manners.

So we would *in the least* have to keep this as an option, and we would
have to work with a deprecation notice for a couple of major versions
before we flip the default.

Having said that, I really like the performance implication of switching
the default to skip the --cherry-pick, *in particular* with an eye toward
VFS for Git (TAFKA GVFS).

> > Next, rename detection.
> >
> > I think that bit us even harder because it tries to look at all the files
> > that have been changed in upstream in the meantime, which can be *a lot*.
> > And if you know that moving files outside of your cozy little sparse
> > checkout is unlikely (or moving from the outside into your checkout), then
> > all this hydration is pretty wasteful. That's why we had to switch off
> > rename detection in GVFS Git.
> 
> Actually to be clear, for each side of the merge, rename detection
> only needs to look at files that were not present in both the
> merge-base and the branch.  Files which were modified but have the
> same name are not involved (break detection is not active.).  But yes,
> I understand how you can still get an awful lot of files, which makes
> it worse when you then go and compare all of them in an O(N*M)
> fashion.

Yes.

It is made worse by the fact that Git never sprouted an option to tell it
about *known* renames, even if that would make tons of sense. I mean, it
is nice that Git tries to detect renames. Really nice. In some cases it
fails, though, and in other cases it just takes a lot of time, so it would
be a real boon if there was a way to skip Git's rename detection. It'd be
similar in spirit to way you can side-step Git's binary file detection, by
using .gitattributes. (Obviously, the rename thing would need to use a
different facility than .gitattributes.)

> >> My guess at what objects are needed by each type:
> >>
> >> At a high level, rebase --am for each commit will need to compare the
> >> commit to its parent to generate a diff (which thus involves walking
> >> over the objects in both the commit and its parent, though it should
> >> be able to skip over subtrees that are equal), and then will need to
> >> look at all the objects in the target commit on which it needs to
> >> apply the patch (in order to properly fill the index for a starting
> >> point, and used later when creating a new commit).
> >
> > No, in --am mode, it does not even need to look at all the objects in the
> > target commits. Only those objects that correspond to the files that are
> > touched by the diff.
> 
> Sorry, I mis-spoke.  Unless you've done something special with GVFS, I
> believe the --am mode would indeed need to read all the _tree_ objects
> in the target commits, but any _file_ object corresponding to a path
> not modified by the other side need not be loaded since we can proceed
> simply knowing its sha1sum.

That *might* be good enough with the "largest repository on the planet".
Emprically, we determined pretty early on that it is not the commits and
trees that make the initial clone unbearably large. It's the blobs.

> > In a massive code base, such as Windows', this makes a huge difference.
> > Even comparing the number of files touched by the patches that are to be
> > rebased to the number of files that were touched in upstream since you
> > rebased last is ridiculous. And a three-way merge needs to consider that
> > latter set of files.
> 
> Actually, the three-way merge isn't all that different.  For a path
> that exists in both branches and the merge base, it first compares
> sha1sums (thus it does need to read all the tree objects), and then if
> one side has not modified a certain path, it knows the merge result is
> the sha1sum from the other side.  Thus, it won't need to read files
> that were changed upstream so long as that path wasn't changed on your
> side.

True.

> In fact, --merge and --interactive have a slight performance advantage
> here: if upstream didn't modify the path, then it doesn't need to read
> the file from your side to do any diff or comparison.  It can just use
> whatever sha1sum you have.  In contrast --am mode will need to read
> the relevant file more than once.  Since it first creates a series of
> patches, it will have to read the file from your commit and its
> parent, create a diff, and then later apply that diff to the target
> version, and since the target version matches the merge base it will
> end up just recovering the version of the file on your side that you
> started with anyway.  (Since that file is already local, though, this
> small performance advantage would currently be lost in the noise of
> the other problems.)

I had not thought of that. You're right, with largish patch series,
interactive rebase might actually be faster than --am based rebase. I have
not even tested this, I guess I really should first look at numbers before
I speak further :0)

> >> If the application of the diff fails, it falls back to a three-way
> >> merge, though the three-way merge shouldn't need any additional objects.
> >
> > The three-way merge needs to reconcile the diff between branch point and
> > your changes to the diff between branch point and upstream's changes. The
> > latter can be a lot bigger than the former, in which case --am's
> > complexity (O(former)) is much nicer than --merge's (O(former u latter)).
> 
> Yeah, renames mess up that whole "a path that exists in both branches
> and the merge base" requirement that I stipulated above, and widens
> the scope considerably and affects performance rather markedly.  On
> the positive side though, the more I read your description, the more I
> think my rename performance work may give a speedup for the GVFS
> usecase far in excess of the factor of 30 I saw for my usecase.  It
> may still not quite match --am mode's outright avoidance of rename
> detection, but I still think it should be pretty impressive.  Time
> will tell...

I am very hopeful.

And I also think that one fallout of your work on that front could be to
teach Git a way to skip/overrule rename detection. After all, it is not
*all* that common that files are renamed. Take Git's commit history, for
example. Apart from the huge builtin/ move (for which rename detection
will fail after many iterations after that move), there were only sporadic
renames (such as sha1_file.c -> sha1-file.c). In "real" repositories,
refactorings are also rather rare, for the same reason: they simply cause
a lot of trouble with a lot of concurrent development going on. If we had
such a facility (say, via notes) to tell Git: look, don't even bother to
try to detect renames, at least in *commit range*, just use this here data
base that I prepared for you, we would not only be able to speed up
operations such as rebase, we also would have an easy way to solve the
problem where the rename detection simply fails after sustained
development because both sides diverged *so much* that Git cannot detect
any similarity.

Another thought that occurred to me: in the context of GitGitGadget, where
I cannot rely on the `amlog` notes (because there are too many
inaccuracies in that information), I was looking for another way to
identify commits in `pu` that corresponds to the original commits (i.e.
the commits by the authors themselves, in GitGitGadget's PRs). Sadly, I
lack the time to work on this with any seriousness, I can only spend a
couple hours here and there on it, but I *think* that there might be a
chance to use something like MinHash and/or Locally Sensitive Hashes to
not only determine near-identical commits, but also correspondences
between commits and mails on the Git mailing list containing patches. And
the very same technology should also be able to speed up rename detection,
much more robustly (and with a much better time complexity) than what we
have now. Did you look in that direction already?

Ciao,
Dscho

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Elijah Newren]

Hi Dscho,

On Mon, Jan 21, 2019 at 7:56 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> while looking for the patches I promised to review much earlier, I
> realized that I never answered to this, even older mail. Sorry about that.
>
> [On a positive note: it only has been, what, exactly 7 months since you
> sent it. My record is, I believe, somewhat around 5 years before I
> responded to a mail.]

No worries; I figured it was mostly an exchange of ideas, not a
hammering down of implementation details, and as such, it was
productive with or without further responses.  In fact, I had somewhat
forgotten about this email.

> On Thu, 21 Jun 2018, Elijah Newren wrote:
>
> > Thanks for all the food for thought.  This is awesome.
>
> :-)
>
> > On Thu, Jun 21, 2018 at 3:57 AM, Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > > On Wed, 20 Jun 2018, Elijah Newren wrote:
> > >> On Sun, Jun 17, 2018 at 2:44 PM, Johannes Schindelin
> > >> <Johannes.Schindelin@gmx.de> wrote:
> > >>
....
> > >> What is it that makes rebase --am need fewer Git objects than rebase
> > >> --merge or rebase -i?
> > >
> > > Multiple things. The most obvious thing: --cherry-pick. When you call `git
....
> > Ooh, that's interesting.  Here's a crazy idea: I'm curious if this is
> > essentially a workaround that we could expunge.
>
> That is indeed a very, very compelling idea.
>
> > In particular, am-based rebases will drop empty commits (that is, either
> > commits that are empty to start, or that after being applied are empty).
> > It never grew a --no-allow-empty option either.  If rebase -i/-m were to
> > behave the same, for consistency sake, we could drop the `rev-list
> > --cherry-pick ...` call.  Granted, that would result in a slight
> > difference in behavior (it could result in conflicts in some cases
> > instead of an automatically empty commit), but may well be worth it for
> > consistencies' sake between rebase backends.  As a side effect of making
> > the backends consistent, it may also provide a nice performance boost
> > for the GVFS case by removing the need to do the --cherry-pick thing.
>
> I very much like the consistency thing. I am slightly worried about the
> ramifications, though.
>
> For example, when I did not use GitGitGadget yet to submit patches, I used
> one of my machines to keep a record of still-in-flight branches, and to
> verify whether a branch had been merged, I simply used `git rebase -i
> upstream/master`. If that showed a `noop`, I was sure that it was merged.
>
> While I no longer have *this* particular use case, I take it as an
> indication that Git users out there are probably using the interactive
> rebase in similar manners.
>
> So we would *in the least* have to keep this as an option, and we would
> have to work with a deprecation notice for a couple of major versions
> before we flip the default.
>
> Having said that, I really like the performance implication of switching
> the default to skip the --cherry-pick, *in particular* with an eye toward
> VFS for Git (TAFKA GVFS).

Handling of empty commits seems to be a bit of a mess in rebase.  --am
just drops them, with no option to do otherwise.  --interactive will
by default drop them if the commit is known apriori to be a clean
cherry-pick, but if the commit is just a subset of something else
upstream then it will instead halt with no option to just drop those
(instead requiring a 'git rebase --skip' for each one after it halts).
interactive rebase has a --keep option to make it not drop the apriori
clean cherry-picks, but will still halt on them anyway much as it does
for commits which were just a subset of upstream changes.  To me, it
would make more sense to have a keep option which doesn't halt
(perhaps acting like a combination of --keep and cherry-pick's
--keep-redundant-commits).  I've been thinking about adding a more
general
   --empty={drop,halt,keep}
option to all types of rebase.  However, I'm curious if we should also
differentiate between starts-empty and becomes-empty.  I know of a lot
of repos which use empty commits for version-related and
publish-related weirdness; perhaps commits which start empty should be
kept, even if commits which become empty during rebase (due to the
same changes having already been made upstream) should be removed.

It's still just an idea floating around in my head.

Anyway, thanks for the extra usecase and backward compatibility
context.  Before worrying about adding extra options and deprecation
periods, though, perhaps we could start by by omitting the
--cherry-pick for implicitly_interactive rebases, like --merge?  (In
particular, I'm thinking that when we switch the default rebase to be
--merge instead of --am[1], we avoid breaking things.)

[1] https://public-inbox.org/git/xmqqh8jeh1id.fsf@gitster-ct.c.googlers.com/

> > > Next, rename detection.
> > >
> > > I think that bit us even harder because it tries to look at all the files
> > > that have been changed in upstream in the meantime, which can be *a lot*.
> > > And if you know that moving files outside of your cozy little sparse
> > > checkout is unlikely (or moving from the outside into your checkout), then
> > > all this hydration is pretty wasteful. That's why we had to switch off
> > > rename detection in GVFS Git.
> >
> > Actually to be clear, for each side of the merge, rename detection
> > only needs to look at files that were not present in both the
> > merge-base and the branch.  Files which were modified but have the
> > same name are not involved (break detection is not active.).  But yes,
> > I understand how you can still get an awful lot of files, which makes
> > it worse when you then go and compare all of them in an O(N*M)
> > fashion.
>
> Yes.
>
> It is made worse by the fact that Git never sprouted an option to tell it
> about *known* renames, even if that would make tons of sense. I mean, it
> is nice that Git tries to detect renames. Really nice. In some cases it
> fails, though, and in other cases it just takes a lot of time, so it would
> be a real boon if there was a way to skip Git's rename detection. It'd be
> similar in spirit to way you can side-step Git's binary file detection, by
> using .gitattributes. (Obviously, the rename thing would need to use a
> different facility than .gitattributes.)

Hmm, I'll have to keep that in the back of my mind.  First I want to
remove all unnecessary time that rename detection takes, which can
often be huge.  But additional performance improvements ideas after
that are something I want to keep a note of.

> > >> My guess at what objects are needed by each type:
> > >>
> > >> At a high level, rebase --am for each commit will need to compare the
> > >> commit to its parent to generate a diff (which thus involves walking
> > >> over the objects in both the commit and its parent, though it should
> > >> be able to skip over subtrees that are equal), and then will need to
> > >> look at all the objects in the target commit on which it needs to
> > >> apply the patch (in order to properly fill the index for a starting
> > >> point, and used later when creating a new commit).
> > >
> > > No, in --am mode, it does not even need to look at all the objects in the
> > > target commits. Only those objects that correspond to the files that are
> > > touched by the diff.
> >
> > Sorry, I mis-spoke.  Unless you've done something special with GVFS, I
> > believe the --am mode would indeed need to read all the _tree_ objects
> > in the target commits, but any _file_ object corresponding to a path
> > not modified by the other side need not be loaded since we can proceed
> > simply knowing its sha1sum.
>
> That *might* be good enough with the "largest repository on the planet".
> Emprically, we determined pretty early on that it is not the commits and
> trees that make the initial clone unbearably large. It's the blobs.
>
> > > In a massive code base, such as Windows', this makes a huge difference.
> > > Even comparing the number of files touched by the patches that are to be
> > > rebased to the number of files that were touched in upstream since you
> > > rebased last is ridiculous. And a three-way merge needs to consider that
> > > latter set of files.
> >
> > Actually, the three-way merge isn't all that different.  For a path
> > that exists in both branches and the merge base, it first compares
> > sha1sums (thus it does need to read all the tree objects), and then if
> > one side has not modified a certain path, it knows the merge result is
> > the sha1sum from the other side.  Thus, it won't need to read files
> > that were changed upstream so long as that path wasn't changed on your
> > side.
>
> True.
>
> > In fact, --merge and --interactive have a slight performance advantage
> > here: if upstream didn't modify the path, then it doesn't need to read
> > the file from your side to do any diff or comparison.  It can just use
> > whatever sha1sum you have.  In contrast --am mode will need to read
> > the relevant file more than once.  Since it first creates a series of
> > patches, it will have to read the file from your commit and its
> > parent, create a diff, and then later apply that diff to the target
> > version, and since the target version matches the merge base it will
> > end up just recovering the version of the file on your side that you
> > started with anyway.  (Since that file is already local, though, this
> > small performance advantage would currently be lost in the noise of
> > the other problems.)
>
> I had not thought of that. You're right, with largish patch series,
> interactive rebase might actually be faster than --am based rebase. I have
> not even tested this, I guess I really should first look at numbers before
> I speak further :0)

I think merge-recursive has some performance problems that are likely
to mask this otherwise inherent advantage that interactive rebase
should enjoy over am-style rebase.  But once those are fixed...

> > >> If the application of the diff fails, it falls back to a three-way
> > >> merge, though the three-way merge shouldn't need any additional objects.
> > >
> > > The three-way merge needs to reconcile the diff between branch point and
> > > your changes to the diff between branch point and upstream's changes. The
> > > latter can be a lot bigger than the former, in which case --am's
> > > complexity (O(former)) is much nicer than --merge's (O(former u latter)).
> >
> > Yeah, renames mess up that whole "a path that exists in both branches
> > and the merge base" requirement that I stipulated above, and widens
> > the scope considerably and affects performance rather markedly.  On
> > the positive side though, the more I read your description, the more I
> > think my rename performance work may give a speedup for the GVFS
> > usecase far in excess of the factor of 30 I saw for my usecase.  It
> > may still not quite match --am mode's outright avoidance of rename
> > detection, but I still think it should be pretty impressive.  Time
> > will tell...
>
> I am very hopeful.
>
> And I also think that one fallout of your work on that front could be to
> teach Git a way to skip/overrule rename detection. After all, it is not
> *all* that common that files are renamed. Take Git's commit history, for
> example. Apart from the huge builtin/ move (for which rename detection
> will fail after many iterations after that move), there were only sporadic
> renames (such as sha1_file.c -> sha1-file.c). In "real" repositories,
> refactorings are also rather rare, for the same reason: they simply cause
> a lot of trouble with a lot of concurrent development going on. If we had
> such a facility (say, via notes) to tell Git: look, don't even bother to
> try to detect renames, at least in *commit range*, just use this here data
> base that I prepared for you, we would not only be able to speed up
> operations such as rebase, we also would have an easy way to solve the
> problem where the rename detection simply fails after sustained
> development because both sides diverged *so much* that Git cannot detect
> any similarity.
>
> Another thought that occurred to me: in the context of GitGitGadget, where
> I cannot rely on the `amlog` notes (because there are too many
> inaccuracies in that information), I was looking for another way to
> identify commits in `pu` that corresponds to the original commits (i.e.
> the commits by the authors themselves, in GitGitGadget's PRs). Sadly, I
> lack the time to work on this with any seriousness, I can only spend a
> couple hours here and there on it, but I *think* that there might be a
> chance to use something like MinHash and/or Locally Sensitive Hashes to
> not only determine near-identical commits, but also correspondences
> between commits and mails on the Git mailing list containing patches. And
> the very same technology should also be able to speed up rename detection,
> much more robustly (and with a much better time complexity) than what we
> have now. Did you look in that direction already?

I've looked at a few different things for speeding up rename
detection, but I've been away from merge-recursive while I've been
working on git repo-filter (or filter-repo or whatever I/we end up
calling it).  I'm hoping to get repo-filter into shape to share with
everyone before or at Git Merge, and then get back to merge
improvements.

I am not familiar with MinHash and Locally Sensitive Hashes, but I'll
add them to the list of things to look into.  Thanks for the idea.

Re: [RFC PATCH 3/3] git-rebase.sh: make git-rebase--interactive the default

[Johannes Schindelin]

Hi Elijah,

On Mon, 21 Jan 2019, Elijah Newren wrote:

> Before worrying about adding extra options and deprecation periods,
> though, perhaps we could start by by omitting the --cherry-pick for
> implicitly_interactive rebases, like --merge?  (In particular, I'm
> thinking that when we switch the default rebase to be --merge instead of
> --am[1], we avoid breaking things.)

There are more implicitly interactive rebase modes, though, such as
`--exec` and `--rebase-merges`. I would like to keep `--cherry-pick` for
them by default, at least I think I would.

> I am not familiar with MinHash and Locally Sensitive Hashes, but I'll
> add them to the list of things to look into.  Thanks for the idea.

Of course! Please let me know if you get to this idea, or if you find
something even better. I'd like to collaborate with you on this, as I
really need to find a better way to identify upstream commits in
GitGitGadget.

Thanks,
Dscho