[PATCH v2 00/48] remote-hg: fixes for next

[PATCH v2 00/48] remote-hg: fixes for next

[Felipe Contreras]

Hi,

Since v1 there's a few backward compatibility fixes, and new patches at the end
for new feature, some which require patches to the remote helper
infraestructure.

I've been cooking these for some time now, and they seem to be mostly ready.
The major changes are:

 1) Use our own custom push() method that allows us to push bookmarks without
    having to force a push.

 2) Switch from revision numbers to SHA-1s in the marks.

 3) Use a common repsitory so multiple remotes chan share the same objects.

 4) Cleaned up and add a ton of tests

 5) Much improved push feedback reporting

With these, we pass all the tests from gitifyhg as well as our own.

Felipe Contreras (48):
  remote-hg: test: be a little more quiet
  remote-hg: trivial reorganization
  remote-helpers: test: cleanup white-spaces
  remote-helpers: test: cleanup style
  remote-helpers: tests: general improvements
  remote-helpers: test: simplify remote URLs
  remote-hg: test: simplify previous branch checkout
  remote-hg: load all extensions
  remote-hg: use a shared repository store
  remote-hg: shuffle some code
  remote-hg: improve node traversing
  remote-hg: add version checks to the marks
  remote-hg: switch from revisions to SHA-1 noteids
  remote-hg: upgrade version 1 marks
  remote-hg: ensure remote rebasing works
  remote-hg: trivial cleanups
  remote-hg: improve progress calculation
  remote-hg: always point HEAD to master
  remote-hg: add tests for 'master' bookmark
  remote-bzr: simplify test checks
  remote-hg: add check_bookmark() test helper
  remote-hg: update bookmarks when using a remote
  remote-hg: add remote tests
  remote-hg: add test to push new bookmark
  remote-hg: add test for diverged push
  remote-hg: add test for bookmark diverge
  remote-hg: add test for new bookmark special
  remote-hg: add test for big push
  remote-hg: add test for failed double push
  remote-hg: reorganize bookmark handling
  remote-hg: force remote bookmark push selectively
  remote-hg: only update necessary revisions
  remote-hg: implement custom push()
  remote-hg: implement custom checkheads()
  remote-hg: pass around revision refs
  remote-hg: check diverged bookmarks
  remote-hg: simplify branch_tip()
  remote-hg: improve branch listing
  remote-hg: use remote 'default' not local one
  remote-hg: improve lightweight tag author
  remote-hg: remove files before modifications
  remote-hg: add setup_big_push() helper
  remote-hg: add check_push() helper
  remote-helpers: improve marks usage
  remote-hg: trivial cleanup
  remote-hg: check if a fetch is needed
  remote-hg: add support for --dry-run
  remote-hg: add support for --force

 contrib/remote-helpers/git-remote-bzr    |   5 +-
 contrib/remote-helpers/git-remote-hg     | 510 ++++++++++++++++------
 contrib/remote-helpers/test-bzr.sh       | 490 +++++++++++----------
 contrib/remote-helpers/test-hg-bidi.sh   |  33 +-
 contrib/remote-helpers/test-hg-hg-git.sh |  44 +-
 contrib/remote-helpers/test-hg.sh        | 723 +++++++++++++++++++++++++++----
 6 files changed, 1291 insertions(+), 514 deletions(-)

-- 
1.8.3.rc3.312.g47657de

[PATCH v2 01/48] remote-hg: test: be a little more quiet

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 8de2aa7..f8d1f9e 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -109,10 +109,10 @@ test_expect_success 'update bookmark' '
   (
   git clone "hg::$PWD/hgrepo" gitrepo &&
   cd gitrepo &&
-  git checkout devel &&
+  git checkout --quiet devel &&
   echo devel > content &&
   git commit -a -m devel &&
-  git push
+  git push --quiet
   ) &&
 
   hg -R hgrepo bookmarks | egrep "devel[	 ]+3:"
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 02/48] remote-hg: trivial reorganization

[Felipe Contreras]

We only need to get the remote dict once.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 1dd3d70..426c1fc 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -857,6 +857,7 @@ def do_export(parser):
 
     if peer:
         parser.repo.push(peer, force=force_push, newbranch=True)
+        remote_bmarks = peer.listkeys('bookmarks')
 
     # handle bookmarks
     for bmark, node in p_bmarks:
@@ -883,8 +884,7 @@ def do_export(parser):
             continue
 
         if peer:
-            rb = peer.listkeys('bookmarks')
-            old = rb.get(bmark, '')
+            old = remote_bmarks.get(bmark, '')
             if not peer.pushkey('bookmarks', bmark, old, new):
                 print "error %s" % ref
                 continue
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 03/48] remote-helpers: test: cleanup white-spaces

[Felipe Contreras]

We prefer tabs to spaces.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-bzr.sh | 460 ++++++++++++++++++-------------------
 contrib/remote-helpers/test-hg.sh  | 176 +++++++-------
 2 files changed, 318 insertions(+), 318 deletions(-)

diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index 5dfa070..a3220c8 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -29,69 +29,69 @@ check () {
 bzr whoami "A U Thor <author@example.com>"
 
 test_expect_success 'cloning' '
-  (bzr init bzrrepo &&
-  cd bzrrepo &&
-  echo one > content &&
-  bzr add content &&
-  bzr commit -m one
-  ) &&
-
-  git clone "bzr::$PWD/bzrrepo" gitrepo &&
-  check gitrepo one master
+	(bzr init bzrrepo &&
+	cd bzrrepo &&
+	echo one > content &&
+	bzr add content &&
+	bzr commit -m one
+	) &&
+
+	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	check gitrepo one master
 '
 
 test_expect_success 'pulling' '
-  (cd bzrrepo &&
-  echo two > content &&
-  bzr commit -m two
-  ) &&
+	(cd bzrrepo &&
+	echo two > content &&
+	bzr commit -m two
+	) &&
 
-  (cd gitrepo && git pull) &&
+	(cd gitrepo && git pull) &&
 
-  check gitrepo two master
+	check gitrepo two master
 '
 
 test_expect_success 'pushing' '
-  (cd gitrepo &&
-  echo three > content &&
-  git commit -a -m three &&
-  git push
-  ) &&
-
-  echo three > expected &&
-  cat bzrrepo/content > actual &&
-  test_cmp expected actual
+	(cd gitrepo &&
+	echo three > content &&
+	git commit -a -m three &&
+	git push
+	) &&
+
+	echo three > expected &&
+	cat bzrrepo/content > actual &&
+	test_cmp expected actual
 '
 
 test_expect_success 'roundtrip' '
-  (cd gitrepo &&
-  git pull &&
-  git log --format="%s" -1 origin/master > actual) &&
-  echo three > expected &&
-  test_cmp expected actual &&
+	(cd gitrepo &&
+	git pull &&
+	git log --format="%s" -1 origin/master > actual) &&
+	echo three > expected &&
+	test_cmp expected actual &&
 
-  (cd gitrepo && git push && git pull) &&
+	(cd gitrepo && git push && git pull) &&
 
-  (cd bzrrepo &&
-  echo four > content &&
-  bzr commit -m four
-  ) &&
+	(cd bzrrepo &&
+	echo four > content &&
+	bzr commit -m four
+	) &&
 
-  (cd gitrepo && git pull && git push) &&
+	(cd gitrepo && git pull && git push) &&
 
-  check gitrepo four master &&
+	check gitrepo four master &&
 
-  (cd gitrepo &&
-  echo five > content &&
-  git commit -a -m five &&
-  git push && git pull
-  ) &&
+	(cd gitrepo &&
+	echo five > content &&
+	git commit -a -m five &&
+	git push && git pull
+	) &&
 
-  (cd bzrrepo && bzr revert) &&
+	(cd bzrrepo && bzr revert) &&
 
-  echo five > expected &&
-  cat bzrrepo/content > actual &&
-  test_cmp expected actual
+	echo five > expected &&
+	cat bzrrepo/content > actual &&
+	test_cmp expected actual
 '
 
 cat > expected <<EOF
@@ -101,29 +101,29 @@ cat > expected <<EOF
 EOF
 
 test_expect_success 'special modes' '
-  (cd bzrrepo &&
-  echo exec > executable
-  chmod +x executable &&
-  bzr add executable
-  bzr commit -m exec &&
-  ln -s content link
-  bzr add link
-  bzr commit -m link &&
-  mkdir dir &&
-  bzr add dir &&
-  bzr commit -m dir) &&
-
-  (cd gitrepo &&
-  git pull
-  git ls-tree HEAD > ../actual) &&
-
-  test_cmp expected actual &&
-
-  (cd gitrepo &&
-  git cat-file -p HEAD:link > ../actual) &&
-
-  printf content > expected &&
-  test_cmp expected actual
+	(cd bzrrepo &&
+	echo exec > executable
+	chmod +x executable &&
+	bzr add executable
+	bzr commit -m exec &&
+	ln -s content link
+	bzr add link
+	bzr commit -m link &&
+	mkdir dir &&
+	bzr add dir &&
+	bzr commit -m dir) &&
+
+	(cd gitrepo &&
+	git pull
+	git ls-tree HEAD > ../actual) &&
+
+	test_cmp expected actual &&
+
+	(cd gitrepo &&
+	git cat-file -p HEAD:link > ../actual) &&
+
+	printf content > expected &&
+	test_cmp expected actual
 '
 
 cat > expected <<EOF
@@ -134,134 +134,134 @@ cat > expected <<EOF
 EOF
 
 test_expect_success 'moving directory' '
-  (cd bzrrepo &&
-  mkdir movedir &&
-  echo one > movedir/one &&
-  echo two > movedir/two &&
-  bzr add movedir &&
-  bzr commit -m movedir &&
-  bzr mv movedir movedir-new &&
-  bzr commit -m movedir-new) &&
-
-  (cd gitrepo &&
-  git pull &&
-  git ls-tree HEAD > ../actual) &&
-
-  test_cmp expected actual
+	(cd bzrrepo &&
+	mkdir movedir &&
+	echo one > movedir/one &&
+	echo two > movedir/two &&
+	bzr add movedir &&
+	bzr commit -m movedir &&
+	bzr mv movedir movedir-new &&
+	bzr commit -m movedir-new) &&
+
+	(cd gitrepo &&
+	git pull &&
+	git ls-tree HEAD > ../actual) &&
+
+	test_cmp expected actual
 '
 
 test_expect_success 'different authors' '
-  (cd bzrrepo &&
-  echo john >> content &&
-  bzr commit -m john \
-    --author "Jane Rey <jrey@example.com>" \
-    --author "John Doe <jdoe@example.com>") &&
-
-  (cd gitrepo &&
-  git pull &&
-  git show --format="%an <%ae>, %cn <%ce>" --quiet > ../actual) &&
-
-  echo "Jane Rey <jrey@example.com>, A U Thor <author@example.com>" > expected &&
-  test_cmp expected actual
+	(cd bzrrepo &&
+	echo john >> content &&
+	bzr commit -m john \
+	  --author "Jane Rey <jrey@example.com>" \
+	  --author "John Doe <jdoe@example.com>") &&
+
+	(cd gitrepo &&
+	git pull &&
+	git show --format="%an <%ae>, %cn <%ce>" --quiet > ../actual) &&
+
+	echo "Jane Rey <jrey@example.com>, A U Thor <author@example.com>" > expected &&
+	test_cmp expected actual
 '
 
 test_expect_success 'fetch utf-8 filenames' '
-  mkdir -p tmp && cd tmp &&
-  test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
-
-  LC_ALL=en_US.UTF-8
-  export LC_ALL
-  (
-  bzr init bzrrepo &&
-  cd bzrrepo &&
-
-  echo test >> "ærø" &&
-  bzr add "ærø" &&
-  echo test >> "ø~?" &&
-  bzr add "ø~?" &&
-  bzr commit -m add-utf-8 &&
-  echo test >> "ærø" &&
-  bzr commit -m test-utf-8 &&
-  bzr rm "ø~?" &&
-  bzr mv "ærø" "ø~?" &&
-  bzr commit -m bzr-mv-utf-8
-  ) &&
-
-  (
-  git clone "bzr::$PWD/bzrrepo" gitrepo &&
-  cd gitrepo &&
-  git -c core.quotepath=false ls-files > ../actual
-  ) &&
-  echo "ø~?" > expected &&
-  test_cmp expected actual
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
+
+	LC_ALL=en_US.UTF-8
+	export LC_ALL
+	(
+	bzr init bzrrepo &&
+	cd bzrrepo &&
+
+	echo test >> "ærø" &&
+	bzr add "ærø" &&
+	echo test >> "ø~?" &&
+	bzr add "ø~?" &&
+	bzr commit -m add-utf-8 &&
+	echo test >> "ærø" &&
+	bzr commit -m test-utf-8 &&
+	bzr rm "ø~?" &&
+	bzr mv "ærø" "ø~?" &&
+	bzr commit -m bzr-mv-utf-8
+	) &&
+
+	(
+	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	cd gitrepo &&
+	git -c core.quotepath=false ls-files > ../actual
+	) &&
+	echo "ø~?" > expected &&
+	test_cmp expected actual
 '
 
 test_expect_success 'push utf-8 filenames' '
-  mkdir -p tmp && cd tmp &&
-  test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
 
-  LC_ALL=en_US.UTF-8
-  export LC_ALL
+	LC_ALL=en_US.UTF-8
+	export LC_ALL
 
-  (
-  bzr init bzrrepo &&
-  cd bzrrepo &&
+	(
+	bzr init bzrrepo &&
+	cd bzrrepo &&
 
-  echo one >> content &&
-  bzr add content &&
-  bzr commit -m one
-  ) &&
+	echo one >> content &&
+	bzr add content &&
+	bzr commit -m one
+	) &&
 
-  (
-  git clone "bzr::$PWD/bzrrepo" gitrepo &&
-  cd gitrepo &&
+	(
+	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	cd gitrepo &&
 
-  echo test >> "ærø" &&
-  git add "ærø" &&
-  git commit -m utf-8 &&
+	echo test >> "ærø" &&
+	git add "ærø" &&
+	git commit -m utf-8 &&
 
-  git push
-  ) &&
+	git push
+	) &&
 
-  (cd bzrrepo && bzr ls > ../actual) &&
-  printf "content\nærø\n" > expected &&
-  test_cmp expected actual
+	(cd bzrrepo && bzr ls > ../actual) &&
+	printf "content\nærø\n" > expected &&
+	test_cmp expected actual
 '
 
 test_expect_success 'pushing a merge' '
-  mkdir -p tmp && cd tmp &&
-  test_when_finished "cd .. && rm -rf tmp" &&
-
-  (
-  bzr init bzrrepo &&
-  cd bzrrepo &&
-  echo one > content &&
-  bzr add content &&
-  bzr commit -m one
-  ) &&
-
-  git clone "bzr::$PWD/bzrrepo" gitrepo &&
-
-  (
-  cd bzrrepo &&
-  echo two > content &&
-  bzr commit -m two
-  ) &&
-
-  (
-  cd gitrepo &&
-  echo three > content &&
-  git commit -a -m three &&
-  git fetch &&
-  git merge origin/master || true &&
-  echo three > content &&
-  git commit -a --no-edit &&
-  git push
-  ) &&
-
-  echo three > expected &&
-  cat bzrrepo/content > actual &&
-  test_cmp expected actual
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	bzr init bzrrepo &&
+	cd bzrrepo &&
+	echo one > content &&
+	bzr add content &&
+	bzr commit -m one
+	) &&
+
+	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+
+	(
+	cd bzrrepo &&
+	echo two > content &&
+	bzr commit -m two
+	) &&
+
+	(
+	cd gitrepo &&
+	echo three > content &&
+	git commit -a -m three &&
+	git fetch &&
+	git merge origin/master || true &&
+	echo three > content &&
+	git commit -a --no-edit &&
+	git push
+	) &&
+
+	echo three > expected &&
+	cat bzrrepo/content > actual &&
+	test_cmp expected actual
 '
 
 cat > expected <<EOF
@@ -271,71 +271,71 @@ origin/trunk
 EOF
 
 test_expect_success 'proper bzr repo' '
-  mkdir -p tmp && cd tmp &&
-  test_when_finished "cd .. && rm -rf tmp" &&
-
-  bzr init-repo bzrrepo &&
-
-  bzr init bzrrepo/trunk &&
-  (
-  cd bzrrepo/trunk &&
-  echo one >> content &&
-  bzr add content &&
-  bzr commit -m one
-  ) &&
-
-  bzr branch bzrrepo/trunk bzrrepo/branch &&
-  (
-  cd bzrrepo/branch &&
-  echo two >> content &&
-  bzr commit -m one
-  ) &&
-
-  git clone "bzr::$PWD/bzrrepo" gitrepo &&
-  (
-  cd gitrepo &&
-  git for-each-ref --format "%(refname:short)" refs/remotes/origin > ../actual
-  ) &&
-
-  test_cmp ../expected actual
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	bzr init-repo bzrrepo &&
+
+	bzr init bzrrepo/trunk &&
+	(
+	cd bzrrepo/trunk &&
+	echo one >> content &&
+	bzr add content &&
+	bzr commit -m one
+	) &&
+
+	bzr branch bzrrepo/trunk bzrrepo/branch &&
+	(
+	cd bzrrepo/branch &&
+	echo two >> content &&
+	bzr commit -m one
+	) &&
+
+	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	(
+	cd gitrepo &&
+	git for-each-ref --format "%(refname:short)" refs/remotes/origin > ../actual
+	) &&
+
+	test_cmp ../expected actual
 '
 
 test_expect_success 'strip' '
-  # Do not imitate this style; always chdir inside a subshell instead
-  mkdir -p tmp && cd tmp &&
-  test_when_finished "cd .. && rm -rf tmp" &&
+	# Do not imitate this style; always chdir inside a subshell instead
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
 
-  (
-  bzr init bzrrepo &&
-  cd bzrrepo &&
+	(
+	bzr init bzrrepo &&
+	cd bzrrepo &&
 
-  echo one >> content &&
-  bzr add content &&
-  bzr commit -m one &&
+	echo one >> content &&
+	bzr add content &&
+	bzr commit -m one &&
 
-  echo two >> content &&
-  bzr commit -m two
-  ) &&
+	echo two >> content &&
+	bzr commit -m two
+	) &&
 
-  git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	git clone "bzr::$PWD/bzrrepo" gitrepo &&
 
-  (
-  cd bzrrepo &&
-  bzr uncommit --force &&
+	(
+	cd bzrrepo &&
+	bzr uncommit --force &&
 
-  echo three >> content &&
-  bzr commit -m three &&
+	echo three >> content &&
+	bzr commit -m three &&
 
-  echo four >> content &&
-  bzr commit -m four &&
-  bzr log --line | sed -e "s/^[0-9][0-9]*: //" > ../expected
-  ) &&
+	echo four >> content &&
+	bzr commit -m four &&
+	bzr log --line | sed -e "s/^[0-9][0-9]*: //" > ../expected
+	) &&
 
-  (cd gitrepo &&
-  git fetch &&
-  git log --format="%an %ad %s" --date=short origin/master > ../actual) &&
+	(cd gitrepo &&
+	git fetch &&
+	git log --format="%an %ad %s" --date=short origin/master > ../actual) &&
 
-  test_cmp expected actual
+	test_cmp expected actual
 '
 
 test_done
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index f8d1f9e..dac3756 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -39,119 +39,119 @@ setup () {
 setup
 
 test_expect_success 'cloning' '
-  test_when_finished "rm -rf gitrepo*" &&
-
-  (
-  hg init hgrepo &&
-  cd hgrepo &&
-  echo zero > content &&
-  hg add content &&
-  hg commit -m zero
-  ) &&
-
-  git clone "hg::$PWD/hgrepo" gitrepo &&
-  check gitrepo zero master
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+	echo zero > content &&
+	hg add content &&
+	hg commit -m zero
+	) &&
+
+	git clone "hg::$PWD/hgrepo" gitrepo &&
+	check gitrepo zero master
 '
 
 test_expect_success 'cloning with branches' '
-  test_when_finished "rm -rf gitrepo*" &&
+	test_when_finished "rm -rf gitrepo*" &&
 
-  (
-  cd hgrepo &&
-  hg branch next &&
-  echo next > content &&
-  hg commit -m next
-  ) &&
+	(
+	cd hgrepo &&
+	hg branch next &&
+	echo next > content &&
+	hg commit -m next
+	) &&
 
-  git clone "hg::$PWD/hgrepo" gitrepo &&
-  check gitrepo next next &&
+	git clone "hg::$PWD/hgrepo" gitrepo &&
+	check gitrepo next next &&
 
-  (cd hgrepo && hg checkout default) &&
+	(cd hgrepo && hg checkout default) &&
 
-  git clone "hg::$PWD/hgrepo" gitrepo2 &&
-  check gitrepo2 zero master
+	git clone "hg::$PWD/hgrepo" gitrepo2 &&
+	check gitrepo2 zero master
 '
 
 test_expect_success 'cloning with bookmarks' '
-  test_when_finished "rm -rf gitrepo*" &&
-
-  (
-  cd hgrepo &&
-  hg bookmark feature-a &&
-  echo feature-a > content &&
-  hg commit -m feature-a
-  ) &&
+	test_when_finished "rm -rf gitrepo*" &&
 
-  git clone "hg::$PWD/hgrepo" gitrepo &&
-  check gitrepo feature-a feature-a
+	(
+	cd hgrepo &&
+	hg bookmark feature-a &&
+	echo feature-a > content &&
+	hg commit -m feature-a
+	) &&
+
+	git clone "hg::$PWD/hgrepo" gitrepo &&
+	check gitrepo feature-a feature-a
 '
 
 test_expect_success 'cloning with detached head' '
-  test_when_finished "rm -rf gitrepo*" &&
+	test_when_finished "rm -rf gitrepo*" &&
 
-  (
-  cd hgrepo &&
-  hg update -r 0
-  ) &&
+	(
+	cd hgrepo &&
+	hg update -r 0
+	) &&
 
-  git clone "hg::$PWD/hgrepo" gitrepo &&
-  check gitrepo zero master
+	git clone "hg::$PWD/hgrepo" gitrepo &&
+	check gitrepo zero master
 '
 
 test_expect_success 'update bookmark' '
-  test_when_finished "rm -rf gitrepo*" &&
-
-  (
-  cd hgrepo &&
-  hg bookmark devel
-  ) &&
-
-  (
-  git clone "hg::$PWD/hgrepo" gitrepo &&
-  cd gitrepo &&
-  git checkout --quiet devel &&
-  echo devel > content &&
-  git commit -a -m devel &&
-  git push --quiet
-  ) &&
-
-  hg -R hgrepo bookmarks | egrep "devel[	 ]+3:"
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	cd hgrepo &&
+	hg bookmark devel
+	) &&
+
+	(
+	git clone "hg::$PWD/hgrepo" gitrepo &&
+	cd gitrepo &&
+	git checkout --quiet devel &&
+	echo devel > content &&
+	git commit -a -m devel &&
+	git push --quiet
+	) &&
+
+	hg -R hgrepo bookmarks | egrep "devel[	 ]+3:"
 '
 
 author_test () {
-  echo $1 >> content &&
-  hg commit -u "$2" -m "add $1" &&
-  echo "$3" >> ../expected
+	echo $1 >> content &&
+	hg commit -u "$2" -m "add $1" &&
+	echo "$3" >> ../expected
 }
 
 test_expect_success 'authors' '
-  mkdir -p tmp && cd tmp &&
-  test_when_finished "cd .. && rm -rf tmp" &&
-
-  (
-  hg init hgrepo &&
-  cd hgrepo &&
-
-  touch content &&
-  hg add content &&
-
-  author_test alpha "" "H G Wells <wells@example.com>" &&
-  author_test beta "test" "test <unknown>" &&
-  author_test beta "test <test@example.com> (comment)" "test <test@example.com>" &&
-  author_test gamma "<test@example.com>" "Unknown <test@example.com>" &&
-  author_test delta "name<test@example.com>" "name <test@example.com>" &&
-  author_test epsilon "name <test@example.com" "name <test@example.com>" &&
-  author_test zeta " test " "test <unknown>" &&
-  author_test eta "test < test@example.com >" "test <test@example.com>" &&
-  author_test theta "test >test@example.com>" "test <test@example.com>" &&
-  author_test iota "test < test <at> example <dot> com>" "test <unknown>" &&
-  author_test kappa "test@example.com" "Unknown <test@example.com>"
-  ) &&
-
-  git clone "hg::$PWD/hgrepo" gitrepo &&
-  git --git-dir=gitrepo/.git log --reverse --format="%an <%ae>" > actual &&
-
-  test_cmp expected actual
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+
+	touch content &&
+	hg add content &&
+
+	author_test alpha "" "H G Wells <wells@example.com>" &&
+	author_test beta "test" "test <unknown>" &&
+	author_test beta "test <test@example.com> (comment)" "test <test@example.com>" &&
+	author_test gamma "<test@example.com>" "Unknown <test@example.com>" &&
+	author_test delta "name<test@example.com>" "name <test@example.com>" &&
+	author_test epsilon "name <test@example.com" "name <test@example.com>" &&
+	author_test zeta " test " "test <unknown>" &&
+	author_test eta "test < test@example.com >" "test <test@example.com>" &&
+	author_test theta "test >test@example.com>" "test <test@example.com>" &&
+	author_test iota "test < test <at> example <dot> com>" "test <unknown>" &&
+	author_test kappa "test@example.com" "Unknown <test@example.com>"
+	) &&
+
+	git clone "hg::$PWD/hgrepo" gitrepo &&
+	git --git-dir=gitrepo/.git log --reverse --format="%an <%ae>" > actual &&
+
+	test_cmp expected actual
 '
 
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 04/48] remote-helpers: test: cleanup style

[Felipe Contreras]

So it's more standardized between all the tests.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-bzr.sh | 88 +++++++++++++++++++++++++-------------
 contrib/remote-helpers/test-hg.sh  | 12 ++++--
 2 files changed, 66 insertions(+), 34 deletions(-)

diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index a3220c8..f760110 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -18,18 +18,23 @@ if ! "$PYTHON_PATH" -c 'import bzrlib'; then
 fi
 
 check () {
-	(cd $1 &&
+	(
+	cd $1 &&
 	git log --format='%s' -1 &&
-	git symbolic-ref HEAD) > actual &&
-	(echo $2 &&
-	echo "refs/heads/$3") > expected &&
+	git symbolic-ref HEAD
+	) > actual &&
+	(
+	echo $2 &&
+	echo "refs/heads/$3"
+	) > expected &&
 	test_cmp expected actual
 }
 
 bzr whoami "A U Thor <author@example.com>"
 
 test_expect_success 'cloning' '
-	(bzr init bzrrepo &&
+	(
+	bzr init bzrrepo &&
 	cd bzrrepo &&
 	echo one > content &&
 	bzr add content &&
@@ -41,7 +46,8 @@ test_expect_success 'cloning' '
 '
 
 test_expect_success 'pulling' '
-	(cd bzrrepo &&
+	(
+	cd bzrrepo &&
 	echo two > content &&
 	bzr commit -m two
 	) &&
@@ -52,7 +58,8 @@ test_expect_success 'pulling' '
 '
 
 test_expect_success 'pushing' '
-	(cd gitrepo &&
+	(
+	cd gitrepo &&
 	echo three > content &&
 	git commit -a -m three &&
 	git push
@@ -64,15 +71,18 @@ test_expect_success 'pushing' '
 '
 
 test_expect_success 'roundtrip' '
-	(cd gitrepo &&
+	(
+	cd gitrepo &&
 	git pull &&
-	git log --format="%s" -1 origin/master > actual) &&
+	git log --format="%s" -1 origin/master > actual
+	) &&
 	echo three > expected &&
 	test_cmp expected actual &&
 
 	(cd gitrepo && git push && git pull) &&
 
-	(cd bzrrepo &&
+	(
+	cd bzrrepo &&
 	echo four > content &&
 	bzr commit -m four
 	) &&
@@ -81,7 +91,8 @@ test_expect_success 'roundtrip' '
 
 	check gitrepo four master &&
 
-	(cd gitrepo &&
+	(
+	cd gitrepo &&
 	echo five > content &&
 	git commit -a -m five &&
 	git push && git pull
@@ -101,7 +112,8 @@ cat > expected <<EOF
 EOF
 
 test_expect_success 'special modes' '
-	(cd bzrrepo &&
+	(
+	cd bzrrepo &&
 	echo exec > executable
 	chmod +x executable &&
 	bzr add executable
@@ -111,16 +123,21 @@ test_expect_success 'special modes' '
 	bzr commit -m link &&
 	mkdir dir &&
 	bzr add dir &&
-	bzr commit -m dir) &&
+	bzr commit -m dir
+	) &&
 
-	(cd gitrepo &&
+	(
+	cd gitrepo &&
 	git pull
-	git ls-tree HEAD > ../actual) &&
+	git ls-tree HEAD > ../actual
+	) &&
 
 	test_cmp expected actual &&
 
-	(cd gitrepo &&
-	git cat-file -p HEAD:link > ../actual) &&
+	(
+	cd gitrepo &&
+	git cat-file -p HEAD:link > ../actual
+	) &&
 
 	printf content > expected &&
 	test_cmp expected actual
@@ -134,32 +151,40 @@ cat > expected <<EOF
 EOF
 
 test_expect_success 'moving directory' '
-	(cd bzrrepo &&
+	(
+	cd bzrrepo &&
 	mkdir movedir &&
 	echo one > movedir/one &&
 	echo two > movedir/two &&
 	bzr add movedir &&
 	bzr commit -m movedir &&
 	bzr mv movedir movedir-new &&
-	bzr commit -m movedir-new) &&
+	bzr commit -m movedir-new
+	) &&
 
-	(cd gitrepo &&
+	(
+	cd gitrepo &&
 	git pull &&
-	git ls-tree HEAD > ../actual) &&
+	git ls-tree HEAD > ../actual
+	) &&
 
 	test_cmp expected actual
 '
 
 test_expect_success 'different authors' '
-	(cd bzrrepo &&
+	(
+	cd bzrrepo &&
 	echo john >> content &&
 	bzr commit -m john \
 	  --author "Jane Rey <jrey@example.com>" \
-	  --author "John Doe <jdoe@example.com>") &&
+	  --author "John Doe <jdoe@example.com>"
+	) &&
 
-	(cd gitrepo &&
+	(
+	cd gitrepo &&
 	git pull &&
-	git show --format="%an <%ae>, %cn <%ce>" --quiet > ../actual) &&
+	git show --format="%an <%ae>, %cn <%ce>" --quiet > ../actual
+	) &&
 
 	echo "Jane Rey <jrey@example.com>, A U Thor <author@example.com>" > expected &&
 	test_cmp expected actual
@@ -171,6 +196,7 @@ test_expect_success 'fetch utf-8 filenames' '
 
 	LC_ALL=en_US.UTF-8
 	export LC_ALL
+
 	(
 	bzr init bzrrepo &&
 	cd bzrrepo &&
@@ -276,23 +302,23 @@ test_expect_success 'proper bzr repo' '
 
 	bzr init-repo bzrrepo &&
 
-	bzr init bzrrepo/trunk &&
 	(
+	bzr init bzrrepo/trunk &&
 	cd bzrrepo/trunk &&
 	echo one >> content &&
 	bzr add content &&
 	bzr commit -m one
 	) &&
 
-	bzr branch bzrrepo/trunk bzrrepo/branch &&
 	(
+	bzr branch bzrrepo/trunk bzrrepo/branch &&
 	cd bzrrepo/branch &&
 	echo two >> content &&
 	bzr commit -m one
 	) &&
 
-	git clone "bzr::$PWD/bzrrepo" gitrepo &&
 	(
+	git clone "bzr::$PWD/bzrrepo" gitrepo &&
 	cd gitrepo &&
 	git for-each-ref --format "%(refname:short)" refs/remotes/origin > ../actual
 	) &&
@@ -331,9 +357,11 @@ test_expect_success 'strip' '
 	bzr log --line | sed -e "s/^[0-9][0-9]*: //" > ../expected
 	) &&
 
-	(cd gitrepo &&
+	(
+	cd gitrepo &&
 	git fetch &&
-	git log --format="%an %ad %s" --date=short origin/master > ../actual) &&
+	git log --format="%an %ad %s" --date=short origin/master > ../actual
+	) &&
 
 	test_cmp expected actual
 '
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index dac3756..8d1154b 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -21,11 +21,15 @@ if ! "$PYTHON_PATH" -c 'import mercurial'; then
 fi
 
 check () {
-	(cd $1 &&
+	(
+	cd $1 &&
 	git log --format='%s' -1 &&
-	git symbolic-ref HEAD) > actual &&
-	(echo $2 &&
-	echo "refs/heads/$3") > expected &&
+	git symbolic-ref HEAD
+	) > actual &&
+	(
+	echo $2 &&
+	echo "refs/heads/$3"
+	) > expected &&
 	test_cmp expected actual
 }
 
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 05/48] remote-helpers: tests: general improvements

[Felipe Contreras]

So that we don't need a temporary directory.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-bzr.sh       | 21 ++++++++++----------
 contrib/remote-helpers/test-hg-bidi.sh   | 15 +++++----------
 contrib/remote-helpers/test-hg-hg-git.sh | 33 +++++++++++---------------------
 contrib/remote-helpers/test-hg.sh        |  7 +++++--
 4 files changed, 31 insertions(+), 45 deletions(-)

diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index f760110..1ae7376 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -190,9 +190,11 @@ test_expect_success 'different authors' '
 	test_cmp expected actual
 '
 
+# cleanup previous stuff
+rm -rf bzrrepo gitrepo
+
 test_expect_success 'fetch utf-8 filenames' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
+	test_when_finished "rm -rf bzrrepo gitrepo && LC_ALL=C" &&
 
 	LC_ALL=en_US.UTF-8
 	export LC_ALL
@@ -223,8 +225,9 @@ test_expect_success 'fetch utf-8 filenames' '
 '
 
 test_expect_success 'push utf-8 filenames' '
+	test_when_finished "rm -rf bzrrepo gitrepo && LC_ALL=C" &&
+
 	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp && LC_ALL=C" &&
 
 	LC_ALL=en_US.UTF-8
 	export LC_ALL
@@ -255,8 +258,7 @@ test_expect_success 'push utf-8 filenames' '
 '
 
 test_expect_success 'pushing a merge' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf bzrrepo gitrepo" &&
 
 	(
 	bzr init bzrrepo &&
@@ -297,8 +299,7 @@ origin/trunk
 EOF
 
 test_expect_success 'proper bzr repo' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf bzrrepo gitrepo" &&
 
 	bzr init-repo bzrrepo &&
 
@@ -323,13 +324,11 @@ test_expect_success 'proper bzr repo' '
 	git for-each-ref --format "%(refname:short)" refs/remotes/origin > ../actual
 	) &&
 
-	test_cmp ../expected actual
+	test_cmp expected actual
 '
 
 test_expect_success 'strip' '
-	# Do not imitate this style; always chdir inside a subshell instead
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf bzrrepo gitrepo" &&
 
 	(
 	bzr init bzrrepo &&
diff --git a/contrib/remote-helpers/test-hg-bidi.sh b/contrib/remote-helpers/test-hg-bidi.sh
index f569697..490a395 100755
--- a/contrib/remote-helpers/test-hg-bidi.sh
+++ b/contrib/remote-helpers/test-hg-bidi.sh
@@ -77,8 +77,7 @@ setup () {
 setup
 
 test_expect_success 'encoding' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -115,8 +114,7 @@ test_expect_success 'encoding' '
 '
 
 test_expect_success 'file removal' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -148,8 +146,7 @@ test_expect_success 'file removal' '
 '
 
 test_expect_success 'git tags' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -177,8 +174,7 @@ test_expect_success 'git tags' '
 '
 
 test_expect_success 'hg branch' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -214,8 +210,7 @@ test_expect_success 'hg branch' '
 '
 
 test_expect_success 'hg tags' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
diff --git a/contrib/remote-helpers/test-hg-hg-git.sh b/contrib/remote-helpers/test-hg-hg-git.sh
index 7f579c8..6f0819c 100755
--- a/contrib/remote-helpers/test-hg-hg-git.sh
+++ b/contrib/remote-helpers/test-hg-hg-git.sh
@@ -114,8 +114,7 @@ setup () {
 setup
 
 test_expect_success 'executable bit' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -150,8 +149,7 @@ test_expect_success 'executable bit' '
 '
 
 test_expect_success 'symlink' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -181,8 +179,7 @@ test_expect_success 'symlink' '
 '
 
 test_expect_success 'merge conflict 1' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	hg init hgrepo1 &&
@@ -216,8 +213,7 @@ test_expect_success 'merge conflict 1' '
 '
 
 test_expect_success 'merge conflict 2' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	hg init hgrepo1 &&
@@ -251,8 +247,7 @@ test_expect_success 'merge conflict 2' '
 '
 
 test_expect_success 'converged merge' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	hg init hgrepo1 &&
@@ -287,8 +282,7 @@ test_expect_success 'converged merge' '
 '
 
 test_expect_success 'encoding' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -327,8 +321,7 @@ test_expect_success 'encoding' '
 '
 
 test_expect_success 'file removal' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -367,8 +360,7 @@ test_expect_success 'file removal' '
 '
 
 test_expect_success 'git tags' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	(
 	git init -q gitrepo &&
@@ -394,8 +386,7 @@ test_expect_success 'git tags' '
 '
 
 test_expect_success 'hg author' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	for x in hg git; do
 		(
@@ -461,8 +452,7 @@ test_expect_success 'hg author' '
 '
 
 test_expect_success 'hg branch' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	for x in hg git; do
 		(
@@ -498,8 +488,7 @@ test_expect_success 'hg branch' '
 '
 
 test_expect_success 'hg tags' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	for x in hg git; do
 		(
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 8d1154b..043f6c6 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -122,6 +122,9 @@ test_expect_success 'update bookmark' '
 	hg -R hgrepo bookmarks | egrep "devel[	 ]+3:"
 '
 
+# cleanup previous stuff
+rm -rf hgrepo
+
 author_test () {
 	echo $1 >> content &&
 	hg commit -u "$2" -m "add $1" &&
@@ -129,8 +132,7 @@ author_test () {
 }
 
 test_expect_success 'authors' '
-	mkdir -p tmp && cd tmp &&
-	test_when_finished "cd .. && rm -rf tmp" &&
+	test_when_finished "rm -rf hgrepo gitrepo" &&
 
 	(
 	hg init hgrepo &&
@@ -139,6 +141,7 @@ test_expect_success 'authors' '
 	touch content &&
 	hg add content &&
 
+	> ../expected &&
 	author_test alpha "" "H G Wells <wells@example.com>" &&
 	author_test beta "test" "test <unknown>" &&
 	author_test beta "test <test@example.com> (comment)" "test <test@example.com>" &&
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 06/48] remote-helpers: test: simplify remote URLs

[Felipe Contreras]

No need to specify $PWD any more.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-bzr.sh       | 12 ++++++------
 contrib/remote-helpers/test-hg-bidi.sh   |  6 +++---
 contrib/remote-helpers/test-hg-hg-git.sh |  6 +++---
 contrib/remote-helpers/test-hg.sh        | 14 +++++++-------
 4 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index 1ae7376..0276db0 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -41,7 +41,7 @@ test_expect_success 'cloning' '
 	bzr commit -m one
 	) &&
 
-	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	git clone "bzr::bzrrepo" gitrepo &&
 	check gitrepo one master
 '
 
@@ -216,7 +216,7 @@ test_expect_success 'fetch utf-8 filenames' '
 	) &&
 
 	(
-	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	git clone "bzr::bzrrepo" gitrepo &&
 	cd gitrepo &&
 	git -c core.quotepath=false ls-files > ../actual
 	) &&
@@ -242,7 +242,7 @@ test_expect_success 'push utf-8 filenames' '
 	) &&
 
 	(
-	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	git clone "bzr::bzrrepo" gitrepo &&
 	cd gitrepo &&
 
 	echo test >> "ærø" &&
@@ -268,7 +268,7 @@ test_expect_success 'pushing a merge' '
 	bzr commit -m one
 	) &&
 
-	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	git clone "bzr::bzrrepo" gitrepo &&
 
 	(
 	cd bzrrepo &&
@@ -319,7 +319,7 @@ test_expect_success 'proper bzr repo' '
 	) &&
 
 	(
-	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	git clone "bzr::bzrrepo" gitrepo &&
 	cd gitrepo &&
 	git for-each-ref --format "%(refname:short)" refs/remotes/origin > ../actual
 	) &&
@@ -342,7 +342,7 @@ test_expect_success 'strip' '
 	bzr commit -m two
 	) &&
 
-	git clone "bzr::$PWD/bzrrepo" gitrepo &&
+	git clone "bzr::bzrrepo" gitrepo &&
 
 	(
 	cd bzrrepo &&
diff --git a/contrib/remote-helpers/test-hg-bidi.sh b/contrib/remote-helpers/test-hg-bidi.sh
index 490a395..4bfbda2 100755
--- a/contrib/remote-helpers/test-hg-bidi.sh
+++ b/contrib/remote-helpers/test-hg-bidi.sh
@@ -22,7 +22,7 @@ fi
 
 # clone to a git repo
 git_clone () {
-	git clone -q "hg::$PWD/$1" $2
+	git clone -q "hg::$1" $2
 }
 
 # clone to an hg repo
@@ -31,7 +31,7 @@ hg_clone () {
 	hg init $2 &&
 	hg -R $2 bookmark -i master &&
 	cd $1 &&
-	git push -q "hg::$PWD/../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
+	git push -q "hg::../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
 	) &&
 
 	(cd $2 && hg -q update)
@@ -43,7 +43,7 @@ hg_push () {
 	cd $2
 	old=$(git symbolic-ref --short HEAD)
 	git checkout -q -b tmp &&
-	git fetch -q "hg::$PWD/../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
+	git fetch -q "hg::../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
 	git checkout -q $old &&
 	git branch -q -D tmp 2> /dev/null || true
 	)
diff --git a/contrib/remote-helpers/test-hg-hg-git.sh b/contrib/remote-helpers/test-hg-hg-git.sh
index 6f0819c..274a8d5 100755
--- a/contrib/remote-helpers/test-hg-hg-git.sh
+++ b/contrib/remote-helpers/test-hg-hg-git.sh
@@ -27,7 +27,7 @@ fi
 
 # clone to a git repo with git
 git_clone_git () {
-	git clone -q "hg::$PWD/$1" $2
+	git clone -q "hg::$1" $2
 }
 
 # clone to an hg repo with git
@@ -36,7 +36,7 @@ hg_clone_git () {
 	hg init $2 &&
 	hg -R $2 bookmark -i master &&
 	cd $1 &&
-	git push -q "hg::$PWD/../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
+	git push -q "hg::../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
 	) &&
 
 	(cd $2 && hg -q update)
@@ -63,7 +63,7 @@ hg_push_git () {
 	cd $2
 	old=$(git symbolic-ref --short HEAD)
 	git checkout -q -b tmp &&
-	git fetch -q "hg::$PWD/../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
+	git fetch -q "hg::../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
 	git checkout -q $old &&
 	git branch -q -D tmp 2> /dev/null || true
 	)
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 043f6c6..4bd7584 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -53,7 +53,7 @@ test_expect_success 'cloning' '
 	hg commit -m zero
 	) &&
 
-	git clone "hg::$PWD/hgrepo" gitrepo &&
+	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo zero master
 '
 
@@ -67,12 +67,12 @@ test_expect_success 'cloning with branches' '
 	hg commit -m next
 	) &&
 
-	git clone "hg::$PWD/hgrepo" gitrepo &&
+	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo next next &&
 
 	(cd hgrepo && hg checkout default) &&
 
-	git clone "hg::$PWD/hgrepo" gitrepo2 &&
+	git clone "hg::hgrepo" gitrepo2 &&
 	check gitrepo2 zero master
 '
 
@@ -86,7 +86,7 @@ test_expect_success 'cloning with bookmarks' '
 	hg commit -m feature-a
 	) &&
 
-	git clone "hg::$PWD/hgrepo" gitrepo &&
+	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo feature-a feature-a
 '
 
@@ -98,7 +98,7 @@ test_expect_success 'cloning with detached head' '
 	hg update -r 0
 	) &&
 
-	git clone "hg::$PWD/hgrepo" gitrepo &&
+	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo zero master
 '
 
@@ -111,7 +111,7 @@ test_expect_success 'update bookmark' '
 	) &&
 
 	(
-	git clone "hg::$PWD/hgrepo" gitrepo &&
+	git clone "hg::hgrepo" gitrepo &&
 	cd gitrepo &&
 	git checkout --quiet devel &&
 	echo devel > content &&
@@ -155,7 +155,7 @@ test_expect_success 'authors' '
 	author_test kappa "test@example.com" "Unknown <test@example.com>"
 	) &&
 
-	git clone "hg::$PWD/hgrepo" gitrepo &&
+	git clone "hg::hgrepo" gitrepo &&
 	git --git-dir=gitrepo/.git log --reverse --format="%an <%ae>" > actual &&
 
 	test_cmp expected actual
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 07/48] remote-hg: test: simplify previous branch checkout

[Felipe Contreras]

@{-1} does the same thing.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg-bidi.sh   | 3 +--
 contrib/remote-helpers/test-hg-hg-git.sh | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/contrib/remote-helpers/test-hg-bidi.sh b/contrib/remote-helpers/test-hg-bidi.sh
index 4bfbda2..88a33e6 100755
--- a/contrib/remote-helpers/test-hg-bidi.sh
+++ b/contrib/remote-helpers/test-hg-bidi.sh
@@ -41,10 +41,9 @@ hg_clone () {
 hg_push () {
 	(
 	cd $2
-	old=$(git symbolic-ref --short HEAD)
 	git checkout -q -b tmp &&
 	git fetch -q "hg::../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
-	git checkout -q $old &&
+	git checkout -q @{-1} &&
 	git branch -q -D tmp 2> /dev/null || true
 	)
 }
diff --git a/contrib/remote-helpers/test-hg-hg-git.sh b/contrib/remote-helpers/test-hg-hg-git.sh
index 274a8d5..e503293 100755
--- a/contrib/remote-helpers/test-hg-hg-git.sh
+++ b/contrib/remote-helpers/test-hg-hg-git.sh
@@ -61,10 +61,9 @@ hg_clone_hg () {
 hg_push_git () {
 	(
 	cd $2
-	old=$(git symbolic-ref --short HEAD)
 	git checkout -q -b tmp &&
 	git fetch -q "hg::../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
-	git checkout -q $old &&
+	git checkout -q @{-1} &&
 	git branch -q -D tmp 2> /dev/null || true
 	)
 }
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 08/48] remote-hg: load all extensions

[Felipe Contreras]

The user might have then configured differently, plus, all of them will
be loaded anyway later on.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 426c1fc..21f8acd 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -339,11 +339,7 @@ def get_repo(url, alias):
     if get_config_bool('remote-hg.insecure'):
         myui.setconfig('web', 'cacerts', '')
 
-    try:
-        mod = extensions.load(myui, 'hgext.schemes', None)
-        mod.extsetup(myui)
-    except ImportError:
-        pass
+    extensions.loadall(myui)
 
     if hg.islocal(url):
         repo = hg.repository(myui, url)
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 09/48] remote-hg: use a shared repository store

[Felipe Contreras]

This way we don't have to have duplicated Mercurial objects.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 35 ++++++++++++++++++++---------------
 1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 21f8acd..7507976 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -343,21 +343,29 @@ def get_repo(url, alias):
 
     if hg.islocal(url):
         repo = hg.repository(myui, url)
+        if not os.path.exists(dirname):
+            os.makedirs(dirname)
     else:
-        local_path = os.path.join(dirname, 'clone')
-        if not os.path.exists(local_path):
+        shared_path = os.path.join(gitdir, 'hg')
+        if not os.path.exists(shared_path):
             try:
-                peer, dstpeer = hg.clone(myui, {}, url, local_path, update=True, pull=True)
+                hg.clone(myui, {}, url, shared_path, update=False, pull=True)
             except:
                 die('Repository error')
-            repo = dstpeer.local()
-        else:
-            repo = hg.repository(myui, local_path)
-            try:
-                peer = hg.peer(myui, {}, url)
-            except:
-                die('Repository error')
-            repo.pull(peer, heads=None, force=True)
+
+        if not os.path.exists(dirname):
+            os.makedirs(dirname)
+
+        local_path = os.path.join(dirname, 'clone')
+        if not os.path.exists(local_path):
+            hg.share(myui, shared_path, local_path, update=False)
+
+        repo = hg.repository(myui, local_path)
+        try:
+            peer = hg.peer(myui, {}, url)
+        except:
+            die('Repository error')
+        repo.pull(peer, heads=None, force=True)
 
     return repo
 
@@ -898,7 +906,7 @@ def fix_path(alias, repo, orig_url):
     subprocess.call(cmd)
 
 def main(args):
-    global prefix, dirname, branches, bmarks
+    global prefix, gitdir, dirname, branches, bmarks
     global marks, blob_marks, parsed_refs
     global peer, mode, bad_mail, bad_name
     global track_branches, force_push, is_tmp
@@ -944,9 +952,6 @@ def main(args):
     if not is_tmp:
         fix_path(alias, peer or repo, url)
 
-    if not os.path.exists(dirname):
-        os.makedirs(dirname)
-
     marks_path = os.path.join(dirname, 'marks-hg')
     marks = Marks(marks_path)
 
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 10/48] remote-hg: shuffle some code

[Felipe Contreras]

In preparation to shift to SHA-1's.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 7507976..62520ee 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -75,6 +75,9 @@ def hgmode(mode):
 def hghex(node):
     return hg.node.hex(node)
 
+def hgbin(node):
+    return hg.node.bin(node)
+
 def hgref(ref):
     return ref.replace('___', ' ')
 
@@ -261,7 +264,7 @@ def get_filechanges(repo, ctx, parent):
     removed = set()
 
     # load earliest manifest first for caching reasons
-    prev = repo[parent].manifest().copy()
+    prev = parent.manifest().copy()
     cur = ctx.manifest()
 
     for fn in cur:
@@ -401,7 +404,7 @@ def export_ref(repo, name, kind, head):
         else:
             committer = author
 
-        parents = [p for p in repo.changelog.parentrevs(rev) if p >= 0]
+        parents = [repo[p] for p in repo.changelog.parentrevs(rev) if p >= 0]
 
         if len(parents) == 0:
             modified = c.manifest().keys()
@@ -450,9 +453,9 @@ def export_ref(repo, name, kind, head):
         print desc
 
         if len(parents) > 0:
-            print "from :%s" % (rev_to_mark(parents[0]))
+            print "from :%s" % (rev_to_mark(parents[0].rev()))
             if len(parents) > 1:
-                print "merge :%s" % (rev_to_mark(parents[1]))
+                print "merge :%s" % (rev_to_mark(parents[1].rev()))
 
         for f in modified_final:
             print "M %s :%u %s" % f
@@ -466,10 +469,10 @@ def export_ref(repo, name, kind, head):
 
     # make sure the ref is updated
     print "reset %s/%s" % (prefix, ename)
-    print "from :%u" % rev_to_mark(rev)
+    print "from :%u" % rev_to_mark(head.rev())
     print
 
-    marks.set_tip(ename, rev)
+    marks.set_tip(ename, head.rev())
 
 def export_tag(repo, tag):
     export_ref(repo, tag, 'tags', repo[hgref(tag)])
@@ -739,7 +742,7 @@ def parse_commit(parser):
     tmp = encoding.encoding
     encoding.encoding = 'utf-8'
 
-    node = repo.commitctx(ctx)
+    node = hghex(repo.commitctx(ctx))
 
     encoding.encoding = tmp
 
@@ -763,7 +766,7 @@ def parse_reset(parser):
     parser.next()
 
     node = parser.repo.changelog.node(mark_to_rev(from_mark))
-    parsed_refs[ref] = node
+    parsed_refs[ref] = hghex(node)
 
 def parse_tag(parser):
     name = parser[1]
@@ -788,7 +791,7 @@ def write_tag(repo, tag, node, msg, author):
             data = fctx.data()
         except error.ManifestLookupError:
             data = ""
-        content = data + "%s %s\n" % (hghex(node), tag)
+        content = data + "%s %s\n" % (node, tag)
         return context.memfilectx(f, content, False, False, None)
 
     p1 = tip.hex()
@@ -832,9 +835,10 @@ def do_export(parser):
             die('unhandled export command: %s' % line)
 
     for ref, node in parsed_refs.iteritems():
+        bnode = hgbin(node)
         if ref.startswith('refs/heads/branches'):
             branch = ref[len('refs/heads/branches/'):]
-            if branch in branches and node in branches[branch]:
+            if branch in branches and bnode in branches[branch]:
                 # up to date
                 continue
             print "ok %s" % ref
@@ -848,11 +852,11 @@ def do_export(parser):
             author, msg = parsed_tags.get(tag, (None, None))
             if mode == 'git':
                 if not msg:
-                    msg = 'Added tag %s for changeset %s' % (tag, hghex(node[:6]));
+                    msg = 'Added tag %s for changeset %s' % (tag, node[:12]);
                 write_tag(parser.repo, tag, node, msg, author)
             else:
                 fp = parser.repo.opener('localtags', 'a')
-                fp.write('%s %s\n' % (hghex(node), tag))
+                fp.write('%s %s\n' % (node, tag))
                 fp.close()
             print "ok %s" % ref
         else:
@@ -866,7 +870,7 @@ def do_export(parser):
     # handle bookmarks
     for bmark, node in p_bmarks:
         ref = 'refs/heads/' + bmark
-        new = hghex(node)
+        new = node
 
         if bmark in bmarks:
             old = bmarks[bmark].hex()
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 11/48] remote-hg: improve node traversing

[Felipe Contreras]

We won't be able to count the unmarked commits, but we are not going to
be able to do that anyway when we switch to SHA-1 ids.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 62520ee..07ea104 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -389,12 +389,16 @@ def export_ref(repo, name, kind, head):
     revs = xrange(tip, head.rev() + 1)
     count = 0
 
-    revs = [rev for rev in revs if not marks.is_marked(rev)]
-
     for rev in revs:
 
         c = repo[rev]
-        (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(c.node())
+        node = c.node()
+
+        if marks.is_marked(c.hex()):
+            count += 1
+            continue
+
+        (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(node)
         rev_branch = extra['branch']
 
         author = "%s %d %s" % (fixup_user(user), time, gittz(tz))
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 12/48] remote-hg: add version checks to the marks

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 07ea104..e2bef7f 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -55,6 +55,8 @@ EMAIL_RE = re.compile('^([^<>]+[^ \\\t<>])?\\b(?:[ \\t<>]*?)\\b([^ \\t<>]+@[^ \\
 AUTHOR_HG_RE = re.compile('^(.*?) ?<(.*?)(?:>(.+)?)?$')
 RAW_AUTHOR_RE = re.compile('^(\w+) (?:(.+)? )?<(.*)> (\d+) ([+-]\d+)')
 
+VERSION = 1
+
 def die(msg, *args):
     sys.stderr.write('ERROR: %s\n' % (msg % args))
     sys.exit(1)
@@ -103,12 +105,19 @@ class Marks:
 
     def __init__(self, path):
         self.path = path
+        self.clear()
+        self.load()
+
+        if self.version < VERSION:
+            self.clear()
+            self.version = VERSION
+
+    def clear(self):
         self.tips = {}
         self.marks = {}
         self.rev_marks = {}
         self.last_mark = 0
-
-        self.load()
+        self.version = 0
 
     def load(self):
         if not os.path.exists(self.path):
@@ -119,12 +128,13 @@ class Marks:
         self.tips = tmp['tips']
         self.marks = tmp['marks']
         self.last_mark = tmp['last-mark']
+        self.version = tmp.get('version', 1)
 
         for rev, mark in self.marks.iteritems():
             self.rev_marks[mark] = int(rev)
 
     def dict(self):
-        return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark }
+        return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark, 'version' : self.version }
 
     def store(self):
         json.dump(self.dict(), open(self.path, 'w'))
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 13/48] remote-hg: switch from revisions to SHA-1 noteids

[Felipe Contreras]

Otherwise we won't know if revisions are replaced.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 42 +++++++++++++++++++-----------------
 1 file changed, 22 insertions(+), 20 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index e2bef7f..ae7699c 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -55,7 +55,7 @@ EMAIL_RE = re.compile('^([^<>]+[^ \\\t<>])?\\b(?:[ \\t<>]*?)\\b([^ \\t<>]+@[^ \\
 AUTHOR_HG_RE = re.compile('^(.*?) ?<(.*?)(?:>(.+)?)?$')
 RAW_AUTHOR_RE = re.compile('^(\w+) (?:(.+)? )?<(.*)> (\d+) ([+-]\d+)')
 
-VERSION = 1
+VERSION = 2
 
 def die(msg, *args):
     sys.stderr.write('ERROR: %s\n' % (msg % args))
@@ -131,7 +131,7 @@ class Marks:
         self.version = tmp.get('version', 1)
 
         for rev, mark in self.marks.iteritems():
-            self.rev_marks[mark] = int(rev)
+            self.rev_marks[mark] = rev
 
     def dict(self):
         return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark, 'version' : self.version }
@@ -143,7 +143,7 @@ class Marks:
         return str(self.dict())
 
     def from_rev(self, rev):
-        return self.marks[str(rev)]
+        return self.marks[rev]
 
     def to_rev(self, mark):
         return self.rev_marks[mark]
@@ -154,19 +154,19 @@ class Marks:
 
     def get_mark(self, rev):
         self.last_mark += 1
-        self.marks[str(rev)] = self.last_mark
+        self.marks[rev] = self.last_mark
         return self.last_mark
 
     def new_mark(self, rev, mark):
-        self.marks[str(rev)] = mark
+        self.marks[rev] = mark
         self.rev_marks[mark] = rev
         self.last_mark = mark
 
     def is_marked(self, rev):
-        return str(rev) in self.marks
+        return rev in self.marks
 
     def get_tip(self, branch):
-        return self.tips.get(branch, 0)
+        return self.tips.get(branch, None)
 
     def set_tip(self, branch, tip):
         self.tips[branch] = tip
@@ -384,7 +384,7 @@ def get_repo(url, alias):
 
 def rev_to_mark(rev):
     global marks
-    return marks.from_rev(rev)
+    return marks.from_rev(rev.hex())
 
 def mark_to_rev(mark):
     global marks
@@ -395,6 +395,10 @@ def export_ref(repo, name, kind, head):
 
     ename = '%s/%s' % (kind, name)
     tip = marks.get_tip(ename)
+    if tip and tip in repo:
+        tip = repo[tip].rev()
+    else:
+        tip = 0
 
     revs = xrange(tip, head.rev() + 1)
     count = 0
@@ -460,16 +464,16 @@ def export_ref(repo, name, kind, head):
         modified_final = export_files(c.filectx(f) for f in modified)
 
         print "commit %s/%s" % (prefix, ename)
-        print "mark :%d" % (marks.get_mark(rev))
+        print "mark :%d" % (marks.get_mark(c.hex()))
         print "author %s" % (author)
         print "committer %s" % (committer)
         print "data %d" % (len(desc))
         print desc
 
         if len(parents) > 0:
-            print "from :%s" % (rev_to_mark(parents[0].rev()))
+            print "from :%s" % (rev_to_mark(parents[0]))
             if len(parents) > 1:
-                print "merge :%s" % (rev_to_mark(parents[1].rev()))
+                print "merge :%s" % (rev_to_mark(parents[1]))
 
         for f in modified_final:
             print "M %s :%u %s" % f
@@ -483,10 +487,10 @@ def export_ref(repo, name, kind, head):
 
     # make sure the ref is updated
     print "reset %s/%s" % (prefix, ename)
-    print "from :%u" % rev_to_mark(head.rev())
+    print "from :%u" % rev_to_mark(head)
     print
 
-    marks.set_tip(ename, head.rev())
+    marks.set_tip(ename, head.hex())
 
 def export_tag(repo, tag):
     export_ref(repo, tag, 'tags', repo[hgref(tag)])
@@ -713,12 +717,12 @@ def parse_commit(parser):
         extra['committer'] = "%s %u %u" % committer
 
     if from_mark:
-        p1 = repo.changelog.node(mark_to_rev(from_mark))
+        p1 = mark_to_rev(from_mark)
     else:
         p1 = '\0' * 20
 
     if merge_mark:
-        p2 = repo.changelog.node(mark_to_rev(merge_mark))
+        p2 = mark_to_rev(merge_mark)
     else:
         p2 = '\0' * 20
 
@@ -760,10 +764,8 @@ def parse_commit(parser):
 
     encoding.encoding = tmp
 
-    rev = repo[node].rev()
-
     parsed_refs[ref] = node
-    marks.new_mark(rev, commit_mark)
+    marks.new_mark(node, commit_mark)
 
 def parse_reset(parser):
     global parsed_refs
@@ -779,8 +781,8 @@ def parse_reset(parser):
     from_mark = parser.get_mark()
     parser.next()
 
-    node = parser.repo.changelog.node(mark_to_rev(from_mark))
-    parsed_refs[ref] = hghex(node)
+    rev = mark_to_rev(from_mark)
+    parsed_refs[ref] = rev
 
 def parse_tag(parser):
     name = parser[1]
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 14/48] remote-hg: upgrade version 1 marks

[Felipe Contreras]

As suggested by Jed Brown; there's no need to re-import all the commits.

Cc: Jed Brown <jed@59a2.org>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index ae7699c..2e4f7ca 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -103,14 +103,20 @@ def get_config_bool(config, default=False):
 
 class Marks:
 
-    def __init__(self, path):
+    def __init__(self, path, repo):
         self.path = path
+        self.repo = repo
         self.clear()
         self.load()
 
         if self.version < VERSION:
-            self.clear()
-            self.version = VERSION
+            if self.version == 1:
+                self.upgrade_one()
+
+            # upgraded?
+            if self.version < VERSION:
+                self.clear()
+                self.version = VERSION
 
     def clear(self):
         self.tips = {}
@@ -133,6 +139,14 @@ class Marks:
         for rev, mark in self.marks.iteritems():
             self.rev_marks[mark] = rev
 
+    def upgrade_one(self):
+        def get_id(rev):
+            return hghex(self.repo.changelog.node(int(rev)))
+        self.tips = dict((name, get_id(rev)) for name, rev in self.tips.iteritems())
+        self.marks = dict((get_id(rev), mark) for rev, mark in self.marks.iteritems())
+        self.rev_marks = dict((mark, get_id(rev)) for mark, rev in self.rev_marks.iteritems())
+        self.version = 2
+
     def dict(self):
         return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark, 'version' : self.version }
 
@@ -973,7 +987,7 @@ def main(args):
         fix_path(alias, peer or repo, url)
 
     marks_path = os.path.join(dirname, 'marks-hg')
-    marks = Marks(marks_path)
+    marks = Marks(marks_path, repo)
 
     if sys.platform == 'win32':
         import msvcrt
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 15/48] remote-hg: ensure remote rebasing works

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg |  1 +
 contrib/remote-helpers/test-hg.sh    | 40 ++++++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 2e4f7ca..e0e68ef 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -621,6 +621,7 @@ def do_import(parser):
     if os.path.exists(path):
         print "feature import-marks=%s" % path
     print "feature export-marks=%s" % path
+    print "feature force"
     sys.stdout.flush()
 
     tmp = encoding.encoding
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 4bd7584..431a5d1 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -37,6 +37,8 @@ setup () {
 	(
 	echo "[ui]"
 	echo "username = H G Wells <wells@example.com>"
+	echo "[extensions]"
+	echo "mq ="
 	) >> "$HOME"/.hgrc
 }
 
@@ -161,4 +163,42 @@ test_expect_success 'authors' '
 	test_cmp expected actual
 '
 
+test_expect_success 'strip' '
+	test_when_finished "rm -rf hgrepo gitrepo" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+
+	echo one >> content &&
+	hg add content &&
+	hg commit -m one &&
+
+	echo two >> content &&
+	hg commit -m two
+	) &&
+
+	git clone "hg::hgrepo" gitrepo &&
+
+	(
+	cd hgrepo &&
+	hg strip 1 &&
+
+	echo three >> content &&
+	hg commit -m three &&
+
+	echo four >> content &&
+	hg commit -m four
+	) &&
+
+	(
+	cd gitrepo &&
+	git fetch &&
+	git log --format="%s" origin/master > ../actual
+	) &&
+
+	hg -R hgrepo log --template "{desc}\n" > expected &&
+	test_cmp actual expected
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 16/48] remote-hg: trivial cleanups

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index e0e68ef..ee60dbf 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -74,11 +74,11 @@ def hgmode(mode):
     m = { '100755': 'x', '120000': 'l' }
     return m.get(mode, '')
 
-def hghex(node):
-    return hg.node.hex(node)
+def hghex(n):
+    return node.hex(n)
 
-def hgbin(node):
-    return hg.node.bin(node)
+def hgbin(n):
+    return node.bin(n)
 
 def hgref(ref):
     return ref.replace('___', ' ')
@@ -734,12 +734,12 @@ def parse_commit(parser):
     if from_mark:
         p1 = mark_to_rev(from_mark)
     else:
-        p1 = '\0' * 20
+        p1 = '0' * 40
 
     if merge_mark:
         p2 = mark_to_rev(merge_mark)
     else:
-        p2 = '\0' * 20
+        p2 = '0' * 40
 
     #
     # If files changed from any of the parents, hg wants to know, but in git if
@@ -826,7 +826,7 @@ def write_tag(repo, tag, node, msg, author):
         return context.memfilectx(f, content, False, False, None)
 
     p1 = tip.hex()
-    p2 = '\0' * 20
+    p2 = '0' * 40
     if not author:
         author = (None, 0, 0)
     user, date, tz = author
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 17/48] remote-hg: improve progress calculation

[Felipe Contreras]

No need to manually keep track of the revision count.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index ee60dbf..48edf3b 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -415,7 +415,7 @@ def export_ref(repo, name, kind, head):
         tip = 0
 
     revs = xrange(tip, head.rev() + 1)
-    count = 0
+    total = len(revs)
 
     for rev in revs:
 
@@ -423,7 +423,6 @@ def export_ref(repo, name, kind, head):
         node = c.node()
 
         if marks.is_marked(c.hex()):
-            count += 1
             continue
 
         (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(node)
@@ -495,9 +494,9 @@ def export_ref(repo, name, kind, head):
             print "D %s" % (fix_file_path(f))
         print
 
-        count += 1
-        if (count % 100 == 0):
-            print "progress revision %d '%s' (%d/%d)" % (rev, name, count, len(revs))
+        progress = (rev - tip)
+        if (progress % 100 == 0):
+            print "progress revision %d '%s' (%d/%d)" % (rev, name, progress, total)
 
     # make sure the ref is updated
     print "reset %s/%s" % (prefix, ename)
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 18/48] remote-hg: always point HEAD to master

[Felipe Contreras]

Mercurial always checks out the 'default' branch, so there's no point in
complicating our lives trying to do something fancier, which causes
different behavior depending on whether the repository is local or
remote.

So let's always use 'default' (which we translate to 'master'), unless
we are in hg-git mode, which expects us to use the 'master' bookmark
instead.

Also, update the tests that used to check for different checkout
behaviors to simply check that the refs are there, remove unnecessary
ones, and fix the ones that expect something different.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg     | 30 ++++++++++++---------------
 contrib/remote-helpers/test-hg-bidi.sh   |  9 ++++----
 contrib/remote-helpers/test-hg-hg-git.sh |  4 +++-
 contrib/remote-helpers/test-hg.sh        | 35 ++++++--------------------------
 4 files changed, 26 insertions(+), 52 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 48edf3b..7c859b8 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -560,22 +560,16 @@ def get_branch_tip(repo, branch):
     return heads[0]
 
 def list_head(repo, cur):
-    global g_head, bmarks
+    global g_head, bmarks, fake_bmark
 
-    head = bookmarks.readcurrent(repo)
-    if head:
-        node = repo[head]
-    else:
-        # fake bookmark from current branch
-        head = cur
-        node = repo['.']
-        if not node:
-            node = repo['tip']
-        if not node:
-            return
-        if head == 'default':
-            head = 'master'
-        bmarks[head] = node
+    if 'default' not in repo:
+        # empty repo
+        return
+
+    node = repo['default']
+    head = 'master' if not 'master' in bmarks else 'default'
+    fake_bmark = head
+    bmarks[head] = node
 
     head = gitref(head)
     print "@refs/heads/%s HEAD" % head
@@ -910,8 +904,8 @@ def do_export(parser):
         if old == new:
             continue
 
-        if bmark == 'master' and 'master' not in parser.repo._bookmarks:
-            # fake bookmark
+        if bmark == fake_bmark or \
+                bmark == 'master' and 'master' not in parser.repo._bookmarks:
             print "ok %s" % ref
             continue
         elif bookmarks.pushbookmark(parser.repo, bmark, old, new):
@@ -946,6 +940,7 @@ def main(args):
     global track_branches, force_push, is_tmp
     global parsed_tags
     global filenodes
+    global fake_bmark
 
     alias = args[1]
     url = args[2]
@@ -979,6 +974,7 @@ def main(args):
     marks = None
     parsed_tags = {}
     filenodes = {}
+    fake_bmark = None
 
     repo = get_repo(url, alias)
     prefix = 'refs/hg/%s' % alias
diff --git a/contrib/remote-helpers/test-hg-bidi.sh b/contrib/remote-helpers/test-hg-bidi.sh
index 88a33e6..86a3f10 100755
--- a/contrib/remote-helpers/test-hg-bidi.sh
+++ b/contrib/remote-helpers/test-hg-bidi.sh
@@ -29,7 +29,6 @@ git_clone () {
 hg_clone () {
 	(
 	hg init $2 &&
-	hg -R $2 bookmark -i master &&
 	cd $1 &&
 	git push -q "hg::../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
 	) &&
@@ -49,8 +48,7 @@ hg_push () {
 }
 
 hg_log () {
-	hg -R $1 log --graph --debug >log &&
-	grep -v 'tag: *default/' log
+	hg -R $1 log --graph --debug
 }
 
 setup () {
@@ -66,6 +64,7 @@ setup () {
 	echo "graphlog ="
 	) >> "$HOME"/.hgrc &&
 	git config --global remote-hg.hg-git-compat true
+	git config --global remote-hg.track-branches true
 
 	HGEDITOR=/usr/bin/true
 	GIT_AUTHOR_DATE="2007-01-01 00:00:00 +0230"
@@ -189,7 +188,7 @@ test_expect_success 'hg branch' '
 	hg_clone gitrepo hgrepo &&
 
 	cd hgrepo &&
-	hg -q co master &&
+	hg -q co default &&
 	hg mv alpha beta &&
 	hg -q commit -m "rename alpha to beta" &&
 	hg branch gamma | grep -v "permanent and global" &&
@@ -225,7 +224,7 @@ test_expect_success 'hg tags' '
 	hg_clone gitrepo hgrepo &&
 
 	cd hgrepo &&
-	hg co master &&
+	hg co default &&
 	hg tag alpha
 	) &&
 
diff --git a/contrib/remote-helpers/test-hg-hg-git.sh b/contrib/remote-helpers/test-hg-hg-git.sh
index e503293..9cb95d8 100755
--- a/contrib/remote-helpers/test-hg-hg-git.sh
+++ b/contrib/remote-helpers/test-hg-hg-git.sh
@@ -27,7 +27,8 @@ fi
 
 # clone to a git repo with git
 git_clone_git () {
-	git clone -q "hg::$1" $2
+	git clone -q "hg::$1" $2 &&
+	(cd $2 && git checkout master && git branch -D default)
 }
 
 # clone to an hg repo with git
@@ -63,6 +64,7 @@ hg_push_git () {
 	cd $2
 	git checkout -q -b tmp &&
 	git fetch -q "hg::../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
+	git branch -D default &&
 	git checkout -q @{-1} &&
 	git branch -q -D tmp 2> /dev/null || true
 	)
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 431a5d1..78af45d 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -21,15 +21,8 @@ if ! "$PYTHON_PATH" -c 'import mercurial'; then
 fi
 
 check () {
-	(
-	cd $1 &&
-	git log --format='%s' -1 &&
-	git symbolic-ref HEAD
-	) > actual &&
-	(
-	echo $2 &&
-	echo "refs/heads/$3"
-	) > expected &&
+	echo $3 > expected &&
+	git --git-dir=$1/.git log --format='%s' -1 $2 > actual
 	test_cmp expected actual
 }
 
@@ -56,7 +49,7 @@ test_expect_success 'cloning' '
 	) &&
 
 	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo zero master
+	check gitrepo HEAD zero
 '
 
 test_expect_success 'cloning with branches' '
@@ -70,12 +63,7 @@ test_expect_success 'cloning with branches' '
 	) &&
 
 	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo next next &&
-
-	(cd hgrepo && hg checkout default) &&
-
-	git clone "hg::hgrepo" gitrepo2 &&
-	check gitrepo2 zero master
+	check gitrepo origin/branches/next next
 '
 
 test_expect_success 'cloning with bookmarks' '
@@ -83,25 +71,14 @@ test_expect_success 'cloning with bookmarks' '
 
 	(
 	cd hgrepo &&
+	hg checkout default &&
 	hg bookmark feature-a &&
 	echo feature-a > content &&
 	hg commit -m feature-a
 	) &&
 
 	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo feature-a feature-a
-'
-
-test_expect_success 'cloning with detached head' '
-	test_when_finished "rm -rf gitrepo*" &&
-
-	(
-	cd hgrepo &&
-	hg update -r 0
-	) &&
-
-	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo zero master
+	check gitrepo origin/feature-a feature-a
 '
 
 test_expect_success 'update bookmark' '
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 19/48] remote-hg: add tests for 'master' bookmark

[Felipe Contreras]

We want to make sure everything works correctly, even if there's a
'master' bookmark in Mercurial.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 68 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 67 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 78af45d..943371a 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -26,13 +26,23 @@ check () {
 	test_cmp expected actual
 }
 
+check_branch () {
+	echo $3 > expected &&
+	hg -R $1 log -r $2 --template '{desc}\n' > actual &&
+	test_cmp expected actual
+}
+
 setup () {
 	(
 	echo "[ui]"
 	echo "username = H G Wells <wells@example.com>"
 	echo "[extensions]"
 	echo "mq ="
-	) >> "$HOME"/.hgrc
+	) >> "$HOME"/.hgrc &&
+
+	GIT_AUTHOR_DATE="2007-01-01 00:00:00 +0230" &&
+	GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" &&
+	export GIT_COMMITTER_DATE GIT_AUTHOR_DATE
 }
 
 setup
@@ -178,4 +188,60 @@ test_expect_success 'strip' '
 	test_cmp actual expected
 '
 
+test_expect_success 'remote push with master bookmark' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+	echo zero > content &&
+	hg add content &&
+	hg commit -m zero &&
+	hg bookmark master &&
+	echo one > content &&
+	hg commit -m one
+	) &&
+
+	(
+	git clone "hg::hgrepo" gitrepo &&
+	cd gitrepo &&
+	echo two > content &&
+	git commit -a -m two &&
+	git push
+	) &&
+
+	check_branch hgrepo default two
+'
+
+cat > expected <<EOF
+changeset:   0:6e2126489d3d
+tag:         tip
+user:        A U Thor <author@example.com>
+date:        Mon Jan 01 00:00:00 2007 +0230
+summary:     one
+
+EOF
+
+test_expect_success 'remote push from master branch' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	hg init hgrepo &&
+
+	(
+	git init gitrepo &&
+	cd gitrepo &&
+	git remote add origin "hg::../hgrepo" &&
+	echo one > content &&
+	git add content &&
+	git commit -a -m one &&
+	git push origin master
+	) &&
+
+	hg -R hgrepo log > actual &&
+	cat actual &&
+	test_cmp expected actual &&
+
+	check_branch hgrepo default one
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 20/48] remote-bzr: simplify test checks

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-bzr.sh | 17 +++++------------
 1 file changed, 5 insertions(+), 12 deletions(-)

diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh
index 0276db0..435b280 100755
--- a/contrib/remote-helpers/test-bzr.sh
+++ b/contrib/remote-helpers/test-bzr.sh
@@ -18,15 +18,8 @@ if ! "$PYTHON_PATH" -c 'import bzrlib'; then
 fi
 
 check () {
-	(
-	cd $1 &&
-	git log --format='%s' -1 &&
-	git symbolic-ref HEAD
-	) > actual &&
-	(
-	echo $2 &&
-	echo "refs/heads/$3"
-	) > expected &&
+	echo $3 > expected &&
+	git --git-dir=$1/.git log --format='%s' -1 $2 > actual
 	test_cmp expected actual
 }
 
@@ -42,7 +35,7 @@ test_expect_success 'cloning' '
 	) &&
 
 	git clone "bzr::bzrrepo" gitrepo &&
-	check gitrepo one master
+	check gitrepo HEAD one
 '
 
 test_expect_success 'pulling' '
@@ -54,7 +47,7 @@ test_expect_success 'pulling' '
 
 	(cd gitrepo && git pull) &&
 
-	check gitrepo two master
+	check gitrepo HEAD two
 '
 
 test_expect_success 'pushing' '
@@ -89,7 +82,7 @@ test_expect_success 'roundtrip' '
 
 	(cd gitrepo && git pull && git push) &&
 
-	check gitrepo four master &&
+	check gitrepo HEAD four &&
 
 	(
 	cd gitrepo &&
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 21/48] remote-hg: add check_bookmark() test helper

[Felipe Contreras]

And check in a more proper way.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 943371a..2f2050c 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -32,6 +32,12 @@ check_branch () {
 	test_cmp expected actual
 }
 
+check_bookmark () {
+	echo $3 > expected &&
+	hg -R $1 log -r "bookmark('$2')" --template '{desc}\n' > actual &&
+	test_cmp expected actual
+}
+
 setup () {
 	(
 	echo "[ui]"
@@ -108,7 +114,7 @@ test_expect_success 'update bookmark' '
 	git push --quiet
 	) &&
 
-	hg -R hgrepo bookmarks | egrep "devel[	 ]+3:"
+	check_bookmark hgrepo devel devel
 '
 
 # cleanup previous stuff
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 22/48] remote-hg: update bookmarks when using a remote

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 7c859b8..76438a6 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -356,6 +356,21 @@ def fixup_user(user):
 
     return '%s <%s>' % (name, mail)
 
+def updatebookmarks(repo, peer):
+    remotemarks = peer.listkeys('bookmarks')
+    localmarks = repo._bookmarks
+
+    if not remotemarks:
+        return
+
+    for k, v in remotemarks.iteritems():
+        localmarks[k] = hgbin(v)
+
+    if hasattr(localmarks, 'write'):
+        localmarks.write()
+    else:
+        bookmarks.write(repo)
+
 def get_repo(url, alias):
     global dirname, peer
 
@@ -394,6 +409,8 @@ def get_repo(url, alias):
             die('Repository error')
         repo.pull(peer, heads=None, force=True)
 
+        updatebookmarks(repo, peer)
+
     return repo
 
 def rev_to_mark(rev):
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 23/48] remote-hg: add remote tests

[Felipe Contreras]

The logic when working with a local repository is totally different from
the one where we work with a remote repository; we need to pull and push
from it.

All this logic is currently not tested at all, so let's introduce a
variable to force the remote behavior.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg |  2 +-
 contrib/remote-helpers/test-hg.sh    | 38 ++++++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 76438a6..61e6c87 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -383,7 +383,7 @@ def get_repo(url, alias):
 
     extensions.loadall(myui)
 
-    if hg.islocal(url):
+    if hg.islocal(url) and not os.environ.get('GIT_REMOTE_HG_TEST_REMOTE'):
         repo = hg.repository(myui, url)
         if not os.path.exists(dirname):
             os.makedirs(dirname)
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 2f2050c..d927495 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -250,4 +250,42 @@ test_expect_success 'remote push from master branch' '
 	check_branch hgrepo default one
 '
 
+GIT_REMOTE_HG_TEST_REMOTE=1
+export GIT_REMOTE_HG_TEST_REMOTE
+
+test_expect_success 'remote cloning' '
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+	echo zero > content &&
+	hg add content &&
+	hg commit -m zero
+	) &&
+
+	git clone "hg::hgrepo" gitrepo &&
+	check gitrepo HEAD zero
+'
+
+test_expect_success 'remote update bookmark' '
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	cd hgrepo &&
+	hg bookmark devel
+	) &&
+
+	(
+	git clone "hg::hgrepo" gitrepo &&
+	cd gitrepo &&
+	git checkout --quiet devel &&
+	echo devel > content &&
+	git commit -a -m devel &&
+	git push --quiet
+	) &&
+
+	check_bookmark hgrepo devel devel
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 24/48] remote-hg: add test to push new bookmark

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index d927495..8b5e191 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -117,6 +117,21 @@ test_expect_success 'update bookmark' '
 	check_bookmark hgrepo devel devel
 '
 
+test_expect_success 'new bookmark' '
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	git clone "hg::hgrepo" gitrepo &&
+	cd gitrepo &&
+	git checkout --quiet -b feature-b &&
+	echo feature-b > content &&
+	git commit -a -m feature-b &&
+	git push --quiet origin feature-b
+	) &&
+
+	check_bookmark hgrepo feature-b feature-b
+'
+
 # cleanup previous stuff
 rm -rf hgrepo
 
@@ -288,4 +303,19 @@ test_expect_success 'remote update bookmark' '
 	check_bookmark hgrepo devel devel
 '
 
+test_expect_success 'remote new bookmark' '
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	git clone "hg::hgrepo" gitrepo &&
+	cd gitrepo &&
+	git checkout --quiet -b feature-b &&
+	echo feature-b > content &&
+	git commit -a -m feature-b &&
+	git push --quiet origin feature-b
+	) &&
+
+	check_bookmark hgrepo feature-b feature-b
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 25/48] remote-hg: add test for diverged push

[Felipe Contreras]

Neither mercurial nor git allows pushing to a remote when it's a
non-fast-forward push. We should be able to detect theses errors and
report them properly.

As opposed to throwing an exception stack-trace.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 8b5e191..252553e 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -318,4 +318,27 @@ test_expect_success 'remote new bookmark' '
 	check_bookmark hgrepo feature-b feature-b
 '
 
+test_expect_failure 'remote push diverged' '
+	test_when_finished "rm -rf gitrepo*" &&
+
+	git clone "hg::hgrepo" gitrepo &&
+
+	(
+	cd hgrepo &&
+	hg checkout default &&
+	echo bump > content &&
+	hg commit -m bump
+	) &&
+
+	(
+	cd gitrepo &&
+	echo diverge > content &&
+	git commit -a -m diverged &&
+	test_expect_code 1 git push 2> error &&
+	grep "^ ! \[rejected\] *master -> master (non-fast-forward)$" error
+	) &&
+
+	check_branch hgrepo default bump
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 26/48] remote-hg: add test for bookmark diverge

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 252553e..e059bbc 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -341,4 +341,33 @@ test_expect_failure 'remote push diverged' '
 	check_branch hgrepo default bump
 '
 
+test_expect_failure 'remote update bookmark diverge' '
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	cd hgrepo &&
+	hg checkout tip^ &&
+	hg bookmark diverge
+	) &&
+
+	git clone "hg::hgrepo" gitrepo &&
+
+	(
+	cd hgrepo &&
+	echo "bump bookmark" > content &&
+	hg commit -m "bump bookmark"
+	) &&
+
+	(
+	cd gitrepo &&
+	git checkout --quiet diverge &&
+	echo diverge > content &&
+	git commit -a -m diverge &&
+	test_expect_code 1 git push 2> error &&
+	grep "^ ! \[rejected\] *diverge -> diverge (non-fast-forward)$" error
+	) &&
+
+	check_bookmark hgrepo diverge "bump bookmark"
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 27/48] remote-hg: add test for new bookmark special

[Felipe Contreras]

>From the point of view of Mercurial, this creates a new branch head, and
requires a forced push.

Ideally, whoever, we would want it to work just like in git; new
branches can be pushed without problems.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index e059bbc..64f0849 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -370,4 +370,19 @@ test_expect_failure 'remote update bookmark diverge' '
 	check_bookmark hgrepo diverge "bump bookmark"
 '
 
+test_expect_failure 'remote new bookmark multiple branch head' '
+	test_when_finished "rm -rf gitrepo*" &&
+
+	(
+	git clone "hg::hgrepo" gitrepo &&
+	cd gitrepo &&
+	git checkout --quiet -b feature-c HEAD^ &&
+	echo feature-c > content &&
+	git commit -a -m feature-c &&
+	git push --quiet origin feature-c
+	) &&
+
+	check_bookmark hgrepo feature-c feature-c
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 28/48] remote-hg: add test for big push

[Felipe Contreras]

With lots branches and bookmarks, non-ff, updated and new.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 113 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 107 insertions(+), 6 deletions(-)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 64f0849..1855d1a 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -27,15 +27,25 @@ check () {
 }
 
 check_branch () {
-	echo $3 > expected &&
-	hg -R $1 log -r $2 --template '{desc}\n' > actual &&
-	test_cmp expected actual
+	if [ -n "$3" ]; then
+		echo $3 > expected &&
+		hg -R $1 log -r $2 --template '{desc}\n' > actual &&
+		test_cmp expected actual
+	else
+		hg -R $1 branches > out &&
+		! grep $2 out
+	fi
 }
 
 check_bookmark () {
-	echo $3 > expected &&
-	hg -R $1 log -r "bookmark('$2')" --template '{desc}\n' > actual &&
-	test_cmp expected actual
+	if [ -n "$3" ]; then
+		echo $3 > expected &&
+		hg -R $1 log -r "bookmark('$2')" --template '{desc}\n' > actual &&
+		test_cmp expected actual
+	else
+		hg -R $1 bookmarks > out &&
+		! grep $2 out
+	fi
 }
 
 setup () {
@@ -385,4 +395,95 @@ test_expect_failure 'remote new bookmark multiple branch head' '
 	check_bookmark hgrepo feature-c feature-c
 '
 
+# cleanup previous stuff
+rm -rf hgrepo
+
+test_expect_failure 'remote big push' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+	echo zero > content &&
+	hg add content &&
+	hg commit -m zero &&
+	hg bookmark bad_bmark1 &&
+	echo one > content &&
+	hg commit -m one &&
+	hg bookmark bad_bmark2 &&
+	hg bookmark good_bmark &&
+	hg bookmark -i good_bmark &&
+	hg -q branch good_branch &&
+	echo "good branch" > content &&
+	hg commit -m "good branch" &&
+	hg -q branch bad_branch &&
+	echo "bad branch" > content &&
+	hg commit -m "bad branch"
+	) &&
+
+	git clone "hg::hgrepo" gitrepo &&
+
+	(
+	cd gitrepo &&
+	echo two > content &&
+	git commit -q -a -m two &&
+
+	git checkout -q good_bmark &&
+	echo three > content &&
+	git commit -q -a -m three &&
+
+	git checkout -q bad_bmark1 &&
+	git reset --hard HEAD^ &&
+	echo four > content &&
+	git commit -q -a -m four &&
+
+	git checkout -q bad_bmark2 &&
+	git reset --hard HEAD^ &&
+	echo five > content &&
+	git commit -q -a -m five &&
+
+	git checkout -q -b new_bmark master &&
+	echo six > content &&
+	git commit -q -a -m six &&
+
+	git checkout -q branches/good_branch &&
+	echo seven > content &&
+	git commit -q -a -m seven &&
+	echo eight > content &&
+	git commit -q -a -m eight &&
+
+	git checkout -q branches/bad_branch &&
+	git reset --hard HEAD^ &&
+	echo nine > content &&
+	git commit -q -a -m nine &&
+
+	git checkout -q -b branches/new_branch master &&
+	echo ten > content &&
+	git commit -q -a -m ten &&
+
+	test_expect_code 1 git push origin master \
+		good_bmark bad_bmark1 bad_bmark2 new_bmark \
+		branches/good_branch branches/bad_branch \
+		branches/new_branch 2> error &&
+
+	grep "^   [a-f0-9]*\.\.[a-f0-9]* *master -> master$" error &&
+	grep "^   [a-f0-9]*\.\.[a-f0-9]* *good_bmark -> good_bmark$" error &&
+	grep "^ \* \[new branch\] *new_bmark -> new_bmark$" error &&
+	grep "^ ! \[rejected\] *bad_bmark2 -> bad_bmark2 (non-fast-forward)$" error &&
+	grep "^ ! \[rejected\] *bad_bmark1 -> bad_bmark1 (non-fast-forward)$" error &&
+	grep "^   [a-f0-9]*\.\.[a-f0-9]* *branches/good_branch -> branches/good_branch$" error &&
+	grep "^ ! \[rejected\] *branches/bad_branch -> branches/bad_branch (non-fast-forward)$" error &&
+	grep "^ \* \[new branch\] *branches/new_branch -> branches/new_branch$" error
+	) &&
+
+	check_branch hgrepo default one &&
+	check_branch hgrepo good_branch "good branch" &&
+	check_branch hgrepo bad_branch "bad branch" &&
+	check_branch hgrepo new_branch '' &&
+	check_bookmark hgrepo good_bmark one &&
+	check_bookmark hgrepo bad_bmark1 one &&
+	check_bookmark hgrepo bad_bmark2 one &&
+	check_bookmark hgrepo new_bmark ''
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 29/48] remote-hg: add test for failed double push

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 1855d1a..c4d98b5 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -486,4 +486,28 @@ test_expect_failure 'remote big push' '
 	check_bookmark hgrepo new_bmark ''
 '
 
+test_expect_failure 'remote double failed push' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+	echo zero > content &&
+	hg add content &&
+	hg commit -m zero &&
+	echo one > content &&
+	hg commit -m one
+	) &&
+
+	(
+	git clone "hg::hgrepo" gitrepo &&
+	cd gitrepo &&
+	git reset --hard HEAD^ &&
+	echo two > content &&
+	git commit -a -m two &&
+	test_expect_code 1 git push &&
+	test_expect_code 1 git push
+	)
+'
+
 test_done
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 30/48] remote-hg: reorganize bookmark handling

[Felipe Contreras]

We don't need to update both internal and remote bookmarks, so let's do
one or the other, and move the shared code earlier, so it's simpler.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 50 ++++++++++++++----------------------
 1 file changed, 19 insertions(+), 31 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 61e6c87..99de7c2 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -885,8 +885,17 @@ def do_export(parser):
             print "ok %s" % ref
         elif ref.startswith('refs/heads/'):
             bmark = ref[len('refs/heads/'):]
-            p_bmarks.append((bmark, node))
-            continue
+            new = node
+            old = bmarks[bmark].hex() if bmark in bmarks else ''
+
+            if old == new:
+                continue
+
+            print "ok %s" % ref
+            if bmark != fake_bmark and \
+                    not (bmark == 'master' and bmark not in parser.repo._bookmarks):
+                p_bmarks.append((ref, bmark, old, new))
+
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
             tag = hgref(tag)
@@ -906,39 +915,18 @@ def do_export(parser):
 
     if peer:
         parser.repo.push(peer, force=force_push, newbranch=True)
-        remote_bmarks = peer.listkeys('bookmarks')
-
-    # handle bookmarks
-    for bmark, node in p_bmarks:
-        ref = 'refs/heads/' + bmark
-        new = node
-
-        if bmark in bmarks:
-            old = bmarks[bmark].hex()
-        else:
-            old = ''
-
-        if old == new:
-            continue
 
-        if bmark == fake_bmark or \
-                bmark == 'master' and 'master' not in parser.repo._bookmarks:
-            print "ok %s" % ref
-            continue
-        elif bookmarks.pushbookmark(parser.repo, bmark, old, new):
-            # updated locally
-            pass
-        else:
-            print "error %s" % ref
-            continue
-
-        if peer:
+        # update remote bookmarks
+        remote_bmarks = peer.listkeys('bookmarks')
+        for ref, bmark, old, new in p_bmarks:
             old = remote_bmarks.get(bmark, '')
             if not peer.pushkey('bookmarks', bmark, old, new):
                 print "error %s" % ref
-                continue
-
-        print "ok %s" % ref
+    else:
+        # update local bookmarks
+        for ref, bmark, old, new in p_bmarks:
+            if not bookmarks.pushbookmark(parser.repo, bmark, old, new):
+                print "error %s" % ref
 
     print
 
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 31/48] remote-hg: force remote bookmark push selectively

[Felipe Contreras]

If we update the 'old' node, we might be updating the remote bookmark
even when our 'new' node is not related at all to what the remote has,
effectively forcing an update.

Let's do that only when forced push is configured.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 99de7c2..4d650b5 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -919,7 +919,8 @@ def do_export(parser):
         # update remote bookmarks
         remote_bmarks = peer.listkeys('bookmarks')
         for ref, bmark, old, new in p_bmarks:
-            old = remote_bmarks.get(bmark, '')
+            if force_push:
+                old = remote_bmarks.get(bmark, '')
             if not peer.pushkey('bookmarks', bmark, old, new):
                 print "error %s" % ref
     else:
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 32/48] remote-hg: only update necessary revisions

[Felipe Contreras]

We don't care about the rest, and in fact, we shouldn't try to push
everything, as there might be garbage from previous failed pushes.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 4d650b5..7added3 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -858,6 +858,7 @@ def do_export(parser):
     global parsed_refs, bmarks, peer
 
     p_bmarks = []
+    p_revs = set()
 
     parser.next()
 
@@ -882,6 +883,7 @@ def do_export(parser):
             if branch in branches and bnode in branches[branch]:
                 # up to date
                 continue
+            p_revs.add(bnode)
             print "ok %s" % ref
         elif ref.startswith('refs/heads/'):
             bmark = ref[len('refs/heads/'):]
@@ -896,6 +898,7 @@ def do_export(parser):
                     not (bmark == 'master' and bmark not in parser.repo._bookmarks):
                 p_bmarks.append((ref, bmark, old, new))
 
+            p_revs.add(bnode)
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
             tag = hgref(tag)
@@ -903,18 +906,20 @@ def do_export(parser):
             if mode == 'git':
                 if not msg:
                     msg = 'Added tag %s for changeset %s' % (tag, node[:12]);
-                write_tag(parser.repo, tag, node, msg, author)
+                tagnode = write_tag(parser.repo, tag, node, msg, author)
+                p_revs.add(tagnode)
             else:
                 fp = parser.repo.opener('localtags', 'a')
                 fp.write('%s %s\n' % (node, tag))
                 fp.close()
+            p_revs.add(bnode)
             print "ok %s" % ref
         else:
             # transport-helper/fast-export bugs
             continue
 
     if peer:
-        parser.repo.push(peer, force=force_push, newbranch=True)
+        parser.repo.push(peer, force=force_push, newbranch=True, revs=list(p_revs))
 
         # update remote bookmarks
         remote_bmarks = peer.listkeys('bookmarks')
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 33/48] remote-hg: implement custom push()

[Felipe Contreras]

The one from mercurial does a ton of things we are not interested in,
and we need some special modifications which are impossible otherwise.

Most of the code is borrowed from Mercurial, and cleaned up, but should
be functionally the same for our purposes, except that multiple heads
are not detected.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 44 ++++++++++++++++++++++++++++++++++--
 1 file changed, 42 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 7added3..c2c1cb8 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -12,7 +12,7 @@
 # For remote repositories a local clone is stored in
 # "$GIT_DIR/hg/origin/clone/.hg/".
 
-from mercurial import hg, ui, bookmarks, context, encoding, node, error, extensions
+from mercurial import hg, ui, bookmarks, context, encoding, node, error, extensions, discovery
 
 import re
 import sys
@@ -854,6 +854,46 @@ def write_tag(repo, tag, node, msg, author):
 
     return tagnode
 
+def push_unsafe(repo, remote, parsed_refs, p_revs):
+
+    force = force_push
+
+    fci = discovery.findcommonincoming
+    commoninc = fci(repo, remote, force=force)
+    common, _, remoteheads = commoninc
+
+    # TODO checkheads
+
+    cg = repo.getbundle('push', heads=list(p_revs), common=common)
+
+    unbundle = remote.capable('unbundle')
+    if unbundle:
+        if force:
+            remoteheads = ['force']
+        return remote.unbundle(cg, remoteheads, 'push')
+    else:
+        return remote.addchangegroup(cg, 'push', repo.url())
+
+def push(repo, remote, parsed_refs, p_revs):
+    if hasattr(remote, 'canpush') and not remote.canpush():
+        print "error cannot push"
+
+    if not p_revs:
+        # nothing to push
+        return
+
+    lock = None
+    unbundle = remote.capable('unbundle')
+    if not unbundle:
+        lock = remote.lock()
+    try:
+        ret = push_unsafe(repo, remote, parsed_refs, p_revs)
+    finally:
+        if lock is not None:
+            lock.release()
+
+    return ret
+
 def do_export(parser):
     global parsed_refs, bmarks, peer
 
@@ -919,7 +959,7 @@ def do_export(parser):
             continue
 
     if peer:
-        parser.repo.push(peer, force=force_push, newbranch=True, revs=list(p_revs))
+        push(parser.repo, peer, parsed_refs, p_revs)
 
         # update remote bookmarks
         remote_bmarks = peer.listkeys('bookmarks')
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 34/48] remote-hg: implement custom checkheads()

[Felipe Contreras]

The version from Mercurial is extremely inefficient and convoluted, this
version achieves basically the same, at least for our purposes.

No functional changes.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 52 +++++++++++++++++++++++++++++++++---
 1 file changed, 49 insertions(+), 3 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index c2c1cb8..7eb809b 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -12,7 +12,7 @@
 # For remote repositories a local clone is stored in
 # "$GIT_DIR/hg/origin/clone/.hg/".
 
-from mercurial import hg, ui, bookmarks, context, encoding, node, error, extensions, discovery
+from mercurial import hg, ui, bookmarks, context, encoding, node, error, extensions, discovery, util
 
 import re
 import sys
@@ -86,6 +86,11 @@ def hgref(ref):
 def gitref(ref):
     return ref.replace(' ', '___')
 
+def check_version(*check):
+    if not hg_version:
+        return True
+    return hg_version >= check
+
 def get_config(config):
     cmd = ['git', 'config', '--get', config]
     process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
@@ -854,6 +859,42 @@ def write_tag(repo, tag, node, msg, author):
 
     return tagnode
 
+def checkheads(repo, remote, p_revs):
+
+    remotemap = remote.branchmap()
+    if not remotemap:
+        # empty repo
+        return
+
+    new = {}
+
+    for node in p_revs:
+        ctx = repo[node]
+        branch = ctx.branch()
+        if not branch in remotemap:
+            # new branch
+            continue
+        new.setdefault(branch, []).append(ctx.rev())
+
+    for branch, heads in new.iteritems():
+        old = [repo.changelog.rev(x) for x in remotemap[branch]]
+        for rev in heads:
+            if check_version(2, 3):
+                ancestors = repo.changelog.ancestors([rev], stoprev=min(old))
+            else:
+                ancestors = repo.changelog.ancestors(rev)
+            found = False
+
+            for x in old:
+                if x in ancestors:
+                    found = True
+                    break
+
+            if found:
+                continue
+
+            raise Exception("non-fast-forward")
+
 def push_unsafe(repo, remote, parsed_refs, p_revs):
 
     force = force_push
@@ -862,7 +903,8 @@ def push_unsafe(repo, remote, parsed_refs, p_revs):
     commoninc = fci(repo, remote, force=force)
     common, _, remoteheads = commoninc
 
-    # TODO checkheads
+    if not force:
+        checkheads(repo, remote, p_revs)
 
     cg = repo.getbundle('push', heads=list(p_revs), common=common)
 
@@ -991,7 +1033,7 @@ def main(args):
     global track_branches, force_push, is_tmp
     global parsed_tags
     global filenodes
-    global fake_bmark
+    global fake_bmark, hg_version
 
     alias = args[1]
     url = args[2]
@@ -1026,6 +1068,10 @@ def main(args):
     parsed_tags = {}
     filenodes = {}
     fake_bmark = None
+    try:
+        hg_version = tuple(int(e) for e in util.version().split('.'))
+    except:
+        hg_version = None
 
     repo = get_repo(url, alias)
     prefix = 'refs/hg/%s' % alias
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 35/48] remote-hg: pass around revision refs

[Felipe Contreras]

So that when a diverge is detected, we know which ref to report an error
for.

Also, since we are not throwing an exception, return a proper error
code.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 34 +++++++++++++++++++++-------------
 contrib/remote-helpers/test-hg.sh    |  6 +++---
 2 files changed, 24 insertions(+), 16 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 7eb809b..b6f85b2 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -857,18 +857,19 @@ def write_tag(repo, tag, node, msg, author):
 
     encoding.encoding = tmp
 
-    return tagnode
+    return (tagnode, branch)
 
 def checkheads(repo, remote, p_revs):
 
     remotemap = remote.branchmap()
     if not remotemap:
         # empty repo
-        return
+        return True
 
     new = {}
+    ret = True
 
-    for node in p_revs:
+    for node, ref in p_revs.iteritems():
         ctx = repo[node]
         branch = ctx.branch()
         if not branch in remotemap:
@@ -893,7 +894,11 @@ def checkheads(repo, remote, p_revs):
             if found:
                 continue
 
-            raise Exception("non-fast-forward")
+            node = repo.changelog.node(rev)
+            print "error %s non-fast forward" % p_revs[node]
+            ret = False
+
+    return ret
 
 def push_unsafe(repo, remote, parsed_refs, p_revs):
 
@@ -903,8 +908,8 @@ def push_unsafe(repo, remote, parsed_refs, p_revs):
     commoninc = fci(repo, remote, force=force)
     common, _, remoteheads = commoninc
 
-    if not force:
-        checkheads(repo, remote, p_revs)
+    if not force and not checkheads(repo, remote, p_revs):
+        return None
 
     cg = repo.getbundle('push', heads=list(p_revs), common=common)
 
@@ -940,7 +945,7 @@ def do_export(parser):
     global parsed_refs, bmarks, peer
 
     p_bmarks = []
-    p_revs = set()
+    p_revs = {}
 
     parser.next()
 
@@ -965,7 +970,7 @@ def do_export(parser):
             if branch in branches and bnode in branches[branch]:
                 # up to date
                 continue
-            p_revs.add(bnode)
+            p_revs[bnode] = ref
             print "ok %s" % ref
         elif ref.startswith('refs/heads/'):
             bmark = ref[len('refs/heads/'):]
@@ -980,7 +985,7 @@ def do_export(parser):
                     not (bmark == 'master' and bmark not in parser.repo._bookmarks):
                 p_bmarks.append((ref, bmark, old, new))
 
-            p_revs.add(bnode)
+            p_revs[bnode] = ref
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
             tag = hgref(tag)
@@ -988,20 +993,23 @@ def do_export(parser):
             if mode == 'git':
                 if not msg:
                     msg = 'Added tag %s for changeset %s' % (tag, node[:12]);
-                tagnode = write_tag(parser.repo, tag, node, msg, author)
-                p_revs.add(tagnode)
+                tagnode, branch = write_tag(parser.repo, tag, node, msg, author)
+                p_revs[tagnode] = 'refs/heads/branches/' + gitref(branch)
             else:
                 fp = parser.repo.opener('localtags', 'a')
                 fp.write('%s %s\n' % (node, tag))
                 fp.close()
-            p_revs.add(bnode)
+            p_revs[bnode] = ref
             print "ok %s" % ref
         else:
             # transport-helper/fast-export bugs
             continue
 
     if peer:
-        push(parser.repo, peer, parsed_refs, p_revs)
+        if not push(parser.repo, peer, parsed_refs, p_revs):
+            # do not update bookmarks
+            print
+            return
 
         # update remote bookmarks
         remote_bmarks = peer.listkeys('bookmarks')
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index c4d98b5..c4e4a1a 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -328,7 +328,7 @@ test_expect_success 'remote new bookmark' '
 	check_bookmark hgrepo feature-b feature-b
 '
 
-test_expect_failure 'remote push diverged' '
+test_expect_success 'remote push diverged' '
 	test_when_finished "rm -rf gitrepo*" &&
 
 	git clone "hg::hgrepo" gitrepo &&
@@ -351,7 +351,7 @@ test_expect_failure 'remote push diverged' '
 	check_branch hgrepo default bump
 '
 
-test_expect_failure 'remote update bookmark diverge' '
+test_expect_success 'remote update bookmark diverge' '
 	test_when_finished "rm -rf gitrepo*" &&
 
 	(
@@ -398,7 +398,7 @@ test_expect_failure 'remote new bookmark multiple branch head' '
 # cleanup previous stuff
 rm -rf hgrepo
 
-test_expect_failure 'remote big push' '
+test_expect_success 'remote big push' '
 	test_when_finished "rm -rf hgrepo gitrepo*" &&
 
 	(
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 36/48] remote-hg: check diverged bookmarks

[Felipe Contreras]

So that we can report a proper error.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 24 ++++++++++++++++++++++++
 contrib/remote-helpers/test-hg.sh    |  2 +-
 2 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index b6f85b2..e8ebc17 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -859,6 +859,23 @@ def write_tag(repo, tag, node, msg, author):
 
     return (tagnode, branch)
 
+def checkheads_bmark(repo, ref, ctx):
+    if force_push:
+        return True
+
+    bmark = ref[len('refs/heads/'):]
+    if not bmark in bmarks:
+        # new bmark
+        return True
+
+    ctx_old = bmarks[bmark]
+    ctx_new = ctx
+    if not repo.changelog.descendant(ctx_old.rev(), ctx_new.rev()):
+        print "error %s non-fast forward" % ref
+        return False
+
+    return True
+
 def checkheads(repo, remote, p_revs):
 
     remotemap = remote.branchmap()
@@ -875,6 +892,13 @@ def checkheads(repo, remote, p_revs):
         if not branch in remotemap:
             # new branch
             continue
+        if not ref.startswith('refs/heads/branches'):
+            if ref.startswith('refs/heads/'):
+                if not checkheads_bmark(repo, ref, ctx):
+                    ret = False
+
+            # only check branches
+            continue
         new.setdefault(branch, []).append(ctx.rev())
 
     for branch, heads in new.iteritems():
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index c4e4a1a..8ea5111 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -380,7 +380,7 @@ test_expect_success 'remote update bookmark diverge' '
 	check_bookmark hgrepo diverge "bump bookmark"
 '
 
-test_expect_failure 'remote new bookmark multiple branch head' '
+test_expect_success 'remote new bookmark multiple branch head' '
 	test_when_finished "rm -rf gitrepo*" &&
 
 	(
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 37/48] remote-hg: simplify branch_tip()

[Felipe Contreras]

It simply picks the last head that is not closed, but we have a stored
list of open heads.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index e8ebc17..8df72d9 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -560,12 +560,8 @@ def do_capabilities(parser):
 
     print
 
-def branch_tip(repo, branch):
-    # older versions of mercurial don't have this
-    if hasattr(repo, 'branchtip'):
-        return repo.branchtip(branch)
-    else:
-        return repo.branchtags()[branch]
+def branch_tip(branch):
+    return branches[branch][-1]
 
 def get_branch_tip(repo, branch):
     global branches
@@ -577,7 +573,7 @@ def get_branch_tip(repo, branch):
     # verify there's only one head
     if (len(heads) > 1):
         warn("Branch '%s' has more than one head, consider merging" % branch)
-        return branch_tip(repo, hgref(branch))
+        return branch_tip(hgref(branch))
 
     return heads[0]
 
@@ -828,7 +824,7 @@ def parse_tag(parser):
 
 def write_tag(repo, tag, node, msg, author):
     branch = repo[node].branch()
-    tip = branch_tip(repo, branch)
+    tip = branch_tip(branch)
     tip = repo[tip]
 
     def getfilectx(repo, memctx, f):
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 38/48] remote-hg: improve branch listing

[Felipe Contreras]

We want to show the remote heads, not the internal ones, which might
have garbage.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 8df72d9..5ad5c1e 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -605,9 +605,12 @@ def do_list(parser):
     list_head(repo, cur)
 
     if track_branches:
-        for branch in repo.branchmap():
-            heads = repo.branchheads(branch)
-            if len(heads):
+        orig = peer if peer else repo
+
+        for branch, heads in orig.branchmap().iteritems():
+            # only open heads
+            heads = [h for h in heads if 'close' not in repo.changelog.read(h)[5]]
+            if heads:
                 branches[branch] = heads
 
         for branch in branches:
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 39/48] remote-hg: use remote 'default' not local one

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 19 +++++++++----------
 contrib/remote-helpers/test-hg.sh    |  2 +-
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 5ad5c1e..a1b22f7 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -580,11 +580,11 @@ def get_branch_tip(repo, branch):
 def list_head(repo, cur):
     global g_head, bmarks, fake_bmark
 
-    if 'default' not in repo:
+    if 'default' not in branches:
         # empty repo
         return
 
-    node = repo['default']
+    node = repo[branch_tip('default')]
     head = 'master' if not 'master' in bmarks else 'default'
     fake_bmark = head
     bmarks[head] = node
@@ -601,18 +601,17 @@ def do_list(parser):
         bmarks[bmark] = repo[node]
 
     cur = repo.dirstate.branch()
+    orig = peer if peer else repo
+
+    for branch, heads in orig.branchmap().iteritems():
+        # only open heads
+        heads = [h for h in heads if 'close' not in repo.changelog.read(h)[5]]
+        if heads:
+            branches[branch] = heads
 
     list_head(repo, cur)
 
     if track_branches:
-        orig = peer if peer else repo
-
-        for branch, heads in orig.branchmap().iteritems():
-            # only open heads
-            heads = [h for h in heads if 'close' not in repo.changelog.read(h)[5]]
-            if heads:
-                branches[branch] = heads
-
         for branch in branches:
             print "? refs/heads/branches/%s" % gitref(branch)
 
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 8ea5111..84df1c3 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -486,7 +486,7 @@ test_expect_success 'remote big push' '
 	check_bookmark hgrepo new_bmark ''
 '
 
-test_expect_failure 'remote double failed push' '
+test_expect_success 'remote double failed push' '
 	test_when_finished "rm -rf hgrepo gitrepo*" &&
 
 	(
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 40/48] remote-hg: improve lightweight tag author

[Felipe Contreras]

Use git's committer.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index a1b22f7..fa76b3f 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -840,13 +840,23 @@ def write_tag(repo, tag, node, msg, author):
 
     p1 = tip.hex()
     p2 = '0' * 40
-    if not author:
-        author = (None, 0, 0)
-    user, date, tz = author
+    if author:
+        user, date, tz = author
+        date_tz = (date, tz)
+    else:
+        cmd = ['git', 'var', 'GIT_COMMITTER_IDENT']
+        process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
+        output, _ = process.communicate()
+        m = re.match('^.* <.*>', output)
+        if m:
+            user = m.group(0)
+        else:
+            user = repo.ui.username()
+        date_tz = None
 
     ctx = context.memctx(repo, (p1, p2), msg,
             ['.hgtags'], getfilectx,
-            user, (date, tz), {'branch' : branch})
+            user, date_tz, {'branch' : branch})
 
     tmp = encoding.encoding
     encoding.encoding = 'utf-8'
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 41/48] remote-hg: remove files before modifications

[Felipe Contreras]

Otherwise replacing a file with a directory doesn't work.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index fa76b3f..bd545aa 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -510,10 +510,10 @@ def export_ref(repo, name, kind, head):
             if len(parents) > 1:
                 print "merge :%s" % (rev_to_mark(parents[1]))
 
-        for f in modified_final:
-            print "M %s :%u %s" % f
         for f in removed:
             print "D %s" % (fix_file_path(f))
+        for f in modified_final:
+            print "M %s :%u %s" % f
         print
 
         progress = (rev - tip)
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 42/48] remote-hg: add setup_big_push() helper

[Felipe Contreras]

So we don't duplicate these commands.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 84df1c3..f594584 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -398,9 +398,7 @@ test_expect_success 'remote new bookmark multiple branch head' '
 # cleanup previous stuff
 rm -rf hgrepo
 
-test_expect_success 'remote big push' '
-	test_when_finished "rm -rf hgrepo gitrepo*" &&
-
+setup_big_push () {
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
@@ -459,8 +457,17 @@ test_expect_success 'remote big push' '
 
 	git checkout -q -b branches/new_branch master &&
 	echo ten > content &&
-	git commit -q -a -m ten &&
+	git commit -q -a -m ten
+	)
+}
 
+test_expect_success 'remote big push' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	setup_big_push
+
+	(
+	cd gitrepo &&
 	test_expect_code 1 git push origin master \
 		good_bmark bad_bmark1 bad_bmark2 new_bmark \
 		branches/good_branch branches/bad_branch \
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 43/48] remote-hg: add check_push() helper

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg.sh | 66 +++++++++++++++++++++++++++++----------
 1 file changed, 49 insertions(+), 17 deletions(-)

diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index f594584..aeab48a 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -48,6 +48,38 @@ check_bookmark () {
 	fi
 }
 
+check_push () {
+	local expected_ret=$1 ret=0 ref_ret=0 IFS=':'
+
+	shift
+	git push origin "$@" 2> error
+	ret=$?
+	cat error
+
+	while read branch kind
+	do
+		case "$kind" in
+		'new')
+			grep "^ \* \[new branch\] *${branch} -> ${branch}$" error || ref_ret=1
+			;;
+		'non-fast-forward')
+			grep "^ ! \[rejected\] *${branch} -> ${branch} (non-fast-forward)$" error || ref_ret=1
+			;;
+		'')
+			grep "^   [a-f0-9]*\.\.[a-f0-9]* *${branch} -> ${branch}$" error || ref_ret=1
+			;;
+		esac
+		let 'ref_ret' && echo "match for '$branch' failed" && break
+	done
+
+	if let 'expected_ret != ret || ref_ret'
+	then
+		return 1
+	fi
+
+	return 0
+}
+
 setup () {
 	(
 	echo "[ui]"
@@ -344,8 +376,9 @@ test_expect_success 'remote push diverged' '
 	cd gitrepo &&
 	echo diverge > content &&
 	git commit -a -m diverged &&
-	test_expect_code 1 git push 2> error &&
-	grep "^ ! \[rejected\] *master -> master (non-fast-forward)$" error
+	check_push 1 <<-EOF
+	master:non-fast-forward
+	EOF
 	) &&
 
 	check_branch hgrepo default bump
@@ -373,8 +406,9 @@ test_expect_success 'remote update bookmark diverge' '
 	git checkout --quiet diverge &&
 	echo diverge > content &&
 	git commit -a -m diverge &&
-	test_expect_code 1 git push 2> error &&
-	grep "^ ! \[rejected\] *diverge -> diverge (non-fast-forward)$" error
+	check_push 1 <<-EOF
+	diverge:non-fast-forward
+	EOF
 	) &&
 
 	check_bookmark hgrepo diverge "bump bookmark"
@@ -468,19 +502,17 @@ test_expect_success 'remote big push' '
 
 	(
 	cd gitrepo &&
-	test_expect_code 1 git push origin master \
-		good_bmark bad_bmark1 bad_bmark2 new_bmark \
-		branches/good_branch branches/bad_branch \
-		branches/new_branch 2> error &&
-
-	grep "^   [a-f0-9]*\.\.[a-f0-9]* *master -> master$" error &&
-	grep "^   [a-f0-9]*\.\.[a-f0-9]* *good_bmark -> good_bmark$" error &&
-	grep "^ \* \[new branch\] *new_bmark -> new_bmark$" error &&
-	grep "^ ! \[rejected\] *bad_bmark2 -> bad_bmark2 (non-fast-forward)$" error &&
-	grep "^ ! \[rejected\] *bad_bmark1 -> bad_bmark1 (non-fast-forward)$" error &&
-	grep "^   [a-f0-9]*\.\.[a-f0-9]* *branches/good_branch -> branches/good_branch$" error &&
-	grep "^ ! \[rejected\] *branches/bad_branch -> branches/bad_branch (non-fast-forward)$" error &&
-	grep "^ \* \[new branch\] *branches/new_branch -> branches/new_branch$" error
+
+	check_push 1 --all <<-EOF
+	master
+	good_bmark
+	branches/good_branch
+	new_bmark:new
+	branches/new_branch:new
+	bad_bmark1:non-fast-forward
+	bad_bmark2:non-fast-forward
+	branches/bad_branch:non-fast-forward
+	EOF
 	) &&
 
 	check_branch hgrepo default one &&
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 44/48] remote-helpers: improve marks usage

[Felipe Contreras]

Always convert to strings (they are unicode because they come from JSON).

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 5 ++++-
 contrib/remote-helpers/git-remote-hg  | 7 +++++--
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 10300c6..f0e5be1 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -116,7 +116,10 @@ class Marks:
         self.last_mark = mark
 
     def get_tip(self, branch):
-        return self.tips.get(branch, None)
+        try:
+            return str(self.tips[branch])
+        except KeyError:
+            return None
 
     def set_tip(self, branch, tip):
         self.tips[branch] = tip
diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index bd545aa..1e61f43 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -165,7 +165,7 @@ class Marks:
         return self.marks[rev]
 
     def to_rev(self, mark):
-        return self.rev_marks[mark]
+        return str(self.rev_marks[mark])
 
     def next_mark(self):
         self.last_mark += 1
@@ -185,7 +185,10 @@ class Marks:
         return rev in self.marks
 
     def get_tip(self, branch):
-        return self.tips.get(branch, None)
+        try:
+            return str(self.tips[branch])
+        except KeyError:
+            return None
 
     def set_tip(self, branch, tip):
         self.tips[branch] = tip
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 45/48] remote-hg: trivial cleanup

[Felipe Contreras]

It's better to catch the exception later on.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 1e61f43..b08f909 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -185,10 +185,7 @@ class Marks:
         return rev in self.marks
 
     def get_tip(self, branch):
-        try:
-            return str(self.tips[branch])
-        except KeyError:
-            return None
+        return str(self.tips[branch])
 
     def set_tip(self, branch, tip):
         self.tips[branch] = tip
@@ -433,10 +430,10 @@ def export_ref(repo, name, kind, head):
     global prefix, marks, mode
 
     ename = '%s/%s' % (kind, name)
-    tip = marks.get_tip(ename)
-    if tip and tip in repo:
+    try:
+        tip = marks.get_tip(ename)
         tip = repo[tip].rev()
-    else:
+    except:
         tip = 0
 
     revs = xrange(tip, head.rev() + 1)
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 46/48] remote-hg: check if a fetch is needed

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 39 ++++++++++++++++++++
 contrib/remote-helpers/test-hg.sh    | 71 +++++++++++++++++++++++++++++++++++-
 2 files changed, 109 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index b08f909..b983627 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -973,6 +973,15 @@ def push(repo, remote, parsed_refs, p_revs):
 
     return ret
 
+def check_tip(ref, kind, name, heads):
+    try:
+        ename = '%s/%s' % (kind, name)
+        tip = marks.get_tip(ename)
+    except KeyError:
+        return True
+    else:
+        return tip in heads
+
 def do_export(parser):
     global parsed_refs, bmarks, peer
 
@@ -995,6 +1004,8 @@ def do_export(parser):
         else:
             die('unhandled export command: %s' % line)
 
+    need_fetch = False
+
     for ref, node in parsed_refs.iteritems():
         bnode = hgbin(node)
         if ref.startswith('refs/heads/branches'):
@@ -1002,6 +1013,16 @@ def do_export(parser):
             if branch in branches and bnode in branches[branch]:
                 # up to date
                 continue
+
+            if peer:
+                remotemap = peer.branchmap()
+                if remotemap and branch in remotemap:
+                    heads = [hghex(e) for e in remotemap[branch]]
+                    if not check_tip(ref, 'branches', branch, heads):
+                        print "error %s fetch first" % ref
+                        need_fetch = True
+                        continue
+
             p_revs[bnode] = ref
             print "ok %s" % ref
         elif ref.startswith('refs/heads/'):
@@ -1017,6 +1038,14 @@ def do_export(parser):
                     not (bmark == 'master' and bmark not in parser.repo._bookmarks):
                 p_bmarks.append((ref, bmark, old, new))
 
+            if peer:
+                remote_old = peer.listkeys('bookmarks').get(bmark)
+                if remote_old:
+                    if not check_tip(ref, 'bookmarks', bmark, remote_old):
+                        print "error %s fetch first" % ref
+                        need_fetch = True
+                        continue
+
             p_revs[bnode] = ref
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
@@ -1037,6 +1066,16 @@ def do_export(parser):
             # transport-helper/fast-export bugs
             continue
 
+    if need_fetch:
+        print
+        return
+
+    if dry_run:
+        if peer and not force_push:
+            checkheads(parser.repo, peer, p_revs)
+        print
+        return
+
     if peer:
         if not push(parser.repo, peer, parsed_refs, p_revs):
             # do not update bookmarks
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index aeab48a..91ddac7 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -65,6 +65,9 @@ check_push () {
 		'non-fast-forward')
 			grep "^ ! \[rejected\] *${branch} -> ${branch} (non-fast-forward)$" error || ref_ret=1
 			;;
+		'fetch-first')
+			grep "^ ! \[rejected\] *${branch} -> ${branch} (fetch first)$" error || ref_ret=1
+			;;
 		'')
 			grep "^   [a-f0-9]*\.\.[a-f0-9]* *${branch} -> ${branch}$" error || ref_ret=1
 			;;
@@ -407,7 +410,7 @@ test_expect_success 'remote update bookmark diverge' '
 	echo diverge > content &&
 	git commit -a -m diverge &&
 	check_push 1 <<-EOF
-	diverge:non-fast-forward
+	diverge:fetch-first
 	EOF
 	) &&
 
@@ -525,6 +528,72 @@ test_expect_success 'remote big push' '
 	check_bookmark hgrepo new_bmark ''
 '
 
+test_expect_success 'remote big push fetch first' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	(
+	hg init hgrepo &&
+	cd hgrepo &&
+	echo zero > content &&
+	hg add content &&
+	hg commit -m zero &&
+	hg bookmark bad_bmark &&
+	hg bookmark good_bmark &&
+	hg bookmark -i good_bmark &&
+	hg -q branch good_branch &&
+	echo "good branch" > content &&
+	hg commit -m "good branch" &&
+	hg -q branch bad_branch &&
+	echo "bad branch" > content &&
+	hg commit -m "bad branch"
+	) &&
+
+	git clone "hg::hgrepo" gitrepo &&
+
+	(
+	cd hgrepo &&
+	hg bookmark -f bad_bmark &&
+	echo update_bmark > content &&
+	hg commit -m "update bmark"
+	) &&
+
+	(
+	cd gitrepo &&
+	echo two > content &&
+	git commit -q -a -m two &&
+
+	git checkout -q good_bmark &&
+	echo three > content &&
+	git commit -q -a -m three &&
+
+	git checkout -q bad_bmark &&
+	echo four > content &&
+	git commit -q -a -m four &&
+
+	git checkout -q branches/bad_branch &&
+	echo five > content &&
+	git commit -q -a -m five &&
+
+	check_push 1 --all <<-EOF
+	master
+	good_bmark
+	new_bmark:new
+	new_branch:new
+	bad_bmark:fetch-first
+	branches/bad_branch:festch-first
+	EOF
+
+	git fetch &&
+
+	check_push 1 --all <<-EOF
+	master
+	good_bmark
+	bad_bmark:non-fast-forward
+	branches/bad_branch:non-fast-forward
+	EOF
+	)
+'
+
 test_expect_success 'remote double failed push' '
 	test_when_finished "rm -rf hgrepo gitrepo*" &&
 
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 47/48] remote-hg: add support for --dry-run

[Felipe Contreras]

This needs a specific patch from Git not applied yet.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 29 +++++++++++++++++++++++++--
 contrib/remote-helpers/test-hg.sh    | 38 ++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index b983627..20abb34 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -557,6 +557,7 @@ def do_capabilities(parser):
     if os.path.exists(path):
         print "*import-marks %s" % path
     print "*export-marks %s" % path
+    print "option"
 
     print
 
@@ -724,6 +725,11 @@ def parse_commit(parser):
             die('Unknown file command: %s' % line)
         files[path] = f
 
+    # only export the commits if we are on an internal proxy repo
+    if dry_run and not peer:
+        parsed_refs[ref] = None
+        return
+
     def getfilectx(repo, memctx, f):
         of = files[f]
         if 'deleted' in of:
@@ -809,7 +815,10 @@ def parse_reset(parser):
     from_mark = parser.get_mark()
     parser.next()
 
-    rev = mark_to_rev(from_mark)
+    try:
+        rev = mark_to_rev(from_mark)
+    except KeyError:
+        rev = None
     parsed_refs[ref] = rev
 
 def parse_tag(parser):
@@ -1007,7 +1016,7 @@ def do_export(parser):
     need_fetch = False
 
     for ref, node in parsed_refs.iteritems():
-        bnode = hgbin(node)
+        bnode = hgbin(node) if node else None
         if ref.startswith('refs/heads/branches'):
             branch = ref[len('refs/heads/branches/'):]
             if branch in branches and bnode in branches[branch]:
@@ -1048,6 +1057,9 @@ def do_export(parser):
 
             p_revs[bnode] = ref
         elif ref.startswith('refs/tags/'):
+            if dry_run:
+                print "ok %s" % ref
+                continue
             tag = ref[len('refs/tags/'):]
             tag = hgref(tag)
             author, msg = parsed_tags.get(tag, (None, None))
@@ -1097,6 +1109,15 @@ def do_export(parser):
 
     print
 
+def do_option(parser):
+    global dry_run
+    _, key, value = parser.line.split(' ')
+    if key == 'dry-run':
+        dry_run = (value == 'true')
+        print 'ok'
+    else:
+        print 'unsupported'
+
 def fix_path(alias, repo, orig_url):
     url = urlparse.urlparse(orig_url, 'file')
     if url.scheme != 'file' or os.path.isabs(url.path):
@@ -1113,6 +1134,7 @@ def main(args):
     global parsed_tags
     global filenodes
     global fake_bmark, hg_version
+    global dry_run
 
     alias = args[1]
     url = args[2]
@@ -1151,6 +1173,7 @@ def main(args):
         hg_version = tuple(int(e) for e in util.version().split('.'))
     except:
         hg_version = None
+    dry_run = False
 
     repo = get_repo(url, alias)
     prefix = 'refs/hg/%s' % alias
@@ -1175,6 +1198,8 @@ def main(args):
             do_import(parser)
         elif parser.check('export'):
             do_export(parser)
+        elif parser.check('option'):
+            do_option(parser)
         else:
             die('unhandled command: %s' % line)
         sys.stdout.flush()
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index 91ddac7..bf3635e 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -594,6 +594,44 @@ test_expect_success 'remote big push fetch first' '
 	)
 '
 
+test_expect_failure 'remote big push dry-run' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	setup_big_push
+
+	(
+	cd gitrepo &&
+
+	check_push 0 --dry-run --all <<-EOF
+	master
+	good_bmark
+	branches/good_branch
+	new_bmark:new
+	branches/new_branch:new
+	bad_bmark1:non-fast-forward
+	bad_bmark2:non-fast-forward
+	branches/bad_branch:non-fast-forward
+	EOF
+
+	check_push 0 --dry-run master good_bmark new_bmark branches/good_branch branches/new_branch <<-EOF
+	master
+	good_bmark
+	branches/good_branch
+	new_bmark:new
+	branches/new_branch:new
+	EOF
+	) &&
+
+	check_branch hgrepo default one &&
+	check_branch hgrepo good_branch "good branch" &&
+	check_branch hgrepo bad_branch "bad branch" &&
+	check_branch hgrepo new_branch '' &&
+	check_bookmark hgrepo good_bmark one &&
+	check_bookmark hgrepo bad_bmark1 one &&
+	check_bookmark hgrepo bad_bmark2 one &&
+	check_bookmark hgrepo new_bmark ''
+'
+
 test_expect_success 'remote double failed push' '
 	test_when_finished "rm -rf hgrepo gitrepo*" &&
 
-- 
1.8.3.rc3.312.g47657de

[PATCH v2 48/48] remote-hg: add support for --force

[Felipe Contreras]

And get rid of the remote-hg.force-push option hack.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 30 +++++++++++++++++-------------
 contrib/remote-helpers/test-hg.sh    | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 13 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 20abb34..0194c67 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -29,9 +29,6 @@ import urlparse, hashlib
 # named branches:
 # git config --global remote-hg.track-branches false
 #
-# If you don't want to force pushes (and thus risk creating new remote heads):
-# git config --global remote-hg.force-push false
-#
 # If you want the equivalent of hg's clone/pull--insecure option:
 # git config --global remote-hg.insecure true
 #
@@ -877,9 +874,6 @@ def write_tag(repo, tag, node, msg, author):
     return (tagnode, branch)
 
 def checkheads_bmark(repo, ref, ctx):
-    if force_push:
-        return True
-
     bmark = ref[len('refs/heads/'):]
     if not bmark in bmarks:
         # new bmark
@@ -888,8 +882,11 @@ def checkheads_bmark(repo, ref, ctx):
     ctx_old = bmarks[bmark]
     ctx_new = ctx
     if not repo.changelog.descendant(ctx_old.rev(), ctx_new.rev()):
-        print "error %s non-fast forward" % ref
-        return False
+        if force_push:
+            print "ok %s forced update" % ref
+        else:
+            print "error %s non-fast forward" % ref
+            return False
 
     return True
 
@@ -936,8 +933,12 @@ def checkheads(repo, remote, p_revs):
                 continue
 
             node = repo.changelog.node(rev)
-            print "error %s non-fast forward" % p_revs[node]
-            ret = False
+            ref = p_revs[node]
+            if force_push:
+                print "ok %s forced update" % ref
+            else:
+                print "error %s non-fast forward" % ref
+                ret = False
 
     return ret
 
@@ -949,7 +950,7 @@ def push_unsafe(repo, remote, parsed_refs, p_revs):
     commoninc = fci(repo, remote, force=force)
     common, _, remoteheads = commoninc
 
-    if not force and not checkheads(repo, remote, p_revs):
+    if not checkheads(repo, remote, p_revs):
         return None
 
     cg = repo.getbundle('push', heads=list(p_revs), common=common)
@@ -1110,11 +1111,14 @@ def do_export(parser):
     print
 
 def do_option(parser):
-    global dry_run
+    global dry_run, force_push
     _, key, value = parser.line.split(' ')
     if key == 'dry-run':
         dry_run = (value == 'true')
         print 'ok'
+    elif key == 'force':
+        force_push = (value == 'true')
+        print 'ok'
     else:
         print 'unsupported'
 
@@ -1142,7 +1146,7 @@ def main(args):
 
     hg_git_compat = get_config_bool('remote-hg.hg-git-compat')
     track_branches = get_config_bool('remote-hg.track-branches', True)
-    force_push = get_config_bool('remote-hg.force-push')
+    force_push = False
 
     if hg_git_compat:
         mode = 'hg'
diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh
index bf3635e..30f4ff6 100755
--- a/contrib/remote-helpers/test-hg.sh
+++ b/contrib/remote-helpers/test-hg.sh
@@ -68,6 +68,9 @@ check_push () {
 		'fetch-first')
 			grep "^ ! \[rejected\] *${branch} -> ${branch} (fetch first)$" error || ref_ret=1
 			;;
+		'forced-update')
+			grep "^ + [a-f0-9]*\.\.\.[a-f0-9]* *${branch} -> ${branch} (forced update)$" error || ref_ret=1
+			;;
 		'')
 			grep "^   [a-f0-9]*\.\.[a-f0-9]* *${branch} -> ${branch}$" error || ref_ret=1
 			;;
@@ -594,6 +597,36 @@ test_expect_success 'remote big push fetch first' '
 	)
 '
 
+test_expect_failure 'remote big push force' '
+	test_when_finished "rm -rf hgrepo gitrepo*" &&
+
+	setup_big_push
+
+	(
+	cd gitrepo &&
+
+	check_push 0 --force --all <<-EOF
+	master
+	good_bmark
+	branches/good_branch
+	new_bmark:new
+	branches/new_branch:new
+	bad_bmark1:forced-update
+	bad_bmark2:forced-update
+	branches/bad_branch:forced-update
+	EOF
+	) &&
+
+	check_branch hgrepo default six &&
+	check_branch hgrepo good_branch eight &&
+	check_branch hgrepo bad_branch nine &&
+	check_branch hgrepo new_branch ten &&
+	check_bookmark hgrepo good_bmark three &&
+	check_bookmark hgrepo bad_bmark1 four &&
+	check_bookmark hgrepo bad_bmark2 five &&
+	check_bookmark hgrepo new_bmark six
+'
+
 test_expect_failure 'remote big push dry-run' '
 	test_when_finished "rm -rf hgrepo gitrepo*" &&
 
-- 
1.8.3.rc3.312.g47657de

Re: [PATCH v2 27/48] remote-hg: add test for new bookmark special

[Eric Sunshine]

On Fri, May 24, 2013 at 10:29 PM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> From the point of view of Mercurial, this creates a new branch head, and
> requires a forced push.
>
> Ideally, whoever, we would want it to work just like in git; new

s/whoever/however/

> branches can be pushed without problems.
>
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>

Re: [PATCH v2 25/48] remote-hg: add test for diverged push

[Eric Sunshine]

On Fri, May 24, 2013 at 10:29 PM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> Neither mercurial nor git allows pushing to a remote when it's a
> non-fast-forward push. We should be able to detect theses errors and

s/theses/these/

> report them properly.
>
> As opposed to throwing an exception stack-trace.
>
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>